From acd@roe.ac.uk Tue Dec 17 15:16:24 2013 Date: Tue, 17 Dec 2013 15:11:01 +0000 From: Clive Davenhall To: rgm@roe.ac.uk, al@roe.ac.uk, mar@roe.ac.uk, etws@roe.ac.uk, msh@roe.ac.uk, njc@roe.ac.uk, nch@roe.ac.uk, rpb@roe.ac.uk, rsc@roe.ac.uk, stv@roe.ac.uk, dmr@roe.ac.uk, acd@roe.ac.uk Cc: dante@astro.puc.cl, jsd@roe.ac.uk, j.p.emerson@qmul.ac.uk, keith@keithnoddle.org, Lorenzo.Rimoldini@unige.ch, mcioni@aip.de, M.J.Jarvis@herts.ac.uk, mike@ast.cam.ac.uk, nigel.metcalfe@durham.ac.uk, norman@astro.gla.ac.uk, pwl@star.herts.ac.uk, rgm@ast.cam.ac.uk, Simon.Dye@nottingham.ac.uk, s.j.warren@ic.ac.uk, tkerr@jach.hawaii.edu, tom.shanks@durham.ac.uk Subject: WFAU VDFS Science Archives progress meeting minutes, 13 December 2013 WFAU VDFS Science Archives progress meeting minutes, 13 December 2013 Present: NJC, RGM, ETWS, MAR, RPB, RSC, ACD Apologies: NCH, MSH, KTN, AL DoNM: 10am Friday 10 January 2014 in the Villas meeting room MAR in the chair (room is booked). Actions discharged this time: ---------------------------- None. Actions partly discharged but continuing: ---------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of DR10 to see if it's worth us hosting and whether it's necessary to keep all of the other releases. Continues. RGM had started a wiki page for WFAU SDSS strategy. However, it needs more work and in particular RGM still had to determine which WFAU archives had cross-matches with which SDSS releases. Actions carried forward from 29/11/2013 meeting: ---------------------------------------------- ACTION: RPB to do the work necessary to make the pending VISTA public releases. Continues. VIKING has been done (DR3), VIDEO had fallen over and needs to be re-run. However, there had been no further progress. ACTION: ALL to add/maintain entries on http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Continues. ACD noted that early in the New Year he would be checking that all important forthcoming deadlines were listed. ACTION: MAR to investigate and quantify the issue with unmatched ATLAS sources. And if the issue warrants, RSC to look into the curation code with a view to pairing up the unmatched sources. Continues: MAR had investigated. He had reported his findings to Nigel Metcalfe and Tom Shanks and is still waiting for a response from them before proceeding further. MAR would e-mail them a reminder during the week beginning Mon. 16/12/13. ACTION: ACD, RPB to formulate a policy for backups and for checking that data could be recovered from the backups. Continues. ACTION: ACD to discuss with RGM the available budget for hardware purchases. Continues. Money was available, but the precise amount still has be definitively determined and looks likely to be less than hoped. Nonetheless the hardware needed urgently had been ordered. ACTION: NJC to sort out the magnitude limits for V1.3 data. Continues. The software to compute these limits appears to be working but NJC wished to carry out additional tests before bringing it into production use. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. ACTION: ALL; when the new OSA interface is ready, have a play with accessing the archive to check that there are no obvious problems. Continues: several people had contributed to the checking, as a result of which several fixes and improvements had been made. However, there was scope for further testing and NJC hoped to contribute further. Specific points and new actions: ------------------------------- Project management: * RGM noted that he needed to e-mail Colin Vincent about an extension to the WFAU rolling grant. ACTION: RGM to e-mail Colin Vincent about an extension to the WFAU rolling grant. WFCAM, VISTA and VST updates: * RGM noted that from 1 January 2014 UKIRT would have new owners but that there had not yet been a public announcement about who these new owners would be. Nonetheless, he reported that it was likely that during January and February 2014 UKIRT would be carrying out fairly standard WFCAM operations as the new owners familiarised themselves with the telescope. WFAU (and CASU) staff involved in WFCAM operations should expect to process a number of UDS-like programmes from data acquired during this period and to generate several non-survey releases from them. More details of the observing strategy that would be employed would be useful. ACTION: RGM would attempt to elucidate further details of the observing strategy to be followed. Comments and issues arising from CASU minutes: * No CASU minutes had been received since the last meeting. Networking: * The networks are performing optimally and are being used heavily to return copies of the processed data to Cambridge to replace those deleted by a hacker. WSA/VSA/OSA/GES Operations: * ETWS noted that he had he had finished the WFCAM metadata ingest and JPGs of the August and September data. * RPB report on VISTA surveys: - A new VIKING release was being produced. The deep products were finished but still to be ingested. - VVV for mid-2012 were being ingested. - The VVV DR2 release was still being outgested. * ETWS noted that the OSA catalogue ingest was complete. * ACD reported that there had not been much activity on GES but that he was working on a tutorial of example SQL queries. ACTION: ACD check progress against the agreed time-line. Hardware and Systems: * RPB reported that he and MSH had upgraded SQL Server from the 2008 to the 2012 versions on some systems. He noted that all the systems that he had done had upgraded without a hitch but that MSH had encountered problems. The current state of play was that: RAMESES4, 6, 7, 10, 11, 13 had all been upgraded. These machines include most of the publicly accessible servers. There was a problem with RAMESES8 (which hosts the public version of SDSS). The following machines have still be updated: RAMESES3, 5, 1 and later 12, 14 Most would be done by Christmas. * ETWS reported that MSH has upgraded debian on menkaure and shepseskaf and ETWS has installed the third party software. RSC noted that SAMBA was now running slowly on these machines and consequently there was sometimes a delay between files being created on the Linux systems and appearing in the Windows file tree. Survey Data Release: * NJC noted that he was working on a VIDEO release for ESO. He was discussing some details with the Consortium lead, Matt Jarvis. * MAR reported that the new UI / Web site to accompany the ATLAS DR1 was almost ready. He hoped to check with Nigel Metcalfe and Tom Shanks on Mon. 16/12/13 that they were happy for it to be released on Wed. 18/12/13. Software: * NJC reported that he had modified or developed code for the VIDEO ESO release and the magnitude limits for DR3. He was developing improved metrics for the stacking and tiling, but this was still in progress. * RSC reported he had made some minor tweaks to CU19. He also noted that a meeting had been held on 12/13/13 to brainstorm possible ways of speeding up the outgesting in preparation for VVV DR3. A wiki page had been set up in which the results of investigations into various possible mechanisms for affecting speed improvements would be recorded. * RPB noted that he expected VVV DR2 to complete early in the New Year. Non-survey Data Release: * There had been a request for a WFCAM non-survey re-release from Roy Gal (University of Hawaii). Deep stacks made for the previous release had been lost due to the problems with disk38. A re-release was planned but was a little delayed because of the recent upgrading of SQL Server installations. * It was also noted that RAMESES5 and 6 were full; more disk space was needed before any more non-survey releases could be made. Astrogrid deployment, VO & Data Analysis services: * MAR noted that Enrique Solano (Universidad del Paıs Vascoa) had requested a SIAP (Simple Image Access Protocol) service for ATLAS DR1. Miscellaneous: * A Gaia launch breakfast will be held in the Stobie room from 8:00am on Thr. 19/12/13. * Maciej Bilicki has a new catalogue with a few sources removed, a new column added and some more descriptive text. * The meeting noted its appreciation for the mince pies provided by ACD. ACD, draft: 13/12/13, completed: 17/12/13. -- Scanned by iCritical.