From mar@roe.ac.uk Mon Nov 21 16:12:07 2016 Date: Mon, 21 Nov 2016 16:12:02 +0000 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 11 November 2016 WFAU VDFS Science Archives progress meeting minutes, 11 November 2016 Present: NJC, RPB, ETWS, MAR, MSH, ACD, RGM, RSC Apologies: STV, DMR, KTN, AL DoNM: 10:00 am Friday 25th November 2016, ** CRAWFORD MEETING ROOM 2 ** ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: ACD to arrange a meeting between DMR, STV and MSH during DMR and STV's next visit to agree (i) a list of enhancements to the TAP server necessary to have a deployable service and (ii) a deadline (or set of deadlines) by which these enhancements will be complete. The meeting would take place next week. Actions carried forward from 28/10/2016 meeting: --------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan DR13 is installed but not yet cross-mounted. However, we still need to decide which SDSS releases to keep so the present action should remain. ACTION: MAR to write some H-ATLAS documentation once H-ATLAS is available. Continues. ACTION: RPB to check the log files and inform NJC of the problems with the previously listed non-survey releases. Continues. ACTION: NJC to investigate and fix the problems with the previously listed non-survey releases. Continues. This action cannot progress until RPB checks the log files and passes on details of the problems (action above). ACTION: MSH to check with DMR about progress towards, and anticipated release date of, a usable TAP service. Continues, (meeting next week) ACTION: RPB to check with CASU whether they had any WFCAM observations made during the gap in the Edinburgh tapes. It was suggested we might like to check here first! Continues. ACTION: NJC to complete the deep-stacking software for VVV. Continues. ACTION NJC: investigate and fix the problem with Wserv8. NJC had contacted CASU to see if they could spot anything in the imcore code that was causing the NaNs as the inputs looked OK. ACTION MAR: reply to DMR's e-mail about publishing statistics for columns. Continues. Specific points and new actions: -------------------------------- Project management: * RGM noted that the WFAU quarterl would be held Monday 19th December. Annual reviews would take place prior to the meeting and the staffing spreadsheet updated. WFCAM, VISTA and VST updates: * Nothing to report. Networking: * Nothing to report; the networks were operating nominally. WSA/VSA/OSA/GES Operations: * ETWS reported that he had: - The latest VISTA data (P97 up to June) has been transferred. - Ingest of the v1.3.1 data into the VSA has finished, while ingests into the VVV are still ongoing. - OSA VPHAS processing is still going. - OSA catalogue ingests for P97 data (April to June) are progressing good. - Created web pages for the latest ATLASDR3 release. Hardware and Systems: * RPB reported that ramses7 had suffered a catastrophic failure when its RAID controller had apparently written over the config and the disk volumes had lost. NJC was in the process of re-creating the TWOMPZ and WISExSCOS databases but several early versions of SDSS had been lost. Database backups to disk were currently in progress and tape backups would soon be resumed on the new Linux/AMANDA set-up. ACTION: ETWS to ask for and re-transfer the missing SDSS databases. MAR asked if someone could look at disk44 as during a recent large batch of image cut-outs, this disk seemed 10-20 times slower than others. ACTION: MSH to investigate speed of disk44 Review of Software Development Deadlines: * The list of software deadlines was briefly discussed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft ACD and NJC had already met offline and gone through the bulk of the items. Other Software: * ETWS noted: - Wrote a script to fix the uninitialised CROWDED flag that prevented metadata ingests. NJC reported progress on the VIKING list driven code and ESO VMC products. MAR noted running a large batch of ATLAS cut-outs for a user and supply Maria-Ida with v1.3.1 tile mags for her sample. RGM mentioned that MS SQL Server on Linux was available and that initial tests carried out by others looked promising ACTION: ACD to set-up a WIKI page on MS SQL Server on Linux and all to add their input and what and how it needs to be tested for our uses. Review of Survey Release Deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * ATLAS DR3 was released/made public. Non-survey Data Release: * None. Astrogrid deployment, VO & Data Analysis services: * MSH noted that he'd set-up the DSA for ATLAS DR3. Miscellaneous: RPB would set-up up a poll re WFAU's Christmas lunch,