From mar@roe.ac.uk Tue Oct 27 16:14:24 2015 Date: Tue, 27 Oct 2015 16:14:18 +0000 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 16 October 2015 WFAU VDFS Science Archives progress meeting minutes, 16 October 2015 Present: RGM, NJC, MAR, ACD, MSH Apologies: RSC, ETWS, STV, KTN, AL, RPB DoNM: 10am Friday 30 October 2015 in the Villas meeting room, ACD in the chair (room is booked). Actions discharged this time: ---------------------------- None Actions carried forward from 25/9/2015 meeting: --------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this action still has low priority. ACTION: NJC noted that VISTA mosaicing crashes with normal filtering, which he should fix. Continues. NJC had implemented a fix and checked it into SVN. However, when it was run 'in anger' it still crashed. NJC to investigate. ACTION: NJC noted that he had received a report of a problem with the ESO VVV DR1 release which he should investigate. In one pointing the merged band catalogue contained fewer sources than expected. Continues. This action was low on NJC's priority list. Actions resulting from the VVV DR3 brainstorming meeting held on 24/2/15: ------------------------------------------------------------------------ ACTION: RPB to experiment with ingest via partitioned tables, along the lines of the SSIS paper. Deferred (changed back from 'continues'). This action has now been overtaken by events: the VVV DR3 processing would now be done differently. However, the technique was still potentially useful for future work and should be investigated, but not as a matter of urgency. ACTION: RGM (was RSC, RGM): (i) RSC to write a succinct summary of the ingest and outgest problem that is the principal bottleneck with the VVV processing (briefly the ingest and outgest speed is limited to approx. 100 Mbit/sec for no apparent reason) and (ii) RGM to compose an e-mail message around this summary and send it to Ani Thakar. Continues. RSC had composed a succinct summary and sent it to RGM. RGM had emailed Ani Thakar but no reply to date. RGM had sent a reminder but had still received no response. Specific points and new actions: -------------------------------- Project management: * RGM reported that the grant application had been submitted. The review panel would likely meet around Christmas with a decision made in January 2016. * NJC noted that he'd set up a wiki page on proposed new surveys/extensions. WFCAM, VISTA and VST updates: * Nothing of note. Networking: * Nothing to report; the networks were performing nominally. WSA/VSA/OSA/GES Operations: * VVV was currently outgesting best match tables. * ACD reported work on progressing GES iDR4. RSC would shortly be taking over ACD's GES work. Hardware and Systems: * MSH reported that works in C2 would lead to multiple shutdowns of systems over the next couple of months. The first planned outage was 30/31 October. ACTION: RPB to ensure we have recent backups of relevant databases especially VVVDR3/4. * MSH also noted that we were getting some spare disks from Eclipse in exchange for some old NAS boxes. Software: * NJC reported that he had: - modified the ESO release code to pick up ATLAS tile centres from the FITS files as the database values were currently stored as single precision which apparently is an issue for ESO. - merged the list driven code into trunk. - worked on automating/standardizing source tables. * ACD reported that he had been re-writing the GES ingest script. Review of Deadlines: ACD led the meeting through the various upcoming deadlines http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * None Non-survey Data Release: * None Astrogrid deployment, VO & Data Analysis services: * Nothing of note Miscellaneous: * NJC noted that ESO/ESA were holding a workshop in Garching 24-27th November "Science Operations 2015: Science Data Management - An ESO/ESA Workshop" Although this clashed with our Quarterly meeting it was felt that a representative should be sent. ACTION: NJC to register a talk at the ESO/ESA workshop.