From acd@roe.ac.uk Sun Nov 8 17:46:52 2015 Date: Sun, 8 Nov 2015 17:46:43 +0000 From: Clive Davenhall To: Bob Mann , Andy Lawrence , "Read, Mike" , Eckhard Sutorius , Mark S. Holliman , Nicholas Cross , Nigel C. Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris , Clive Davenhall Cc: "dante@astro.puc.cl" , James Dunlop , "j.p.emerson@qmul.ac.uk" , ktn keithnoddle.org , "Lorenzo.Rimoldini@unige.ch" , "mcioni@aip.de" , "matt.jarvis@astro.ox.ac.uk" , "mike@ast.cam.ac.uk" , "nigel.metcalfe@durham.ac.uk" , "norman@astro.gla.ac.uk" , "pwl@star.herts.ac.uk" , "rgm@ast.cam.ac.uk" , "Simon.Dye@nottingham.ac.uk" , "s.j.warren@ic.ac.uk" , "tkerr@jach.hawaii.edu" , "tom.shanks@durham.ac.uk" Subject: WFAU VDFS Science Archives progress meeting minutes, 30 October 2015 WFAU VDFS Science Archives progress meeting minutes, 30 October 2015 Present: MSH, RGM, STV, ETWS, MAR, RPB, DMR, ACD Apologies: NJC, RSC, KTN, AL DoNM: 10am Friday 13 November 2015 in the Villas meeting room, MAR in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: RPB to ensure we have recent backups of relevant databases especially VVVDR3/4. Done. VVV DR3 was now fully backed-up. ACTION: NJC to register a talk at the ESO/ESA workshop. Done. Actions carried forward from 16/10/2015 meeting: ----------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this action still has low priority. ACTION: NJC noted that VISTA mosaicing crashes with normal filtering, which he should fix. Continues. NJC had implemented a fix and checked it into SVN. However, when it was run 'in anger' it still crashed. NJC to investigate. ACTION: NJC noted that he had received a report of a problem with the ESO VVV DR1 release which he should investigate. In one pointing the merged band catalogue contained fewer sources than expected. Continues. This action was low on NJC's priority list. Actions resulting from the VVV DR3 brainstorming meeting held on 24/2/15: ------------------------------------------------------------------------ ACTION: RPB to experiment with ingest via partitioned tables, along the lines of the SSIS paper. Deferred (changed back from 'continues'). This action has now been overtaken by events: the VVV DR3 processing would now be done differently. However, the technique was still potentially useful for future work and should be investigated, but not as a matter of urgency. ACTION: RGM (was RSC, RGM): (i) RSC to write a succinct summary of the ingest and outgest problem that is the principal bottleneck with the VVV processing (briefly the ingest and outgest speed is limited to approx. 100 Mbit/sec for no apparent reason) and (ii) RGM to compose an e-mail message around this summary and send it to Ani Thakar. Continues. RSC had composed a succinct summary and sent it to RGM. RGM had emailed Ani Thakar but no reply to date. RGM had sent a reminder but had still received no response. RGM was planning to contact Alex Szalay on another matter and would take the opportunity to raise the issue with him. Specific points and new actions: -------------------------------- Project management: * Nothing to report. WFCAM, VISTA and VST updates: * An announcement has been made that UKIRT will close in 2023 in order to permit the construction of new telescopes on the Mauna kea summit. Networking: * The networks had been performing nominally. A few transient glitches had been observed, but nothing serious. * MSH noted that all networks would be down on 31/10/15 (the day after the meeting) to facilitate modification of the server room. They should be restored by the end of the day. WSA/VSA/OSA/GES Operations: * ETWS had transferred all the VSA, WSA and OSA data for the past couple of months. He had also nearly finished ingesting the re-transferred data for the next VST/ATLAS release, but had to interrupt the ingest due to the OSA database being moved to a new server. * RPB had moved the the OSA databases to a larger disk (from RAMSES2 to RAMSES12). In addition to facilitating continuing OSA work this change would also free-up additional space for use by DMR and STV. * RPB had also started processing VMC P94, but it was running slowly and he needed to investigate. Hardware and Systems: * MSH reported that: - Eclipse computing would be visiting later in the day of the meeting to bring some new disks as part of a trade-in for old NAS boxes. - The networks would be down on 31/11/15. - The main shutdown occasioned by the refurbishment of the computer room would occur on 20 November. Software: * MAR noted that there were problems with the latest ATLAS release; he had put in a quick hack but it had not worked. * ETWS had made a number of minor updates. * ACD reported that he appeared to have the basics of ingesting the iDR4 parameter files working. There remained, however, problems caused by infelicities in some of the FITS files of parameters, but he did not expect these to be serious. Review of Deadlines: * ACD led the meeting through the various upcoming deadlines: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * VVV DR4 (which is DR3 rebadged for consistency with ESO) had been released to the VVV Team for checking. Non-survey Data Release: * None. Astrogrid deployment, VO & Data Analysis services: * DMR reported that good progress was being made implementing a new public TAP service. He noted that a good public administrative interface had still to be defined. There was some discussion of the UCD served by the WFAU TAP service in the light of the recent announcement by the CDS of a service for translating UCD 1 values in to UCD 1+. WFAU already serves UCD 1+ values but the translation from UCD 1 may not be optimal. MSH noted that he had a list of equivalences and a simple script to make the translation. ACTION MSH to pass to DMR his list of UCD 1 / UCD 1+ equivalences and a complete list of columns in all the WFAU databases. DMR would use MSH's list as the basis for his translation. The new UCDs would need to be checked before making the new TAP service public. Miscellaneous: * MSH noted that he would be away for the week beginning 2/11/15, attending a EUCLID meeting. * RGM variously noted: - Tom Shanks had passed on an Italian GTO/GST proposal for an extension to the ATLAS Survey. Tom had suggested CASU and WFAU to process and handle the data, and both units were happy with this suggestion; it should not involve a major amount of work. However, it was not known whether the Italian group would want the processing and archiving to be done in the UK. - Marcus Ebert would be arriving shortly. His notional start date was the beginning of the month but he may not arrive on-site for another week or so. He would be working half-time on the LSST project. ACD, original: 1/11/15, revised: 8/11/15.