From acd@roe.ac.uk Fri Nov 4 11:22:52 2016 Date: Fri, 4 Nov 2016 11:22:28 +0000 From: Clive Davenhall To: Bob Mann , Andy Lawrence , "Read, Mike" , Eckhard Sutorius , Mark S. Holliman , Nicholas Cross , Nigel C. Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris , Clive Davenhall Cc: "dante@astro.puc.cl" , James Dunlop , "j.p.emerson@qmul.ac.uk" , ktn keithnoddle.org , "Lorenzo.Rimoldini@unige.ch" , "mcioni@aip.de" , "matt.jarvis@astro.ox.ac.uk" , "mike@ast.cam.ac.uk" , "nigel.metcalfe@durham.ac.uk" , "norman@astro.gla.ac.uk" , "pwl@star.herts.ac.uk" , "rgm@ast.cam.ac.uk" , "Simon.Dye@nottingham.ac.uk" , "s.j.warren@ic.ac.uk" , "tom.shanks@durham.ac.uk" Subject: WFAU VDFS Science Archives progress meeting minutes, 28 October 2016 WFAU VDFS Science Archives progress meeting minutes, 28 October 2016 Present: NJC, RPB, ETWS, MAR, MSH, ACD Apologies: RGM, RSC, STV, DMR, KTN, AL DoNM: 10:00 am Friday 11th November 2016 in the Villas Meeting Room MAR in the chair (room is booked). Actions discharged this time: ---------------------------- None. Actions carried forward from 9/9/2016 meeting: --------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan DR13 is installed but not yet cross-mounted. However, we still need to decide which SDSS releases to keep so the present action should remain. ACTION: MAR to write some H-ATLAS documentation once H-ATLAS is available. Continues. ACTION: RPB to check the log files and inform NJC of the problems with the previously listed non-survey releases. Continues. ACTION: NJC to investigate and fix the problems with the previously listed non-survey releases. Continues. This action cannot progress until RPB checks the log files and passes on details of the problems (action above). ACTION: MSH to check with DMR about progress towards, and anticipated release date of, a usable TAP service. Continues. NEW ACTION: ACD to arrange a meeting between DMR, STV and MSH during DMR and STV's next visit to agree (i) a list of enhancements to the TAP server necessary to have a deployable service and (ii) a deadline (or set of deadlines) by which these enhancements will be complete. ACTION: RPB to check with CASU whether they had any WFCAM observations made during the gap in the Edinburgh tapes. It was suggested we might like to check here first! Continues. ACTION: NJC to complete the deep-stacking software for VVV. Continues. Specific points and new actions: -------------------------------- Project management: * Joe Zuntz would be joining the IfA on Mon. 31/10/16 as a Chancellor's Fellow with specific responsibility for the interaction between the Institute and the Higgs Centre. He was likely to be heavily involved in WFAU activities. WFCAM, VISTA and VST updates: * Nothing to report. Networking: * Nothing to report; the networks were operating nominally. WSA/VSA/OSA/GES Operations: * ETWS reported that he had: - Successfully run the script to find reprocessed VISTA v1.3.1 tiles. - Transferred all v1.3.1 tiles. For the first six months of data he had also ingested the metadata and created the JPGs. The remaining files are currently being processed. - Retransferred some broken WFCAM files, created the JPGs and ingested the metadata and catalogues. - Ingested a night of WFCAM data, that had previously failed to ingest because of a corrupt file. - restarted VPHAS after a hiccup due to the recent Linux security update. - Updated MjdObs and DateObs from GpsObs where they were missing in the WFCAM files (20150724). - Created the GES schema browser for the line-list release. * RPB reported that he had: - Started processing the VIKING P96 data, but it crashed. NJC had since fixed the problem and RPB could re-run. - Started running the non-survey programme Wserve8, but it too crashed. ACTION NJC: investigate and fix the problem with Wserve8. - Begun moving ingest databases between servers. Specifically VVV had moved to the new server RAMSES18. The VSA was still to be moved and would be addressed once he returned from his imminent leave. * NJC noted that he was running another set of matched apertures for Steve Warren. Hardware and Systems: * RAMSES18 was up and running. * Every server had been upgraded to run SQL SERVER 2014 except RAMSES11. * A raid card had been replaced on unas. * A quote for a new NAS box had been requested. Review of Software Development Deadlines: * The list of software deadlines was briefly discussed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft ACD and NJC would carry out a complete review outwith the meeting. Other Software: * ETWS had: - Written and enhanced scripts to find reprocessed VISTA v1.3.1 tiles at CASU and to parse the lists so that they are usable by our ingest software. - Installed CASUTools v1.0.33 on userkaf, where they will be tested before being rolled out onto the other servers. - Installed swarp on userkaf for RGM's student * NJC had fixed a bug in ProgrammeBuilder for VIKING and worked on the matched aperture software. Review of Survey Release Deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * None. Non-survey Data Release: * None. Astrogrid deployment, VO & Data Analysis services: * DMR had suggested that some standard statistics should be published for all the columns in the various tables in order to improve query optimisation in the VO software and this idea was discussed. ACTION MAR: reply to DMR's e-mail about publishing statistics for columns. Miscellaneous: * NJC mentioned that the next VMC meeting will be held at ESO during June 2017. ACD, Original: 28/10/16, revised: 4/11/16.