From mar@roe.ac.uk Wed Jul 20 15:45:09 2016 Date: Wed, 20 Jul 2016 15:44:58 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 15 July 2016 WFAU VDFS Science Archives progress meeting minutes, 15 July 2016 Present: ETWS, ACD, STV, RSC, RPB, MAR, MSH Apologies: NJC, DMR, KTN, AL, RGM DoNM: 10:00 am Friday 29th July 2016 in the Villas Meeting Room ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: MSH, DMR install and configure the Nagios (https://www.nagios.org/) system monitoring package to monitor the availability of our database services. Discharged, it was decided not to install Nagios as this stage. ACTION: ACD to speak to DMR about the scope and schedule for installing and configuring Nagios and its intended subsequent use. Discharged, it was decided not to install Nagios as this stage. ACTION: NJC to set-up WISExSCOS neighbour tables. Continues. ACTION: ETWS (with RSC) to contact Microsoft SQL Server helpdesk for advice about the outgest problem that is the principal bottleneck with the VVV processing (briefly the ingest and outgest speed is limited to approx. 100 Mbit/sec for no apparent reason). This action has arisen due to the failure of Ani Thakar and Alex Szalay to reply to queries on this issue. Removed, this is now covered under software deadlines. Actions carried forward from 1/7/2016 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan We currently have space so it was decided to start transferring DR12 ACTION: ETWS to transfer SDSS DR12/13 Continues. ETWS had had contacted Ani Thakar, who had advised that SDSS DR13 would be released at the end of July (and that SDSS DR12 had recently been removed from the SDSS servers in preparation). RGM was happy to wait for DR13 but did not want further significant delays as one of his students planned to work with the date. ACTION: RPB/NJC to fix the errors encountered while trying to make non-survey releases U/14B/UA15, U/14B/UA16, U/14B/UA18 and U/14/BUA19. Continues. NJC believed that he had fixed the problems. RPB re-ran the releases, two of which worked, but the other two failed and require further investigation by NJC. Re-runs would be attempted for the previous failed attempts. ACTION: RPB to restore VSA replication. Continues. RPB had restored OSA replication but was holding off restoring VSA replication until the current VVV updates had completed, which is likely to take approximately another week. Still waiting for VSA ingests to complete. ACTION: ACD to liaise with DMR and STV to establish a set of deadlines for their work. Continues. ACTION: MAR to look at H-ATLAS documentation. Continues. Specific points and new actions: -------------------------------- Project management: * Nothing for the minutes. WFCAM, VISTA and VST updates: * Nothing for the minutes. Networking: * Nothing to report; the networks were operating nominally. WSA/VSA/OSA/GES Operations: * ETWS reported the following progress: - CU2 was running on the latest WSA ingest however there were more broken files than usual (some with 0 or 8 extensions). - VSA CU4 ingests for Feb-Mar of P96. - The OSA VPHAS release had fallen over. * ACD noted he was looking into a GES bug report concerning some missing abundances. Hardware and Systems: * MSH noted that we were in the process of ordering a new database node. This would come with 12Gb/s disks. Review of Software Release Deadlines: * Progress towards the software deadlines was reviewed briefly. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft Other Software: * ETWS noted that in working on creating a test DB for LSST ETWS he needed to increase the string length limit for SQL queries. ACTION: RPB,NJC to determine if the necessary OGLE neighbour tables had been set-up and that they would appear in the next VMC release. Review of Survey Release Deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * None Non-survey Data Release: * None. RPB was currently running a batch of projects. Astrogrid deployment, VO & Data Analysis services: * STV reported that the servers with openStack and the new TAP and cone services would soon be deployed. A replacement for the Astrogrid registry, REGTAP, was also in the pipeline. Miscellaneous: * Nothing for the minutes