From mar@roe.ac.uk Mon Jun 20 12:45:42 2016 Date: Mon, 20 Jun 2016 12:45:20 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 10 June 2016 WFAU VDFS Science Archives progress meeting minutes, 10 June 2016 Present: NJC, ETWS, RGM, MAR, MSH, RSC Apologies: STV, DMR, KTN, AL, ACD, RPB DoNM: 10:00 am Friday 24th June 2016 in *** Crawford meeting room 1 **** ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION MAR: Contact CASU to resume the monthly teleconferences and arrange the date of the next one. Discharged: Scheduled for the following Wednesday. ACTION MAR: Check the revised VSA documentation and give feedback to NJC. Discharged. Actions carried forward from 27/5/2016 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan We currently have space so it was decided to start transferring DR12 NEW ACTION: ETWS to transfer SDSS DR12 ACTION: MSH, DMR install and configure the Nagios (https://www.nagios.org/) system monitoring package to monitor the availability of our database services. Continues. ACTION: ACD to speak to DMR about the scope and schedule for installing and configuring Nagios and its intended subsequent use. Continues ACTION: RPB/NJC to fix the errors encountered while trying to make non-survey releases U/14B/UA15, U/14B/UA16, U/14B/UA18 and U/14/BUA19. Continues. NJC believed that he had fixed the problems. RPB re-ran the releases, two of which worked, but the other two failed and require further investigation by NJC. Check status with RPB. ACTION: RPB to restore VSA replication. Continues. RPB had restored OSA replication but was holding off restoring VSA replication until the current VVV updates had completed, which is likely to take approximately another week. ACTION: ACD to liaise with DMR and STV to establish a set of deadlines for their work. Continues. Actions resulting from the VVV DR3 brainstorming meeting held on 24/2/15: ------------------------------------------------------------------------ ACTION: ETWS (with RSC) to contact Microsoft SQL Server helpdesk for advice about the outgest problem that is the principal bottleneck with the VVV processing (briefly the ingest and outgest speed is limited to approx. 100 Mbit/sec for no apparent reason). This action has arisen due to the failure of Ani Thakar and Alex Szalay to reply to queries on this issue. Continues. ETWS did not feel that he was the best person to progress this issue. ACD should discuss how to proceed with RSC. ACTION: ACD to discuss with RSC the best way to progress investigations of the outgest speed bottleneck. Leave as an action for now until ACD present. Specific points and new actions: -------------------------------- Project management: * WFAU Quarterly Monday 13th 13:30 in the Management meeting room. WFCAM, VISTA and VST updates: * Nothing to report. Networking: * The University firewall had continued to cause problems (broken pipes to sessions). MSH reported that an attempted fix, re-routing via Leeds, was scheduled for the following Tuesday. WSA/VSA/OSA/GES Operations: * ETWS reported the following progress: - OSA: All P96 data in and illumination correction updates applied. - WSA: fixed some broken jpegs. The re-processed pawsky files were all in. The GPS nebulised catalogue files needed a bit more work to get them] to ingest. - VSA: the VVV objID update had finished. CU4 was being run on the latest data. * NJC noted various work and fixes on ESO releases for VIKING, VMC and VIDEO. The VMC detection table in the database generated for the ESO release was currently empty, this was being looked into. We had also been contacted about ESO releases for VPHAS and ATLAS. Hardware and Systems: * MSH reported that ramses17 was installed, online and accessible (100TB). Review of Software Release Deadlines: * Progress towards the software deadlines was reviewed See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft Other Software: * NJC reported work on the WISExSCOS database and 2MPZ as well as ESO release fixes. ACTION: NJC to set-up WISExSCOS neighbour tables. * MAR noted that he'd ingested the SCOS extended source catalogue which now needed to be copied over to the SSA database. * RSC reported updates to the provenance filler code to handle two versions of calibration files and a bug fix to allow parallel CU19 on the same machine. RGM asked about H-ATLAS, NJC responded that we were still waiting on the data. ACTION: MAR to look at H-ATLAS documentation. Maria-Rosa had requested we host some of the OGLE VMC catalogues. Review of Survey Release Deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * Room had been found on ramses15 for VHSv20160507 and this P95 VHS release was now available. Non-survey Data Release: * None Astrogrid deployment, VO & Data Analysis services: * Nothing to report. Miscellaneous: * Nothing for the minutes