From mar@roe.ac.uk Fri Oct 27 11:37:27 2017 Date: Fri, 27 Oct 2017 11:37:19 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Roy Williams , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU data services meeting minutes, 19 October 2017 WFAU data services meeting minutes, 19 October 2017 Present: RPB, ETWS, MAR, NJC, RSC Apologies: DMR, STV, NCH, KTN, AL, ACD, RDW, MSH, RGM DoNM: 11:00 am Wednesday 1st November 2017, Villas Meeting Room MAR in the chair (room is booked). Actions discharged this time: ----------------------------- ACTION: MAR to arrange the date of the next CASU/WFAU telecon with Mike Irwin. Discharged. Another would be arranged in the coming weeks where GAIA DR2 astrometric re-processing would be discussed. ACTION (amended): RPB, ETWS, NJC Generate a Gaia DR1 neighbours table for the most recent WSA release and the VVV DR4 release. Discharged. ACTION: MAR to update the release deadlines tiwki doc with P98 VSA releases. Discharged. ACTION: ACD/ETWS to investigate rolling back the GES ingest code. ETWS had been re-writing rather than rolling back the code. Actions carried forward from 15/9/2017 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan DR13 is installed but not yet cross-mounted. However, we still need to decide which SDSS releases to keep so the present action should remain. ACTION: MAR to write some H-ATLAS documentation once H-ATLAS is available. Continues. ACTION: NJC to check if any of the new VISTA surveys warranted a P98 release. Continues Actions carried forward from the Backup Strategy Meeting held on 27/1/2017: -------------------------------------------------------------------------- ACTION: RPB to create a twiki page documenting the backup procedure. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. Tuesday 28 February 2017 was subsequently agreed as a target date for completing these notes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsProcedures Continues. ACTION: RPB to create a twiki page documenting the procedure to restore datasets from a backup. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsRecovery Continues. ACTION: RGM to ask Steven Duffield to review the WFAU backup procedures. Continues. This action could not progress until RPB had written notes describing the backup and recovery procedures. ACTION (amended): MSH (was all) set up a meeting to review tests of SQL server on Linux prior to the next telecon with John Hopkins. Continues, awaits results of testing. Actions carried forward from the SQL Server on Linux Meeting 27/1/2017: --------------------------------------------------------------------------- ACTION: NJC to run the "testall" script as a basic sanity check. Check all file paths and change to Linux style. Should run in 30 minutes. ACTION: RPB to run curation scripts on the WSA database to test re-doing a previous release. ACTION: MSH to check the crossmount/crossmatch/crossshare capabilities, connect two more servers, one Windows and one Linux. Candidate machines are ramses1 and ramses2. Continues. Specific points and new actions: -------------------------------- Project management: * Nothing for the minutes WFCAM, VISTA and VST updates: * Nothing for the minutes Networking: * All WFAU networks were working nominally. WSA/VSA/OSA/GES Operations: * ETWS reported that the VSA was CU1'ed up to June 2017 and CU4'ed up to April 2017. WSA was CU3'ed up to August 2017, and there had been some issues with the flagging of "standard obs" for a non-survey project. GES was in a debug/ingest cycle. * VVV was running constraints * VMC P98 had ingested the deep products. * VPHAS P95 release running * ATLAS P98 release running * OSA copied to ramses20, OSAP98 on ramses12. * UKIDSS NJC and MAR had been investigating issues thrown up by the grouping and deep stacking of GCS and LAS. Hardware and Systems: * Nothing for the minutes Status of Backups: * RPB, RGM and RDW had held a meeting to discuss backups, the main outcome appeared to be a decision on the venue for the Unit's Xmas lunch. Review of Software Release Deadlines: * The prioritised list of software items is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft Other Software: * ETWS had been working on the GES ingest code * NJC had been working on fieldID groupings, VPHAS, and multi-epoch mergeLogs. * RSC had added an option to CU19 to copy detection tables over. RSC noted that some ppErrBit flagging had been done on detection tables but not recorded in programmeFrame ACTION: RSC/NJC look into ppErrBit inconsistencies. NJC raised the issue of how to best curate VVV/VVVX. Separate projects or combine? Short period when both were observed. But otherwise they would effectively be split by date. Eventually would be released together. ESO considerations? ACTION: ALL to consider advantages/disadvantages of combining or keeping VVV/VVVX separate Review of survey deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * Nothing to report. Non-survey Data Release: * N089C0102v20171002 was released. Astrogrid deployment, VO & Data Analysis services: Nothing for the minutes Miscellaneous: * RPB would shortly be sending round a doodle poll for the Xmas lunch.