From mar@roe.ac.uk Thu Nov 30 16:25:33 2017 Date: Thu, 30 Nov 2017 16:25:19 +0000 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Roy Williams , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU data services meeting minutes, 29 November 2017 WFAU data services meeting minutes, 29 November 2017 Present: MAR, RSC, RDW, RGM, MSH, ETWS, NJC Apologies: DMR, STV, NCH, AL, ACD, RPB DoNM: 11:00 am Wednesday 13th December 2017, Villas Meeting Room MAR in the chair (room is booked). Actions discharged this time: ----------------------------- ACTION: NJC to check if any of the new VISTA surveys warranted a P98 release. Discharged, P99 releases are likely to be the first required by new surveys. ACTION: RSC/NJC look into ppErrBit inconsistencies. Discharged. Code amendments checked in, and confirmed working. ACTION: MSH to email RGM and Jim a short summary of the current situation with regard to C2 and a significant power outage. MSH had met with ATC IT services. Currently they have no automatic shutdown system. Ewan Brown agreed to look into the issues. It was agreed to wait two weeks and see what they came back with before possibly escalating things. Actions carried forward from 1/11/2017 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan DR13 is installed but not yet cross-mounted. However, we still need to decide which SDSS releases to keep so the present action should remain. ACTION: MAR to write some H-ATLAS documentation once H-ATLAS is available. Continues. ACTION: ALL to consider advantages/disadvantages of combining or keeping VVV/VVVX separate. Continues, NJC had had some discussions with the PIs, but no conclusion yet. Our preference is to keep separate with maybe one final combined release. NJC to discuss further with Phil ACTION: RPB to add note to twiki to note filegroup issue and fix. Continues Actions carried forward from the Backup Strategy Meeting held on 27/1/2017: -------------------------------------------------------------------------- ACTION: RPB to create a twiki page documenting the backup procedure. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. Tuesday 28 February 2017 was subsequently agreed as a target date for completing these notes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsProcedures Continues. ACTION: RPB to create a twiki page documenting the procedure to restore datasets from a backup. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsRecovery Continues. ACTION: RGM to ask Steven Duffield to review the WFAU backup procedures. Continues. This action could not progress until RPB had written notes describing the backup and recovery procedures. ACTION (amended): MSH (was all) set up a meeting to review tests of SQL server on Linux prior to the next telecon with John Hopkins. Continues, awaits results of testing. Actions carried forward from the SQL Server on Linux Meeting 27/1/2017: --------------------------------------------------------------------------- ACTION: NJC to run the "testall" script as a basic sanity check. Check all file paths and change to Linux style. Should run in 30 minutes. This failed fairly quickly as XP_CMDSHELL not available under Linux. ACTION: RPB to check out how file permissions operate under MSSQL on Linux with a view to replacing XP_CMDSHELL with ssh operations. ACTION: RPB to run curation scripts on the WSA database to test re-doing a previous release. ACTION: MSH to check the crossmount/crossmatch/crossshare capabilities, connect two more servers, one Windows and one Linux. Candidate machines are ramses1 and ramses2. Continues. Specific points and new actions: -------------------------------- Project management: * RGM noted that the plan was to hold staff reviews before Christmas. The allocation of staff effort to various projects would also be reviewed. WFCAM, VISTA and VST updates: * Telecon with CASU Wednesday 6th December. Networking: * All WFAU networks were working nominally. WSA/VSA/OSA/GES Operations: * VSA/WSA data transferred up to end Oct 2017 * VVV source merging. * An issue with the WG15 content had been spotted by Cambridge. A new file had been generated, transferred and ingested into GESiDR5. ACTION: MAR/NJC to check requirements of the VINROUGE VISTA project. Hardware and Systems: * MSH noted that our oldest NAS box Pami was misbehaving, the data were being copied over to another disk. * The new DB node, ramses21, had arrived. * The new Linux upgrade would be carried out shortly, starting with the web servers. MSH would send round a note on how to set-up ssh logins for the new authentication. Status of Backups: * With RPB absent, assumed to be on track. Review of Software Release Deadlines: * The prioritised list of software items is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft Other Software: * NJC had been working on the common queries code and fieldID implementation. * ETWS had been working on 3rd party software installations. NJC note that he'd like the latest version of CASUtools installed. Review of survey deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * None Non-survey Data Release: * None Astrogrid deployment, VO & Data Analysis services: * MSH had set up services for some of the older VSA releases and GPS in DR10 Miscellaneous: Nothing for the minutes