From acd@roe.ac.uk Mon Jul 3 14:42:54 2017 Date: Mon, 3 Jul 2017 13:42:37 +0000 From: Clive Davenhall To: Bob Mann , Andy Lawrence , "Read, Mike" , Eckhard Sutorius , Mark S. Holliman , Nicholas Cross , Nigel C. Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris , Clive Davenhall , Roy Williams Cc: "dante@astro.puc.cl" , James Dunlop , "j.p.emerson@qmul.ac.uk" , ktn keithnoddle.org , "Lorenzo.Rimoldini@unige.ch" , "mcioni@aip.de" , "matt.jarvis@astro.ox.ac.uk" , "mike@ast.cam.ac.uk" , "nigel.metcalfe@durham.ac.uk" , "norman@astro.gla.ac.uk" , "pwl@star.herts.ac.uk" , "rgm@ast.cam.ac.uk" , "Simon.Dye@nottingham.ac.uk" , "s.j.warren@ic.ac.uk" , "tom.shanks@durham.ac.uk" Subject: WFAU VDFS Science Archives progress meeting minutes, 23 June 2017. WFAU VDFS Science Archives progress meeting minutes, 23 June 2017 Present: STV, NJC, RPB, MSH, ETWS, RDW, RGM, ACD Apologies: MAR, RSC, DMR, KTN, AL, NCH DoNM: 10:00 am Friday 7 July 2017, Villas Meeting Room MAR in the chair (room is booked). Actions discharged this time: ----------------------------- ACTION: NJC to contact Robert Lupton to check if he already has code for computing model magnitudes which can be adapted. Done. NJC had contacted Robert Lupton but though he has software to compute model magnitudes it was not suitable for adoption by WFAU as it was too closely integrated into and reliant upon various other external software items. However, subsequent discussion revealed that Mike Irwin was planning to write suitable software. Actions carried forward from 23/6/2017 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan DR13 is installed but not yet cross-mounted. However, we still need to decide which SDSS releases to keep so the present action should remain. ACTION: MAR to write some H-ATLAS documentation once H-ATLAS is available. Continues. ACTION: RPB to check with CASU whether they had any WFCAM observations made during the gap in the Edinburgh tapes. It was suggested we might like to check here first! Continues. ACTION: MAR to arrange the date of the next CASU/WFAU telecon with Mike Irwin. Continues. ACTION: MAR to check that any data on RAMSES3 and RAMSES5 which is still required is copied elsewhere. Continues. This item now requires urgent attention. Actions carried forward from the Backup Strategy Meeting held on 27/1/2017: -------------------------------------------------------------------------- ACTION: RPB to make a full tape backup of VVV promptly after it completes and henceforth make a copy annually. Continues. ACTION: NJC, In the context of the VVV, to check which curation steps can be paused without misadventure and hence compile a list of points at which backups may be made during the ingest. The list should be added to the twiki. Continues. NJC had a list of suitable pause points in his head. He would write it down in a twiki page. ACTION: NJC (was NJC, ETWS, MSH, MAR) to create a twiki page comprising a list of all the external datasets that WFAU hosts. The information included for each dataset should include: its name, size, where and when it originates and what WFAU uses it for. Continues. ETWS had created a page which is largely complete. However, a few queries remain and NJC will liaise with RPB and MAR to resolve them. Once NJC is satisfied with the list the action would be complete but henceforth ETWS would periodically check the list. ACTION: RPB to create a twiki page documenting the backup procedure. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. Tuesday 28 February 2017 was subsequently agreed as a target date for completing these notes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsProcedures Continues. ACTION: RPB to create a twiki page documenting the procedure to restore datasets from a backup. These notes should be usable by someone not routinely involved in the backups and, for example, include the physical location of the tapes. A stub page for RPB to complete is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/BackupsRecovery Continues. ACTION: ACD to obtain an estimate of the likely cost of a fire safe. Continues. ACD now had an estimate of the volume of tapes to be stored. The next step is to investigate suitable safes. ACTION: RGM to ask Steven Duffield to review the WFAU backup procedures. Continues. This action could not progress until RPB had written notes describing the backup and recovery procedures. ACTION: RPB to create a tape backup of the Gaia commissioning data at terra:/disk1/gaia/KAKAI_OLD and hand the tape to NCH for safe-keeping. Continues. ACTION (amended): RPB Generate a Gaia DR1 neighbours table for the most recent WSA release and the VVV DR4 release. Continues. ACTION (amended): MSH, RSC, NJC (was all) run further tests of SQL server on Linux. Continues. Informal testing has continued, but more formal tests are needed. ACTION (amended): MSH (was all) set up a meeting to review tests of SQL server on Linux prior to the next telecon with John Hopkins. Continues. Specific points and new actions: -------------------------------- Project management: * ACD introduced Roy Williams (RDW) who would be taking over many of ACD's duties (though probably not chairing alternate VDFSSA meetings). From the autumn ACD would largely retire, but would continue at a level of half a day per week, largely to manage the ASTERICS grant. WFCAM, VISTA and VST updates: * All the instruments were working as expected. Networking: * All WFAU networks were working nominally. There was an issue with a LAN/ULAN switch, but this was an issue for the ATC Computing Support. The symptom was reduced data rates from machines on-site at the ROE; external machines were unaffected. WSA/VSA/OSA/GES Operations: * ETWS reported that he was still working his way through the OSA data. The newly and re-created list-driven files have been transferred and ingested. The catalogues are also ingested and the FinaliseCatalogueIngest step should finish during the weekend of 24-25/6/17. * RPB reported that: - The VHS P97 team release was being prepared. - The VIKING release was also in preparation and currently running CU19. - The VVV release was still in progress. There had been some issues with creating tiles which were being investigated. - The new UKIDSS release was being made available to users. Hardware and Systems: * MSH reported that: - The OSA server was in a fragile state, with disks randomly dropping off-line. The contents were being backed-up to other disks as a precaution. - Installation of the new networks switch was complete and it was working satisfactorily. - The older switches would be re-configured during the week beginning Mon. 26/6/17. - The new NAS box ordered a few weeks ago had not arrived. ACTION ACD: check to establish the state of the order for the new NAS box. Status of Backups: * RPB reported that the backup of Ramses17 had failed again. He suspected that the reason for the failure was that though the tape library can hold 80 tapes it is partitioned into two slots, each of 40 tapes and that these slots are not large enough for the Ramses17 backup. He would continue to investigate. Review of Software Development Deadlines: * This item was skipped to avoid delaying the meeting. ACD and RDW would go through the software deadline with NJC during the week beginning Mon. 26/6/17. The prioritised list of software items is available at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/PrepareSoft Other Software: * NJC noted he had made some changes to the ESO release software following feedback on the recent VPHAS and VIKING ESO releases. * ETWS reported that in between working on the OSA ingests he had made various minor updates and spent some time on the BrowserParser. Review of Survey Release Deadlines: * Progress towards forthcoming survey deadlines was reviewed. See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * None to report. Non-survey Data Release: * None to report. NJC noted that he had a visitor arriving during the week beginning 26/6/17 who may need some special non-survey releases. Astrogrid deployment, VO & Data Analysis services: * STV reported that the new OSA TAP service was deployed and had been successfully presented at the Shanghai IVOA interoperability meeting. Additional TAP services will be deployed in due course. Miscellaneous: * NJC noted that a VVV meeting would be held at the ROE during 3-5/7/17. * MSH noted that he would be away from 5/7/17 and NJC that he too would be away from 7/7/17. ACD, 27/6/17.