From acd@roe.ac.uk Fri Feb 14 17:09:58 2014 Date: Fri, 14 Feb 2014 15:28:37 +0000 From: Clive Davenhall To: rgm@roe.ac.uk, al@roe.ac.uk, mar@roe.ac.uk, etws@roe.ac.uk, msh@roe.ac.uk, njc@roe.ac.uk, nch@roe.ac.uk, rpb@roe.ac.uk, rsc@roe.ac.uk, stv@roe.ac.uk, dmr@roe.ac.uk, acd@roe.ac.uk Cc: dante@astro.puc.cl, jsd@roe.ac.uk, j.p.emerson@qmul.ac.uk, keith@keithnoddle.org, Lorenzo.Rimoldini@unige.ch, mcioni@aip.de, matt.jarvis@astro.ox.ac.uk, mike@ast.cam.ac.uk, nigel.metcalfe@durham.ac.uk, norman@astro.gla.ac.uk, pwl@star.herts.ac.uk, rgm@ast.cam.ac.uk, Simon.Dye@nottingham.ac.uk, s.j.warren@ic.ac.uk, tkerr@jach.hawaii.edu, tom.shanks@durham.ac.uk Subject: WFAU VDFS Science Archives progress meeting minutes, 7 February 2014 WFAU VDFS Science Archives progress meeting minutes, 7 February 2014 Present: RPB, RGM, ETWS, RSC, NJC, ACD Apologies: MSH, MAR, NCH, KTN, AL DoNM: 10am Friday 28 February 2014 in the Villas meeting room MAR in the chair (room is booked). Note: the meeting is a week later than expected in order to avoid a clash with the visit by the new owners of UKIRT. Actions discharged this time: ---------------------------- ACTION: ACD to give MSH an estimate of the minimum amount available, as a matter of urgency. Done. MSH had been given a minimum amount and was now ordering high-priority equipment. ACTION: RPB, MSH investigate setting up a local nightly SVN backup. Discharged. This item would be included in the report describing a revised backup strategy that RPB and ACD were preparing. Actions partly discharged but continuing: ---------------------------------------- None. Actions carried forward from 24/1/2014 meeting: ---------------------------------------------- ACTION: RSC to investigate modifying CU7 to recover ATLAS VST unmatched sources around frameset boundaries. Continues. Investigations had begun and a strategy had been worked out. However, this action now has lower priority. Nonetheless a fix should be put in place before the ATLAS meeting on 14-16 April (see below) and it should be followed by a consortium release shortly after the meeting. ACTION: RPB to do the work necessary to make the pending VISTA public releases. Continues. VIKING has been done (DR3), The VMC release had been done but needed to be checked before it was copied to the public servers. It seemed surprisingly small. NEW ACTION: NJC to perform a quick check on the VMC release. ACTION: ALL to add/maintain entries on http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Continues. ACD reiterated that he would be revising the entries shortly. ACTION: ACD, RPB to formulate a policy for backups and for checking that data could be recovered from the backups. Continues. RPB and ACD had discussed the current procedures and how they might be enhanced. A report (in the form of a wiki page) was in preparation and would be available for comment in a few days. RPB solicited for suggestions about how restored backups might be checked for validity. Suggestions included: checksums on the database container files, SELECT COUNT(*) on constituent tables and investigating whether SQL Server included any functions for this purpose. Finally RPB noted that it may be necessary to buy additional agents for SYMANTEC Backup Exec. ACTION: ACD to discuss with RGM the available budget for hardware purchases. Continues. The amount of money available for hardware in the current rolling grant was now largely understood. Also RGM had e-mailed Colin Vincent with an estimate of the cost of a one-year extension at the same level as the current rolling grant, but was still waiting for a reply, which would affect the final hardware budget. ACTION: RGM to investigate the size and content (how does it differ from DR9?) of DR10 to see if it's worth us hosting and whether it is necessary to keep all of the other releases. Continues. RGM had started a wiki page for WFAU SDSS strategy (http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan). However, it needs more work and in particular RGM still had to determine which WFAU archives had cross-matches with which SDSS releases. ACTION: NJC to sort out the magnitude limits for V1.3 data. Continues. The software was largely written, but NJC had not had the time to finish and test it due to pressure of Gaia work. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. Specific points and new actions: ------------------------------- Project management: * RGM reported: - (As mentioned above) he had e-mailed Colin Vincent about a one year extension to the rolling grant but was still awaiting a reply. Once Colin had replied we could look into organising the next WFAU Quarterly meeting, which would probably be in March or April. ACTION: ALL: make sure that all forthcoming absences during the next couple of months are entered into the 'staff absences' Wiki page. - Representatives of one of the new owners of UKIRT would be visiting ROE on 21/2/14. In the short term WFCAM observations would continue in the standard mode. The new owner who would be visiting was particularly interested in observing artificial satellites, which, of course, are trailed in standard observing. Mike Irwin had developed software for analysing such satellite trails. We would need to archive the analysis of the satellite trails, which would presumably involve developing a suitable database schema. It would be nice to have a mock-up available by the time of the visit. ACTION: RGM: E-mail Mike Irwin to ask for an example of the satellite data. ACTION: NJC: Set up a schema to hold these data. The outline plan for the meeting was that ETWS would reprise his talk describing operations, MAR would describe the user interface work and then there would be a discussion, for which NJC should be on hand. - There would be an ATLAS meeting in Durham during 14-16 April (Easter week) and a VVV meeting in Santiago during April. WFCAM, VISTA and VST updates: * New sky baffles had been fitted to the VST. The sky background characteristics of the images would now be different and CASU were investigating how this would affect processing the images. Comments and issues arising from CASU minutes: * Comments on the minutes of the meeting dated 22/1/4: - It is encouraging and pleasing that the disaster recovery plan for retrieving data from Edinburgh is working satisfactorily. - For the VST, Nigel Metcalfe had supplied some new colour terms and new zero points were being tried. If these parameters are adopted a wholesale re-processing may be necessary. - A text-file, rather than PDF version of the listing of VST and VISTA grades may become available, which would be very useful. Networking: * ETWS reported that the networks continued to work well. WSA/VSA/OSA/GES Operations: * RPB reported that the VISTA VHS P90 release was underway. * ETWS reported: - The recovery operation to transfer data back to CASU was now complete. - The latest released VISTA data from September and October 2013 had been transferred. - OSA catalogue data for August and September 2013 had been ingested. - Some JPGs that were corrupted by the disk38 problem had been recreated. - New browser pages for the latest WSA non-survey releases had been created. Hardware and Systems: * RPB reported that the up-to-date copy of the backup software (Backup Exec from SYMANTEC) had been received. It had been difficult to install but now seems to be working fine, and appears to be faster than it used to be. * ETWS noted that MSH has upgraded the version of debian on thoth and ETWS had subsequently installed the latest third-party software. Survey Data Release: * There had been no survey releases since the last meeting. ACTION: RPB to ask NJC to carry out checks on the latest VVV release when it was ready, in preparation for a release. ACTION: RSC to set a date for completing the actions from the brainstorming meeting held shortly before Christmas. (This action is in preparation for the next brainstorming meeting, which will be arranged once the VVV release is complete.) Software: * RSC noted that a fix had been added to circumvent the slow samba on ingest. * ETWS reported that he had fixed some CU4 timeout problems that occurred while trying to connect to the VVV database when it was very busy running catalogue ingests. Non-survey Data Release: * RPB noted that there had been a U/12B/D1 WSA non-survey release. ACTION: NJC to remind Omar Almaini about the UKIDSS DR11 final data release. It was hoped to make the release shortly and there was a possibility of value-added catalogues. Astrogrid deployment, VO & Data Analysis services: * Nothing to report (and MSH was absent). Miscellaneous: * NJC noted that he had been contacted by a documentary film- maker who were interested in producing a documentary about UKIRT. He would be speaking to them later in the day (7/2/14). ACD, draft: 10/2/14, completed: 14/2/14. -- Scanned by iCritical.