From acd@roe.ac.uk Fri Sep 6 19:05:57 2013 Date: Fri, 6 Sep 2013 19:05:50 +0100 From: Clive Davenhall To: Mike Read Cc: Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , Matt Jarvis , Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 6 September 2013 WFAU VDFS Science Archives progress meeting minutes, 6 September 2013 Present: RSC, ETWS, MSH, RPB, NJC, ACD Apologies: NCH, MAR, RGM, KTN, AL DoNM: 10am Friday 20 September 2013 in the Villas meeting room MAR in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: All to think about items that could be included in the WFAU poster. Done: RGM had decided on the topic for his poster. Actions partly discharged but continuing: ---------------------------------------- ACTION: MSH/NJC to look at current and future VVV load server storage requirements so that RAMSES14 could be set-up optimally. Continues: RAMSES14 was now configured to use Raid50 and ready to start ingesting VVV 1.3. However, NJC still had to calculate the total data volume required. ACTION: MAR/NJC to approach VISTA PIs about their public releases. Continues: NJC had forwarded details to MAR, but it was unclear whether MAR had contacted the PIs. Actions carried forward from 23/8/2013 meeting: ---------------------------------------------- ACTION: RPB to have a go at releasing other VISTA PI DBs (e.g. for JPE) Continues: action reassigned from MAR to RPB. A release had been attempted but failed. MAR's part of the release succeeded but more work was required by RPB. ACTION RPB and NJC: RPB to check the logs, identify the error and forward to NJC, who would investigate and fix it. Continues: RPB had found and forwarded the error but NJC had not yet fixed it. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. Specific points and new actions: ------------------------------- Project management: * ACD recalled that he had recently circulated a message requesting everyone to add details of forthcoming deadlines that they were aware of to Twiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines He had been underwhelmed by the response to date and reiterated his request. The rationale for this exercise is that as the number of projects increases it becomes increasingly difficult to keep carry all the relevant dates in ones' head and some more reliable mechanism to keep track of them is needed. ACTION: circulate another request soliciting additions to the page. WFCAM, VISTA and VST updates: * Nothing to report. Comments and issues arising from CASU minutes: * Nothing to report. Networking: * MSH noted that there had been a network outage of 27/8/13. * ETWS noted: - Lightpath migration to Janet6 will take place for UKLight on 26 September. - the operating system on his desktop computer was scheduled to be upgraded on or shortly after Mon. 9/9/13 and he anticipated that this would cause him some disruption. WSA/VSA/OSA/GES Operations: * ETWS noted: - Restarted catalogue ingests of 2013 data into the OSA, that were stopped due to the latest NASbox problems, - he was going through the VISTA v1.3 catalogue ingests and fixing failed ingests (a tedious and time-consuming process). * RSC noted that more reprocessed and additional GES data have been received. Hardware and Systems: * MSH reported:- - the NAS box is back up, - RAMESES14 is now operational, - automatic backups using rsync are in place on all NAS devices: disk37 now contains a complete copy of disk34. ACTION: RPB to set up the backup jobs as superuser, so that they can be configured to preserve ownership etc. ACTION: RPB to set up a routine backup of the GES data (which occupies an insignificant amount of disk space). Survey Data Release: * RSC noted that there will be a re-release of the GES iDR1, but it will probably have to wait until MAR returns from leave. * NJC reported: - VISTA VVV DR1 is in the process of being passed to ESO for validation, - VISTA VVV DR2 is in preparation but will not be ready for several months. Software: * NJC reported: - some progress had been made on the match aperture pipeline, but not as much as hoped, - attempts to get the source extractor to work were continuing (largely involving re-arranging existing code rather than writing new code). * ETWS reported some small fixes to update WFCAM testing software to be used with VISTA data * RSC reported: - It turns out that the combined views of the monthly-split VVV detection tables don't perform as well as the original combined tables in a variety of cases, so I've undertaken a refactoring of the remaining major curation tasks for the VVV that have not already been modified to access the monthly-split detection tables directly instead of using the views. This has now been done for all source merging tasks including notably CU7 & the SynopticSource merger and CU19 now outgests the VVV detection table one monthly table at a time rather than in chunks determined by row counts. So between these updates, the refactoring of CU6 and the tasks that were modified previously to allow updates of the detection tables to occur, we should now not be using the VVV detection views for any significant curation task. - The bug that was putting VVV monthly-detection tables into the wrong file group has been squashed (we think?) in time for the preparation of, and catalogue ingest into, the new VVV ingest database on ramses14. Non-survey Data Release: * RPB noted that a non-survey VSA release had been attempted, but there had been insufficient disk space. He hoped to complete this release next week. ACTION: (RPB completing this release is already an action; see above.) Astrogrid deployment, VO & Data Analysis services: * MSH reported: - he had done a few miscellaneous VO things in connection with the recent Interoperability meeting, - A new public TAP service for UKID DR9 was available. Miscellaneous: * ACD noted the note about office housekeeping recently circulated by Mark Collins. * ACD noted that volunteers were needed for WFAU displays for the forthcoming ROE Open Days. ACTION: ACD to identify and press-gang volunteers to assist with the Open Days. ACD, draft: 6//9/13, completed: 6/9/13. -- Scanned by iCritical.