From mar@roe.ac.uk Mon Jan 13 16:09:05 2014 Date: Mon, 13 Jan 2014 15:30:02 +0000 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , Dave Morris , Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 10 January 2014 WFAU VDFS Science Archives progress meeting minutes, 10 January 2014 Present: RGM, ETWS, MAR, RPB, RSC, ACD, MSH Apologies: NCH, KTN, AL, NJC DoNM: 10am Friday 24 January 2014 in the Villas meeting room ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: RGM to e-mail Colin Vincent about an extension to the WFAU rolling grant. Done, though further exchanges are needed. ACTION: ALL; when the new OSA interface is ready, have a play with accessing the archive to check that there are no obvious problems. Done, all important issues had been addressed and the new interface is live. ACTION: UKIRT transition, RGM would attempt to elucidate further details of the observing strategy to be followed. Done, RGM had passed on some information supplied by Tom Kerr. Initial observations would be in a familiar format but that might change. ACTION: GES, ACD to check progress against the agreed time-line. Done, ACD noted we're a bit behind but the SQL cookbook had been updated. ACTION: MAR to investigate and quantify the issue with unmatched ATLAS sources. And if the issue warrants, RSC to look into the curation code with a view to pairing up the unmatched sources. Nigel Metcalfe indicated that he'd like us to recover as many of the real parings as possible. Given the apparent sub-optimum pointing of VST ATLAS observations this will mean re-factoring the source merging to recover objects across framesets (generate additional framesets?) ACTION: RSC to investigate modifying CU7 to recover ATLAS VST unmatched sources around frameset boundaries. Actions partly discharged but continuing: ---------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of DR10 to see if it's worth us hosting and whether it's necessary to keep all of the other releases. Continues. RGM had started a wiki page for WFAU SDSS strategy. However, it needs more work and in particular RGM still had to determine which WFAU archives had cross-matches with which SDSS releases. Actions carried forward from 13/12/2013 meeting: ---------------------------------------------- ACTION: RPB to do the work necessary to make the pending VISTA public releases. Continues. VIKING has been done (DR3), VIDEO had fallen over and needed to be re-run. ACTION: ALL to add/maintain entries on http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Continues. ACD noted he would be going through the entries shortly. ACTION: ACD, RPB to formulate a policy for backups and for checking that data could be recovered from the backups. Continues. ACTION: ACD to discuss with RGM the available budget for hardware purchases. Continues. ACTION: NJC to sort out the magnitude limits for V1.3 data. Continues. (NJC not present) ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. Specific points and new actions: ------------------------------- Project management: Nothing for the minutes. WFCAM, VISTA and VST updates: Nothing of note. Comments and issues arising from CASU minutes: MAR noted that the last CASU minutes had mentioned they'd obtained revised ESO grades for VISTA/VST observations. He had asked CASU for a copy of the latest grades and Eduardo would be supplying them (or the means to retrieve them) on his return to Cambridge. Networking: ETWS noted that transfers of WFCAM data back to CASU are progressing well. WSA/VSA/OSA/GES Operations: ETWS noted that VSA VVV ingests are still ongoing. Hardware and Systems: Ramses13 had recently crashed. Osorkon had been reporting problems with some of its arrays/disks, the data seemed OK and were currently being copied off the affected volumes (disk02 and disk05). MSH noted that takelot and pami would be coming off maintenance soon, the plan was to let this happen and just buy some spare parts that would likely fix any issues that arise. Survey Data Release: RPB reported that following a slight hiccup with full disks, VVV DR2 was nearing completion. The VIKING P90 release was almost complete. The ATLASv20131127 (P90) proprietary release and the ATLAS DR1 public release were available through the new UI. Software: ETWS reported: -- Added the possibility to ingest parallel into different monthly tables to the ingest software. RSC reported: CU19 required a couple of modifications to keep the VVV DR2 release running: a bug fix to handle the outgest of the last monthly detection that has different SQL constraints to the others and a memory optimisation to the process of splitting the vvvSource table outgest into chunks. RGM noted that he'd been doing some background reading on SQL server looking at the issue of speeding up VVV ingests. In particular use of the switch metadata operator in conjunction with partitioned tables warranted further investigations. Non-survey Data Release: The re-release of the WSERV2 database (Roy Gal) had been completed (deep stacks from the previous release had been lost on disk38). Astrogrid deployment, VO & Data Analysis services: MAR noted that he fixed a bug in the ATLAS SIAP service. Dave Morris had raised a couple of IVOA DAL/TAP issues for discussion. The items are on redmine, issues 303 and 304. Miscellaneous: ACD mentioned that he'd shortly be sending round a list of staff effort allocations and projects for individuals to check their entries. RGM noted that all the PDRs had been completed. -- Scanned by iCritical.