From mar@roe.ac.uk Wed Jun 27 17:40:09 2012 Date: Wed, 27 Jun 2012 17:40:02 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Andrew Lawrence , Bob Mann , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Norman Gray , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives team meeting minutes, 22/6/12 Present: MAR, RSC, NJC, ETWS, Apologies: JPE, AL, KTN, STV, MSH, RGM, RPB, NCH, NG DoNM: 10am Friday July 6th 2012 in the Vista Hut (subject to non-demolition); NCH in the chair. Actions discharged this time: ----------------------------- ACTION: RSC/MAR to create a FITS metadata header file utility for ESO catalogue deliverables NJC, RSC and MAR have produced something that seems to convince the ESO validator all is well. ACTION: NG/ETWS/RGM to get together to make sure UCD1+ developments are ready to fold in to phase-III Vista deliverables Done for the columns needed for ESO. ACTION: NCH to email VVV co-PIs to get agreement on scope of current phase-III catalogue deliverables VVV have been in touch ACTION: NJC to discuss with VIDEO PI the requirements for current phase-III catalogue deliverables NJC has discussed this with Matt and we've been asked to supply some deep mosaics products. ACTION: MAR to attend next VMC teleconf on Thurs 14th at 9am BST. Done. Unsurprisingly they would like a release ASAP after the March data are available. Actions partly discharged but continuing: ----------------------------------------- ACTION: RSC to scope out the areas/situations where we'd like to test MonetDB i.e. where do the current bottlenecks lie? Continues; of the few present no-one had had the chance to look into this further. Actions carried forward from 08/06/12 meeting --------------------------------------------- ACTION: RPB to set up WSA mirror/sync for flat file DBs using MS SQL replication. Continues. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues; this one still has low priority. ACTION: RPB to complete the LTO-5 tape backups of VVV and SSA DBs and SSS flat file datasets. And VVV DR1. And WISE-ASS (snig). Continues; Specific points and new actions: -------------------------------- Project management: All were encouraged to read and add to the grant renewal case docs. WFCAM, VISTA and VST updates: Nothing to report this time. Comments and issues arising from CASU minutes: No new minutes this time. Networking: Transfer speeds from CASU are still very slow. Tests writing to /dev/null and local disks seem to rule out problems with WFAU's network. Hopefully investigations at CASU might help pin-point the problem. WSA/VSA/OSA/GES Operations: ETWS noted: -- Finished transfer of April data for WFCAM. Transfer speeds were still slow, so I ran some tests and got the same low (~5MB/s) speeds independent of were the data ended up, local disk, NAS box, or /dev/null. Mike Irwin investigated there speed between their storage and apm7 where UKLight is plugged in, and got speeds of up to 40MB/s, so it seems that it is (again, as some years ago) a UKLight connection problem. As far as I know MSH has checked the connection to menkaure, I'm awaiting Mike's check results of the connectivity in Cambridge. ETWS also mentioned that CASU has re-processed all WFCAM data to what we will be calling version 5 (inclusion of avconf values in cats). It was thought that we would stick with the version(s) we currently have, certainly for DR10 and given the current transfers speeds. Hardware and Systems: Nothing of note this time. Survey Data Release: UKIDSS DR10 (sans GPS) awaiting LAS eyeball results and QC. VISTA surveys: VMC: Good progress had been made with the production of ESO Phase 3 catalogues. The CAT and MPHOT test products now pass the validator. VIKING: Awaiting confirmation from Will that he's happy for the files to be uploaded to ESO. VVV: QC was currently being applied to P87 so as to list deprecated files for exclusion from CASU's upload of pipeline products to ESO. Software: ETWS noted: -- Updated CU4 to ingest data into the monthly VVV detection tables. -- Wrote a wrapper script to update the VSA template schemas with the UCD1+'s from a list given by Norman's code. -- Updated pyfits to version 3.0.8 on all our curation servers. RSC noted: We noticed that the view joining the new monthly-split VVV detection tables only offers read access to the data, so that any curation scripts requiring to update the VVV detection table will need to be modified to update the monthly tables directly. As photometric recalibration (CU8) is currently disabled for VVV releases, this means that the only regular process that needs updating is the propagation of deprecations into the detection table. I've contacted the PyFITS developers to ask for direct support in their FITS table construction API for the various additional column header keywords required by ESO, such as TUCD and TCOMM. A work around has been put in place for the current set of ESO releases. Also, further work on the GES schema has been required. Non-survey Data Release: None. Astrogrid deployment & Data Analysis services: Nothing of note this time. Miscellaneous: Nowt -- Scanned by iCritical.