From mar@roe.ac.uk Wed Feb 1 09:40:29 2012 Date: Tue, 31 Jan 2012 17:12:59 +0000 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Andrew Lawrence , Bob Mann , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Norman Gray , Stephen Warren , Tom Kerr , Tom Shanks Subject: VDFS meeting minutes 27/1/12 Minutes of WFAU VDFS Science Archive meeting: January 27th 2012 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: ETWS, RPB, NJC, ACD, MAR, MSH, STV, RSC, RGM Apologies: JPE, AL, NCH, KTN, C37, 89X DoNM: 10am Friday February 10th 2012 in the VH (NCH in the chair) Actions discharged this time: ----------------------------- ACTION: RPB to try monthly detection table partitions for VVV. Some of the later CUs have now been tested and the assignment of multiframeID/partition for deep products has been implemented. Discharged. Actions partly discharged but continuing: ----------------------------------------- ACTION: RPB/RSC to roll out latest curation client / DB server share configuration for routine ops use. Continues; RSC to carry out parellelised code tests Actions carried forward from 13/01/12 meeting --------------------------------------------- ACTION: RPB to set up WSA mirror/sync for flat file DBs using MS SQL replication. Continues; the presence of mergelog tables in the VSA replication caused a clash with a survey release. So replication had been turned off! No-one could remember if mergelog tables needed to be in the sync DBs so they'll be removed and then see if that breaks anything. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues. ACTION: RPB to complete the LTO-4 tape backups of VVV and SSA DBs and SSS flat file datasets. Continues. Specific points and new actions: -------------------------------- Project management: Nothing to report. WFCAM, VISTA and VST updates: No news to report this time. Comments and issues arising from CASU minutes: No new minutes this time. Networking: Had been performing well until Menkaure had stopped working that morning. WSA/VSA/OSA Operations: ETWS reported: -- Transferred the latest WFCAM data up to December 2011. JPGs are created and metadata extraction is under way. -- Transferred the version 0.9 VST data, so ACD can finalise the schemas. Hardware and Systems: Kakai and Menkaure were currently showing problems. MSH reported that the new cluster node, ramses12, was ready for use and that another cluster node and processing machine had been ordered. There then followed a discussion on further hardware requirements (another cluster node, processing machine, new Tape library and a machine to host Monet DB tests) and how best to use ramses12 (VVV load server vs public DB server). Specific requests/costings to be forwarded to RGM. Software: ETWS reported: -- Started 3rd party upgrades on all servers after MSH's kernel upgrades. This will include a downgrade to the old gcc4.5.2 to be able to run the CU4 code. NJC reported: -- I have worked with RPB to sort out the issues with CU13 and made updates to ProgrammeBuilder and Automator to produce a releaseNum once at the beginning of a release which is used throughout. Also the ProductProcessing table is now reset at the beginning of a release. -- I have also been working with RSC to produce functions to select data in a particular pointing. This is almost complete. I have made a modification to the CU3 ingestion to sort out Provenance or reprocessed stacks when the tiles have not been reprocessed and I have fixed a small bug in CU6. RSC reported: -- Various modifications to CU19 to support (i) VISTA world releases from existing proprietary release databases that only include data associated with the specific fields released to ESO (ii) insertion of GPS DR8 into the UKIDSS DR8 database such that the reprocessed metadata replaces all existing GPS data only whilst other survey metadata remain as before (iii) performance improvements under the experimental new separate curation-server-side ingest share drive. MAR reported: -- Worked on UI so that it will work with VSA public releases. Survey Data Release: RPB reported that a VMC database, including P87 data, had been released to the consortium. VISTA public releases. VIDEO - RSC raised an issue over whether the contents of the ESO DR1 release matched the planned VSA public release. ACTION: RSC to provide details of the ESO VIDEO DR1 release compared with the proposed VSA release. ACTION: RGM to liaise with Matt Jarvis over VIDEO public release. VMC - scheduled for April to account for v1.1 content of the VSA VMC DR1 release. VVV - still processing/preparing DR1 database. VIKING - As yet no response from PI with regard to DR1 content. NJC volunteered to phone Will. VHS - a DR1 database had been released to the consortium for checks. The planned world release date is 18th February 2012. RGM had set up the TWiki page for ideas on publicising VSA public releases. The release of GPS into UKIDSS DR8 and associated metadata fix was expected to finish in about one week. Non-survey Data Release: Nothing to report this time. MAR suggested that it it would be good to try and slot in some non-survey releases during quite periods on ramses1. Astrogrid deployment & Data Analysis services: Nothing to report other than the euorvotech.org domain had been lost resulting in some broken links on the interweb. Miscellaneous: NJC reported that an ESO workshop on "Science from Next Generation Imaging and Spectroscopic Surveys" was scheduled for 15-18th October 2012. The announcement mentioned VISTA, VST and GES so appeared right up our strasse, volunteers please. -- Scanned by iCritical.