From nch@roe.ac.uk Fri Aug 1 20:03:16 2008 Date: Fri, 1 Aug 2008 12:46:00 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 1/8/08 Minutes of WFAU VDFS Science Archive meeting: 1st August 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RPB, ETWS, RSC, NJC, MAR, LGR, PMW Apologies: JPE, JMS, AL, BCW, MSH, RGM DoNM: 10am, Friday 8th August 2008 in the Vista Hut Actions discharged this week: ----------------------------- None Actions partly discharged but continuing: ----------------------------------------- ACTION: RPB to test out attaching a normal database as an MS SQL "reporting database" in readonly mode on thutmose/hatshepsut RPB noted that some tests with attaching databases as readonly on read-write OS filesystems were inconclusive; in any case, a spare volume on public server thutmose needs to be used to set up a readonly volume for further tests. Actions carried forward from 25/07/08 meeting: ---------------------------------------------- ACTION: RPB to formulate backup plan arising from meeting Continues; currently wholesale new backups running; formal policy to be documented. NCH noted that he has made a TWiki page with information on the legacy SuperCOS dataset backups, and this could be used as a model. Specific points and new actions: -------------------------------- Project management: PMW distributed the progress reporting charts for July which the team filled in during the meeting. WFCAM & VISTA updates: With reference to last week's minutes concerning VISTA test data, NCH noted that a billet-doux has been received from our Leader JPE as follows: "CONFIDENTIALITY CLAUSE You will appreciate that this is 'Technical' data only, and not at all 'first-light'. To ensure that the focus is on engineering issues the data/image/products must NOT be distributed outside the VDFS project team at WFAU/CASU in any way, nor shown in any form. In case of any doubt contact the PI j.p.emerson@qmul.ac.uk. The above is a strict injunction!" Comments and issues arising from CASU fortnightly minutes: No new minutes this week Networking: Nothing of note this week. WSA Operations: ETWS noted - Transferred re-processed 08A data. - Fixed timestamp oddities that occured during ingest of some 08A data. RPB noted that 08A data has finished downloading and flat file access is now available for all dates. Jpegs still have to be created for most of June. NCH, noting that the 08A data is all in except a few JPEGs, expressed the opinion that this is superb performance by JAC/CASU/WFAU in transfering, processing and archiving (for flat-file access of processed file products) all data within 8 weeks of observation. In some cases (e.g. June 08A), data have become available to users within 6 weeks of observation, which is truly marvellous... ETWS noted that this would have been even shorter had it not been for various hardware glitches. Hardware RPB reported: "Takelot has become much more stable since it started using the second ethernet card, so we can safely conclude that the problems were caused by the first card playing up. Eclipse should be coming out on Monday to replace the dodgy card. Initial tests of mounting Read-Only reporting databases have been unsuccessful. The spare volume on thutmose needs to be formatted and mounted for further testing. Due to action that on Monday morning during the scheduled "at risk" period." RPB also noted that the Windows catalogue servers will all undergo a preventative reboot on Monday morning. Server room C1 continues to suffer periodic heat build-up problems, so WFAU have switched off non-essential servers sneferu and thor; horus will be switched off over the weekend. Keep your fingers crossed for snow ... Software: ETWS reported: - Developed CU1 further for better handling of deprecated data and list-driven transfers. - Enhanced CU2 to be able to be run with multiple instances on a multi-processor server like khafre. MAR noted MulitGetImage now allows users to do multiple wavebands at the same time. Otherwise, MAR has been porting old www-wfau web server (i.e. SSS/SHS & 6dF stuff) to the new web server horus. NJC reported redoing figures for the synoptic paper based on DXS-DR4 and adding in code to give more useful astrometric statistics. Survey Data Release: Regarding GPS DR4, ETWS noted that there remain a few odds and sods (specifically "_fud" files from CASU) that need to be retransfered and reingested before launching into the curation processes prior to release. RSC reminded everyone that we must not forget to finish off the quality error bit flagging once all the data is present and correct in the archive. MAR also reminded the team that the global QC1 needs to be applied as well. NCH noted that retrospective QC1 needs to be done to close a few reprocessing and other loopholes. Non-survey Data Release: MAR noted that all registered 08A nonSurvey programmes now have flat file access. Astrogrid deployment & Data Analysis services PMW and LGR noted that preliminary testing of the optical/IR image analysis tool is now complete, and that a simple web interface should be constructed to allow some external beta-testers to give it a run. ACTION: MAR and LGR to get together Tues (am) next week to sort out a web interface for the new pixel analysis tool. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================