From nch@roe.ac.uk Fri Oct 2 16:01:57 2009 Date: Fri, 2 Oct 2009 12:05:42 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 02/10/09 Minutes of WFAU VDFS Science Archive meeting: October 2nd 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: RSC, PMW, MSH, ETWS, RPB, NCH Apologies: JPE, JMS, AL, NJC, RGM, MAR DoNM: 10am, Friday October 9th 2009 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: NCH to draft an interface control document for VSA to ESO-SAF Discharged; thanks to MJI for immediate and clear (!) response; awaiting detailed response from JPE (and AL if possible...!) ACTION: RGM to email MJI and agree on a form of words along the lines of "things ain't as bad as we thought..." Discharged; several issues arose in the course of this - see below Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review the VSA metadata schema in trunk/sql SQL script VSA_MultiframeSchema.sql ACTION: ETWS to check for corresponding essential FITS keys for VSA ingest against those defined for the WSA Continues; awaiting the latest nightly pipeline products from CASU... ACTION: RPB: backup all DB-driven products after UDS DR5. Continues; this can now be done; the team agreed that all science image products (diffs, mosaics, stacks) should be backed up from /disk*/wsa/products/ and RSC asked that /disk05 is included for various bits and pieces. Migration of public DBs to the cluster servers takes priority, so this has had to be delayed. Actions carried forward from 28/09/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: PMW noted that it's time once again to complete the progress charts and to forward plan Q4'09 (with heavy emphasis on VSA deployment and servicing the SV data): ACTION: NCH & PMW to meet Monday 2pm to flesh out the Q4 plan WFCAM & VISTA updates: Following his communications with MJI, RGM noted that there is a plan to have a meeting at ESO before Xmas to go through the nitty-gritty of all the pipelines, once the SV data have been run through them, with the SV groups and the PSPIs and subject to agreement with the organisers it would be good to send someone from WFAU. Also CASU are planning a meeting in the New Year with the PSPIs, and RGM and MJI have agreed in principle to merge that with the VSA one we had in mind, and to have it held in Cambridge. Comments and issues arising from CASU minutes: No new minutes this week. Networking: All quiet on the network interfaces. WSA Operations: RPB reported: "DR6 progresses well. LAS and GCS are both fully source-merged, reseamed, and neighbour tables have been generated. Next step is the DXS though, since this involves a schema change, a full backup of the WSA will be done first over the weekend. Nearly all external catalogues now installed on the cluster. For now, though, the tape library has been moved back onto Ahmose for the WSA backup." NCH has checked the deblend fix in the DBs, and also done a few quick checks on the GCS and LAS DR6 products, and things look fine. Hardware and Systems: PMW noted that he has placed the order for the thoth webserver replacement this week, not the week before as NCH put in the last minutes. NCH noted that the cluster SSA deployment is building indexes currently, then will be ready to hand over to MAR for final tweaks followed by backup and moving to the nodes on which it is to reside. >From the point of view of VSA deployment, the team noted that node 2 of the cluster is yet to have a working NFS mount set up ACTION: RPB to investigate share /mnt on ramses2 Software: NJC has exhaustively tested the synoptic schema changes for WSA/VSA using DXS DR6 and various other datasets as an example. RSC reported: "I've been investigating the best of way of creating and updating mirrored release databases on the cluster, which is applicable to both CU19 releases and CU22 metadata mirrors. I now have a working implementation for CU19 that creates a release database directly on a given public catalogue server and then copies to the mirror server. Our Python documentation software, epydoc, no longer seems to be supported by its developer. So I've patched the latest version to make it Python 2.6 compatible, which has allowed me to update our online software documentation for the first time in six months." ETWS noted: - Continued the upgrade of compressed image related tools for VISTA. Updated the thumbnail tool to create pages for all versions not only the last. Added a tool that automatically finds JPEGs that might have been corrupted. Survey Data Release: Hopefully still on track for DR6 phase-1 at the end of next week. Non-survey Data Release: Nothing to report this week. Astrogrid deployment & Data Analysis services: Nothing new to report this week. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================