From nch@roe.ac.uk Fri Nov 13 15:35:22 2009 Date: Fri, 13 Nov 2009 15:02:10 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 13/11/09 Minutes of WFAU VDFS Science Archive meeting: November 13th 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: PMW, NCH, MAR, RSC, ETWS, NJC Apologies: JPE, JMS, RGM, RPB, AL, MSH DoNM: 10am, Friday November 20th 2009 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: NJC to email Terapix to ask about support for RICE compression in S-Extractor Discharged; response was that they'd like to implement this but don't have the staff resources at the moment. ACTION: ETWS and NCH to copy up as much VISTA PR data as is available Discharged (once the secret police had sanctioned the release of data to WFAU) ACTION: NCH and MAR to communicate details of proto-VSA access to archive listing etc asap. Discharged; (although the secret police insist that no-one may look at the data anyway) ACTION: AL to email to SJW (and JPE?) about an infrared survey session at NAM'10 in Glasgow Apparently discharged given emails circulating amongst the higher-ups... Actions partly discharged but continuing: ----------------------------------------- ACTION: RPB: backup all DB-driven products after UDS DR5. Awaiting delivery of SCSI card Actions carried forward from 06/11/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: AL has been in ESO this week, and has spoken to ESO-SAF folks concerning issues related to survey products. A common understanding has been reached, but it remains to be seen how things work in practice. PMW and NCH both opined that we should keep our heads down, do what we do and then let the community decide on how best to do their science ... WFCAM & VISTA updates: NCH noted that communications with VIKING and VIDEO PIs have cleared up various questions regarding their respective survey schemas. VIDEO will likely operate in a similar way to UKIDSS-UDS (deep stacks delivered by their team plus SExtractor configurations to run catalogue production at the VSA) while VIKING are happy to run like the LAS (standard CASU source extraction). NCH asked NJC to make appropriate adjustments to the template schemas to close out the developments on value-added half-light radii attributes computed at the VSA end on ingest ACTION: NJC to finalise VSA half-light radii detection table attributes. PMW, ETWS, MAR and NCH have been working closely on the VSA metadata schema with the latest processed files from CASU; the only major issue has been over the need to hardwire a bunch of "rules" to work out which file belongs to which survey (commissioning, SV, survey dry-runs etc.). ETWS has come up with a solution and will test asap. Otherwise, the VSA scripts are in pretty good shape, but RSC noted the indexes need some attention particularly with respect to the calibration programme and metadata indexes for speedy response in the UI archive listing applications. ACTION: NCH to sort out VSA_Indices.sql Comments and issues arising from CASU minutes: Nothing new this week. Networking: Nothing new this week. WSA/VSA Operations: MAR noted a couple missing DBs on the cluster nodes which MSH added, and MSH has also fixed/updated several of the scripts used for cluster ops accordingly (as as well as adding some documention about how/when to use them). MAR noted that VSA web pages are under SVN control at the usual place, and a cron job will be set up to mirror the changes on the web server. NCH asked all to keep an eye open for errors and to make changes to keep the online docs as accurate and useful as possible. Semester 08B WFCAM non-survey database releases continue to be pushed along; RSC noted that issues concerning cross-semester projects had arisen but a method of coping with this has been thought out. Hardware and Systems: Nothing to report this week. Software: RSC noted: "Updated more modules to make use of the new framework to become archive agnostic, notably FitsUtils and Utilities. With help from ETWS we completed successful testing of all scripts affected by these interface changes. More modules and script used by the later stages of curation are still to be updated. Performed regression tests on the trunk software for WFCAM data, which revealed some issues, so we must make sure to have complete and successful regression tests result prior to releasing the software changes for VISTA data. Investigated with NJC the cause of one of the 08B non-surveys failing to release. Turns out that we now have non-surveys with multiframe data shared between several programmeIDs, yet still with detection data in individual detection tables, which is not the intended design of our database model and so is not supported by our software. This can be easily fixed by moving the detection data into a single table, and can be included in the automated curation procedure once the VISTA development pressure is off." NCH asked RSC to start to review the quality error-bit flagging process with respect to those conditions as might be appropriate to VISTA data. ACTION: RSC to review quality bit flagging for the VSA Survey Data Release: No news this week. Non-survey Data Release: NJC noted (after the meeting) that STH has enquired about a synoptic survey database release of the standard star calibration data. NJC plans to check over the performance of the all-new process and then hand over to ops to produce a release DB asap. Astrogrid deployment & Data Analysis services: MSH reported "Testing on the latest DSA version and it's new features are about finished. TAP access will be available for all our datasets once Mark T returns from the IVOA interop and releases an updated version of VODesktop (the current TAP client interface is broken). Also included in the new release will hopefully be a client for secure conesearch, as the current VODesktop lacks this. OGSA-DAI/ADMIRE testing continues with some updated libraries they have provided for fixing interaction between their services and STILTS." Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================