From nch@roe.ac.uk Fri Oct 23 17:22:55 2009 Date: Fri, 23 Oct 2009 14:59:56 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting mins 23/10/09 Minutes of WFAU VDFS Science Archive meeting: October 16th 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: PMW, RPB, NCH, MAR, RSC, MSH Apologies: JPE, JMS, RGM, ETWS, AL, NJC DoNM: 10am, Friday October 30th 2009 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: NCH to communicate with some VISTA PS and the SV PIs to finalise some schema details. Discharged; reply received from VIDEO so far. Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review the VSA metadata schema in trunk/sql SQL script VSA_MultiframeSchema.sql ACTION: ETWS to check for corresponding essential FITS keys for VSA ingest against those defined for the WSA Continues; awaiting the latest nightly pipeline products from CASU; NCH is visiting Cambridge next week and will try to touch base with the folks to get an update. ACTION: RPB: backup all DB-driven products after UDS DR5. Continues; this can now be done; the team agreed that all science image products (diffs, mosaics, stacks) should be backed up from /disk*/wsa/products/ and RSC asked that /disk05 is included for various bits and pieces. Migration of public DBs to the cluster servers takes priority, so this has had to be delayed (see Hardware below) Actions carried forward from 16/10/09 meeting: ---------------------------------------------- ACTION: NJC to email Terapix to ask about support for RICE compression in S-Extractor Continues Specific points and new actions: -------------------------------- Project management: Nothing new to report this week. WFCAM & VISTA updates: PMW noted that JPE noted that SV started on schedule this week. NCH and ETWS received a note on some minor reset anomaly and read noise issues from JAC via MJI, since WFCAM has gone back on UKIRT (NCH forwarded to MAR in case of any fall-out in QC). NCH noted that communications have been received from VIDEO and VHS PIs, with collaboration TWikis being set up etc. Clearly things are beginning to ramp up with VISTA, and it is very important that WFAU is ready to react quickly once the processed SV flat files are available. NCH asked that we go through, as far as possible, a dummy run of setting up the VSA ingest DB, image metadata mirror and UI to make sure this can be done without too much fuss. ACTION: NCH & RSC to update the VSA schema and install ramses2.VSA ACTION: RSC to check out metadata data mirror production for the VSA ACTION: MAR to continue to shake down VSA user interface for serving SV programmes Comments and issues arising from CASU minutes: WFAU eagerly await new VISTA processed products to shake down things at our end... we hope to get some prior to being suddenly presented with the SV data proper ...! Networking: Nothing of note this week. WSA Operations: RPB noted that 09B transfer ops are up to date save a couple of days. Hardware and Systems: RPB reported that transfer of operations to the cluster can take place next week (ish) once all the public-facing DBs have been copied over and are installed. Everyone agreed that we might as well do this asap since it will simplify operations. Software: RSC noted: - Modified CU3 to propagate unfiltered image deprecations into detector frame entries following manual fix to the WSA and release databases. - Continued regression tests on NJC's changes that were released into the release_12 branch: Fixed shallow non-survey curation and a few more minor bugs. - Verified all Python code in src/curation/* with pyflakes/epydoc & pydev. Moved scripts used for one-off fixes into src/fixers and those only required for testing into src/testers, to keep the helper script collection down to a managable level. - I'll commit the software changes required for cluster operations into trunk ready for a new release branch next week when we intend to move the WSA over to the cluster. Survey Data Release: Nothing new to note this week. Non-survey Data Release: MAR noted that flat file access is now up-to-date as possible given the availability of data and existing non-survey registrations. RSC noted that 09A survey-like DB releases await CU4 ingests of the catalogue detections for the semester, and that these can commence once ops are transfered to the new cluster system. Astrogrid deployment & Data Analysis services: MSH noted an annoying security breach recently on the VOTech TWiki server, and this has precipitated the need to rebuild the affected system. At the same time, it has been noted that the old web server thoth is somewhat out of date as regards security updates to it's linux kernel, so he is expediting the set-up of the new web server to replace asap. MSH reported that something strange is afoot in the DSAs serving UKIDSS data, with a user down south having problems accessing. Also, publication of UKIDSSDR6PLUS awaits the appropriate metadocs for the DSA. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================