From nch@roe.ac.uk Fri Mar 12 16:08:57 2010 Date: Fri, 12 Mar 2010 15:24:00 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting, 12/03/10 Minutes of WFAU VDFS Science Archive meeting: March 12th 2010 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, RSC, RPB, MAR, NJC Apologies: JPE, AL, KTN, MSH, RGM, PMW DoNM: 10am, Friday March **26th** 2010 in the VISTA Hut Actions discharged this week: ----------------------------- None this week. Actions partly discharged but continuing: ----------------------------------------- None this week. Actions carried forward from 05/03/10 meeting: ---------------------------------------------- ACTION: RPB & NJC (with help when required from RSC) set up cross-neighbour tables between Stripe 82 and WSA/VSA surveys as appropriate. Continues Specific points and new actions: -------------------------------- Project management: Nothing worthy of note this week. WFCAM & VISTA updates: Some good news circulating about UKIRT: it seems that survey operations will continue until March 2012 (UKIRT operating in "reduced" mode). Comments and issues arising from CASU minutes: No new mins this week; the VSA team are eagerly awaiting V0.8 VIRCam data... Networking: U/09A/2 PI Phil Lucas noted that nearly half of his data are missing from his recently prepared SQL DB. Turns out some kind of glitch in JAC-CASU transfers occured for the nights of 12/13th July 2009. Checks at WFAU and JAC seem to indicate that no other nights at anytime have been affected by the glitch. JAC, CASU and WFAU have undertaken to sort this out as a top priority. It is not known what programmes other than 09A/2 are affected by this (e.g. not sure if a small amount of data have been missed in UKIDSS DR7). Transfer speed investigations will continue next week with khafre on MSH's return from leave. WSA/VSA Operations: ETWS noted: - Re-transferred and ingested 09B files that where re-done by MJI. - Updated the CreateFitsAttributeList code to check all VISTA files in a given date range against the existing schema to spot differences in the attribute's type size and existence of attributes. - Fixed the wrong entries of the illumination correction table name in the WSA and created scripts to be run on the other affected databases. RPB noted that 09B catalogue ingests are complete; RSC and RPB are going to have a final look at CU4 ingests to test if an order-by hint speeds them up even more. RPB suggested that tests of splitting the VSA into individual DBs for survey-based ingest will be done when the next iteration of processed VIRCam data arrives. Hardware and Systems: RPB noted that the new NAS box needs some industrial-strength rackmount gear and this has been ordered from Eclipse. Disk space on ramses1 H:\ seems to be short, and RPB noted that some very old SSA ingest files need to be scrubbed. NCH asked that a backup of the cluster-deployed SSA be done prior to this. ACTION: RPB to backup the cluster-deployed SSA DB. Software: RSC reported: "Reviewed all of the binary file ingest use cases in our software to ensure the modifications to improve the CU4 ingest performance don't have any adverse effects. Identified the possibility of further performance enhancements to CU4 ingests by specifying the ingest as ordered by primary key. Synchronised the software branches in SVN where appropriate and ensured the database schema description files were correctly representing the current databases in both branches. Updated database queries in the software to return Python 2.6 namedtuples in preference to tuples where possible to make certain common code examples more clear and bug-free (e.g. this would have prevented the indexing bug that followed a schema change to the variability tables)." Survey Data Release: MAR asked about DR8 schedule, and noted that eyeballing needs to be done to keep up the momentum. ACTION: NCH to prod UKIDSS SHs concerning DR8 eyeballing. Non-survey Data Release: Lots of 09B survey-like DBs have been prepared this week and are in the process of being released to PIs. It was noted that an enquiry from University of Hertfordshires concerning the transit DB was received earlier in the week; turns out access permissions were incorrectly set (now fixed). However the team noted that this release DB (consisting, if memory serves, of UKIDSS "poor seeing" data as originally used for the planetary transit survey, possibly supplemented by campaign data) is now way out of date. NCH agreed to contact the PIs to see what they might like in terms of an updated release. ACTION: NCH to contact Transit survey PIs to ask about requirements for an updated DB release. Astrogrid deployment & Data Analysis services: Nothing to report this week. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================