From nch@roe.ac.uk Tue Dec 1 17:36:20 2009 Date: Fri, 20 Nov 2009 15:16:42 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting mins, 20/11/09 Minutes of WFAU VDFS Science Archive meeting: November 20th 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: PMW, NCH, MAR, RSC, ETWS, NJC, MSH, RPB Apologies: JPE, JMS, RGM, AL DoNM: 10am, Friday November 27th 2009 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: NCH to sort out VSA_Indices.sql Discharged (turns out there were some bugs in there that needed fixing). Actions partly discharged but continuing: ----------------------------------------- ACTION: RPB: backup all DB-driven products after UDS DR5. STILL awaiting delivery of SCSI card; RPB suggested we cannibalise ahmose's now that the tape drive has been shifted to the cluster. Scrapheap Challenge here we come. ACTION: NJC to finalise VSA half-light radii detection table attributes. Continues; almost done; some discussion regarding units of these attributes and it was decided to stick with instumental (i.e. pixel) units for consistency with other similar attributes in the standard catalogue list. NJC will also add in an error attribute. ACTION: RSC to review quality bit flagging for the VSA Continues; RSC has started to look at this, and now awaits some SV catalogue ingests to check out the details. Actions carried forward from 13/11/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: Nothing of note this week. WFCAM & VISTA updates: Thanks to CASU for supplying example VISTA tiles; the inevitable exchange of emails ensued concerning various header issues, but we think we can see our way through them now ... Comments and issues arising from CASU minutes: Nothing new this week. Networking: Nothing new this week. WSA/VSA Operations: ETWS noted that he had transferred the DR6 GPS reprocessed data. RPB noted: "Fixed the DXS problem that came up just before I went on holiday. The DXSvariability table should now be correct in the WSA. It was decided to fix DR6 by simply removing the dodgy data from the DR6 copy on the load server and awaiting the next phase release of the GPS data to automatically fill in the correct data. CU3'd the reprocessed GPS data and ingested successfully. On MAR's request, altered UKIDSS databases on the cluster to grant "SHOWPLAN" access to wsaro user and (where appropriate) worldwsaro. Finished copying the fullSSA database to the cluster. Will attach as read/write initially so MAR can check the HTM functions are working properly before it goes live." Hardware and Systems: RPB noted that the tape library system is now on the cluster and there have been some issues concerning migrating the BackupExec database, but this is sortable. MSH asked about which of the old non-clustered servers should become the test DB server for development, and after some discussion it was agreed that hatshepsut should have this honour. Software: RSC noted: "Continued development of the archive agnostic framework for the software, updating more modules including FitsToDb (used to update Provenance) & QualityControl (used to update detection quality flags) to become archive agnostic. Replaced hard-wired deprecation code values with a new class of system constants and migrated more archive-dependent database constants over to the archive-agnostic system constants module. Updated VSA database design definition to allow TestVSA databases to be created. Started implementing Python unit tests in the form of doctests that are invoked during the documentation builds with some example cases in the Utilities module only at this stage." ETWS noted finishing the upgrade of CU2 and 3 for non-tiled VISTA data. Survey Data Release: NCH noted that a user had reported some weird "missing statistics" warnings popping up in the freeform SQL results pages. Turns out that (at least) DR6+ had this feature, the causes of which are unknown. Results are not affected, but inefficient query plans may occur for some queries, so we need to keep an eye on this, and make changes in CU19 if necessary. Non-survey Data Release: A dozen 08B non-survey prepared DB release have been created, and will be released to the respective PIs as soon as the interface is switched to speak to the cluster where they now reside. Astrogrid deployment & Data Analysis services: MSH reported problems with the latest DSA (TAP-supporting) release from the AG suite, which is a shame since the IVOA finally approved TAP last week and world-wide VO users will likely require this facility from the WFAU VO services. MAR and ETWS have been investigating some problems reported by a user when making many list-driven measurements in one WFCAM frame using the online service. The problem seems to be somewhere in the Python wrapping code of imcore_list (or possibly within that code itself) but more investigation is needed, and the VSA shake-down is soaking up all resources at present. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================