From nch@roe.ac.uk Mon Nov 15 20:46:41 2010 Date: Mon, 15 Nov 2010 09:55:14 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren , Tom Kerr Subject: WFAU VDFS Science Archive project meeting mins 12/11/10 Minutes of WFAU VDFS Science Archive meeting: November 12th 2010 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RSC, ETWS, MAR Apologies: JPE, AL, RGM, KTN, MSH, NJC, RPB DoNM: Friday November 26th in the Vista Hut Actions discharged this week: ----------------------------- ACTION: ETWS to review external survey updates/additions Discharged; on a related issue, NCH has been in communication with CDS concerning including the latest world release of UKIDSS in Vizier, along with the legacy SSA dataset. ACTION: RPB to make UKIDSS DR5 (sans GPS and possibly UDS) world readable Discharged; MAR needs to mod the interface to protect the UDS. Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 29/10/10 meeting --------------------------------------------- ACTION: TBD to draft and MOU on behalf of PSPIs for ESO-SAF deliveries Continues; on hold until at least next PSPI teleconf, and possibly until "Phase 3" workshop at ESO. ACTION: NJC to suggest workshop at next PSPI meeting. NJC noted that he needs to liaise with JPE about the next teleconf. Specific points and new actions: -------------------------------- Project management: Nothing new this week. WFCAM & VISTA updates: No updates this week. Comments and issues arising from CASU minutes: The team noted the minutes of the meeting of 27th October. ACTION: ETWS/RPB to update WSA/VSA schemas for new keyword NICOMB The team noted the description of the shallow tile photometric non-uniformity issue, and now understands a) the reluctance to release tile SW to us, and b) RGMcM's new requirement to have both paw-print and tile-based band merged products! We will watch developments closely... Networking: Nothing new to report. WSA/VSA Operations: ETWS noted -- Transferred, created JPGs and ingested the metadata for September for WFCAM as well as for VISTA. Catalogue processing is under way and should be finished beginning of next week. -- Mainly worked on downloading external catalogue data and writing schemas and software to preprocess it. Hardware and Systems: ETWS has updated the Starlink software on khafre ... ACTION: RSC to test updated Starlink on khafre ... and plans to roll out the change on the other curation servers next week subject to successful test. ETWS noted that we need to keep an eye on the disk space situation - likely have 3 months worth remaining for WSA/VSA so need to think about an early Xmas present (i.e. 100 TB NAS box) for ops to play with. The team noted that 2 new DB cluster nodes plus a beefy curation server have been delivered and are in the process of being set up. MAR noted an instability in the tomcat web server seshat caused by dodgy AST C/Java implementation (memory leaky code apparently). MAR is trying to work around the problem by rewriting the offending bits of code. Software: RSC noted: "The post-CU4 curation software updates to handle the new split ingest database infrastructure are on hold pending the finalisation of the location of the VSAVVV database on the new cluster hardware and more importantly a decision on the exact curation procedure regarding whether or not automatic metadata synchronisation using SQLServer functionality is a practical solution. NJC has been hard at work fixing yet another performance problem discovered in CU6 that was holding up the releases of non-surveys with high density fields. I've profiled the new solution which appears to be very effective in this case and should not result in any regressions to previous bug fixes. However, we won't be 100% certain of its VISTA performance until the next survey release curation runs are performed owing to difficulties in benchmarking the current software against the previous releases due to schema evolution." NJC reported poor performance of a standard query in UKIDSS DR8 DXS variability tables earlier in the week. MAR and RPB traced the problem to missing column statistics in the release DB; it turns out that combined stats (as are presently being computed) are not sufficient so individual column stats runs are needed. RPB has tested the likely speed issue of such a change, and it seems feasible that this can be implemented as standard in release DB script CU19 (not tried on the gpsDetection yet though!). NJC also noted permissions issues with some user-defined functions in some databases. ACTION: NCH to ask RPB to look into the perennial functions permissions problem. MAR has been working on a new batch thumbnail service that allows up to 10,000 individual FITS images to be generated as a quicker scale-out version of the existing multi getImage service. Survey Data Release: Nothing to report this week. Non-survey Data Release: RPB and MAR have prepared a combined non-survey project, available for world access, in a Kepler field at the request of Phil Lucas. Astrogrid deployment & Data Analysis services: Nothing this time. Miscellaneous: Nowt else this time. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ============================================================= -- Scanned by iCritical.