From nch@roe.ac.uk Fri Jul 2 16:32:31 2010 Date: Fri, 2 Jul 2010 16:26:03 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren , Tom Kerr Subject: WFAU VDFS Science Archive weekly project meeting minutes, 2/7/10 Minutes of WFAU VDFS Science Archive meeting: July 2nd 2010 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: ETWS, RPB, NJC, MAR, MSH, NCH, RGM Apologies: JPE, KTN, AL, RSC DoNM: Friday July 9th in the Vista Hut Actions discharged this week: ----------------------------- ACTION: RPB/KTN to order several crates of LTO-4 tapes. Discharged ACTION NCH to decide on frame set tolerance for VVV source merging as their is a disparity between the documentation and the database. Discharged by RPB ACTION: NCH to ask MJI for a code snippet just so that we can be sure we implement at the archive end exactly the recommended CASU algorithm for saturated flux recovery Discharged; thanks to MJI for the response (implementation to be done by ETWS/NJC - see Software below) ACTION: MAR to add a new web page detailing the main points of difference when access the ingest VSA as opposed to static survey release DBs Discharged Actions partly discharged but continuing: ----------------------------------------- ACTION: RSC & NJC to document new operational procedures on the TWiki Continues (nearly done - NJC needs to add some new UDS/SExtractor things) ACTION: ETWS to try some test ingests of VIRCam tiles. Continues; small changes will likely be necessary when the finalised version 1.0 products are delivered. All noted also that provenance issues and QC scripts to auto-deprecate any incomplete tile products will need close attention once tiles are successfully ingested. Testing of new keyword implementation is also ongoing. ACTION: ETWS to enhance the early CUs to cope better with CASU's process versioning Continues; all done bar the testing (a few minor automation issues in the transfer) ACTION on NJC to test SExtractor on shepseskaf and finish off CU13 upgrades Continues; ETWS noted this should be tested on menkaure also. Actions carried forward from 25/06/10 meeting --------------------------------------------- ACTION: TBD to draft and MOU on behalf of PSPIs for ESO-SAF deliveries Continues; RGM volunteered to email F. Comeron at ESO-SAF to get the ball rolling. ACTION NJC to suggest workshop at next PSPI meeting. ACTION: ETWS to enhance the schema browser parser to include a distinct "VSA QC" entry that excludes objects not accessible to any, but includes views (e.g. vmcMultiframe etc.) and details of the partitioned detection table schema as employed in the ingest DB. Continues; RPB needs to create a views script from the definitions in place in the VSA DB, and also needs to pass over the list of accessible objects. Specific points and new actions: -------------------------------- Project management: Nothing news-worthy this week. WFCAM & VISTA updates: No updates this week. Comments and issues arising from CASU minutes: Lots of help/input and new mins (of meeting of 30th June) from CASU this week: Thanks to MJI for code snippet for the saturation calibration recovery and also for the info concerning low-level photometric corrections to tile products in TAN projection: ACTION: ETWS to check that distortion correction is switched OFF for VIRCam tiles in TAN projection in CU4 ACTION: NJC to check that distortion correction is switched OFF for VIRCam tile in TAN projection in CU13 ETWS noted that he has been communicating directly with MJI/EGS concerning the prototype tiles, hence the apparent lack of feedback to JRL on these. Finally the team noted the promise of the availability today of new version 1.0 VIRCam products up to 23rd April today - great stuff! Networking: NCH noted that MJI has outlined CASU's wait-and-see policy on pushing VISTA pipeline standard products to ESO-SAF on behalf of the PSPIs, (the worry being of course that we don't want to do it from both ends). RGM noted that he's emailed F. Comeron to get the ball rolling on the coordination of ingests. WSA/VSA Operations: RPB reported: "VVV pairing radius now set to 1 arcsec. cyg0B7 uber-nonsurvey causing various problems, largely as a result of being the first of this kind of beastie we've ever attempted. Slow and steady wins the race. VSA users and views created to allow access to the ingest database via the gui. Will script all this stuff to replay next time we recreate the VSA from scratch." ETWS noted -- Tested transfers for the non-integer version numbers. -- Updated monitor pages to reflect this latest changes to non-integer version numbers. -- Updated code to set and get MultiframeIDs and file timestamps from the extension in case of tiles. Hardware and Systems: MSH noted that there is a little confusion over the site power-outage scheduled for last Tuesday; it seems Scottish Power may have okayed the existing system after all and no downtime was necessary. NCH asked RGM what was the status of the WFAU hardware part of the "sustainability" pot bid. (RGM looked into this immediately after the meeting and cleared up exactly what has been asked for from where in IfA/School funds). ETWS noted that the most recent PyFITS seems to have a nasty memory leak again, so the default version has reverted to the old one. Software: The team discussed implementation of the saturation correction technique, and it was agreed to apply to the apermag calibrations at ingest for both WFCAM and VIRCam data, according to the prescription detailed by MJI. ACTION: ETWS (with help from MJC where necessary) to implement aper mag saturation correction in CU4/exnumeric NJC noted that he's been speeding up one of the slower parts of CU6 using Tamas Budavari's "half-space" methods for efficiently evaluating image footprints. MAR noted that a VSA user had reported weird behaviour in the interface cross-id service that has been traced to the underlying HTM DLL provided by SDSS/Hopkins. Some suscpicion that we need the latest version (not yet available) for SQL 2008 and the Windows server OS - an email has been sent. Survey Data Release: MAR reported steady progress with DR8 QC; UDS DRx (where x is the subject of debate elsewhere) is still TBD but the team noted CASU's minute regarding likely reprocessing of the H-band data. Non-survey Data Release: RPB and NJC noted that a WFCAM mega-non-survey (i.e. 4 non-surveys combined) was giving a few headaches but is nearly there; apparently another user has requested such a beast for their data also, so it is felt to be a worthwhile exercise. Astrogrid deployment & Data Analysis services: MSH reported a new Finnish user of secure VO services. Miscellaneous: Nowt else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================