From nch@roe.ac.uk Fri Jan 30 14:02:53 2009 Date: Fri, 30 Jan 2009 13:48:30 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 30/01/09 Minutes of WFAU VDFS Science Archive meeting: 30th January 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, RPB, NJC, RSC, PMW, MAR, LGR Apologies: JPE, JMS, AL, RGM, MSH (excused to go weight-lifting) DoNM: 10am, Friday February 6th 2009 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: NCH to contact SJW to see if he's interested in 08B going into DR5... Discharged; the answer was an emphatic "no"; MJI also suggested not to do so since the 08B recataloguing is being used as a test of a fix to the Petrosian (and Kron and Hall) flux problem for blended objects. So back to plan (08)A: DR5 will be up to and including 08A as before. ACTION: RPB to put a note on the WSA downtime pages and get MAR to bounce the message to wsa-announce. Discharged Actions partly discharged but continuing: ----------------------------------------- ACTION: ETWS to parse that latest keyword info into the VSA metadata schema files. Continues; NCH noted that he intends to adjust the catalogue template in the light of user confusion regarding *Detection.objID references from *Source. ACTION: NCH to assign U/08B/14 H2/K frames to the GPS in the WSA. Continues; done for V1 08B but will need to be redone after V2 ingest. Actions carried forward from 23/01/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: PMW noted that discussions with STFC this week culminated in the JeS resubmissions for the RG being completed and ratified and that the research council assured us that it can be turned around at their end within a week or so... WFCAM & VISTA updates: NCH noted that JAC have advised that WFCAM is safely back on the telescope, and that the first UKIDSS night will be Sat 31st Jan. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: RPB noted that the UKLight usage agreement is about to expire, and he has handled updating of the request to keep the link active between WFAU and CASU, at the same time updating the contact details of those involved. WSA Operations: CASU have finished already the 08A recataloguing, and ETWS and MAR transfered a small test sample as an independent sanity check at this end - everything seems to be fine, so the full 08A recatalogued dataset will be retransfered and ingested starting next week when the operations servers are back online after the big move. Some discussions concerning the best way to unpick the old data, and also to propagate the 08A QC codes through into the reingested metadata, but on the whole things should be put right quickly. ETWS reported: - Finalised the browser parser for the release of the new non-surveys and created the latest set of webpages. - Tested and implemented changes to CU1 to get the reprocessed data transferred with the smallest impact operations. - Got some scripts from Ani Thakar to create our own split version of the SDSS BestDR7 so we can go ahead with transfers and give it a try as soon as the hardware is in place. RPB noted that an ad-hoc differential WSA backup was performed on Wednesday evening in advance of taking the load server offline to upgrade a couple of RAID arrays and shift the whole lot through to the new server room. Hardware: Server moves commence today, so archive services will be disrupted over the next two days. MSH and RPB request that volunteers should be in by 10am tomorrow to help shift all the units. ETWS suggested that we think about buying some more CPU grunt for later this year when we're handling both WFCAM and VISTA data. The suggestion is to buy a khafre-like system (64 bit, v. large memory, multi-proc) to provide the necessary fire power. Software: RSC reported: "Made some improvements to the running of curation scripts to make reliable background runs less tricky, at least for scos. Even more deep non-survey curation bugs have appeared. So not to get too overwhelmed with all the problems that need fixing I've created a list for them in trac under the new milestone "Improved Deep/Synoptic Curation". The biggest problem is that the calibration of intermediate stacks that form deep stacks in crowded, high source density fields is very inefficient and effectively blocks automated non-survey curation. I've started investigating solutions for some of these problems which are documented in trac tickets. Otherwise I'm continuing to upgrade the software to support the 64-bit system with Python 2.6 and the new mxODBC library." MAR noted that he has reorganised the web site documentation in the SVN: https://forge.roe.ac.uk/trac/WFAUweb/browser/trunk/webpages is now in https://forge.roe.ac.uk/trac/WFAUweb/browser/trunk/webpages/wsa Things in /trunk/webpages/wsa get published to http://surveys.roe.ac.uk/wsa/ while things in /trunk/webpages/common get published to http://surveys.roe.ac.uk/wsa/common and in time http://surveys.roe.ac.uk/vsa/common Survey Data Release: Nothing new to report Non-survey Data Release: RPB and MAR noted that more project processing has been done and PIs will be informed first thing next week for the latest ones, rather than immediately before we switch off all services to move the servers. Astrogrid deployment & Data Analysis services: LGR reported: "Intra-UKIDSS tool now includes luptitudes. The updated version is linked to the interface (added a luptitude parameter option --thanks MAR). All posted on SVN with full docs in README files. Sent emails to beta testers for feedback. Now tracking the source of the %-level discrepancy. Setting ppErrBits=0 improves the agreement for the worst cases, but not in general. Statistical test on DXS intermediate frames to follow." Miscellaneous: RSC noted that he has registered for the "practical semantic astronomy" workshop in Glasgow at the beginning of March. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================