From nch@roe.ac.uk Fri May 11 16:12:27 2007 Date: Fri, 11 May 2007 16:05:38 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , John Taylor , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Mark Holliman , Peredur Williams , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 11 May 07 Minutes of WFAU VDFS Science Archive meeting: 11th May 2007 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, PMW, RSC, MAR, ETWS, JB, LGR Apologies: JDT, BCW, JPE, AL, NJC, MSH, RGM, JMS DONM: 10am, Friday 18th May 2007 in the Plate Library Actions discharged this week: ----------------------------- ACTION: NCH to book a projector for the day Discharged ACTION: PMW to book buffet lunch and coffee for appropiate numbers. Discharged Actions partly discharged but continuing: ----------------------------------------- The following from last time partly done but continue: o) Add in a default row for every detector appearing in every detection table (for schema consistency when querying merged sources and individual detections) - ACTION: RSC & NCH Continues ACTION: Technical focus group to meet after next week's project meeting to sort out the general recalibration problem. Continues after this meeting. Actions carried forward from 4/05/07 meeting: ---------------------------------------------- The following from last time continue: ACTION: NCH to discuss and review AstroWISE interfacing with MAR and JDT. ACTION: NCH to start some hard-nosed negotiations with the UKIDSS PI and CSS concerning the contents and timing of DR3. - CONTINUES; started at this week's UKIDSS Survey Heads meeting; NCH refused to be drawn on any committment to DR3 schedules until it becomes clearer what is to be included and when the early 06B processed data will be available. In the meantime, UKIDSS Survey Heads are to look at when their project observations were taken during 06B, and communicate a wish list for inclusion in DR3 to NCH. ACTION: AL to check on the exact wording of the survey release policy with reference to world release of proprietary survey data. - CONTINUES Specific points and new actions: -------------------------------- Project management: NCH reported back from the UKIDSS Survey Heads meeting held on Wednesday this week. Issues of note: i) World release of UKDISS databases starts with the EDR in July; since the UDS catalogue table is empty a request to include a link to the Nottingham catalogue pages was made. ii) DR3: NCH refused to be tied down to a timescale; Survey Heads are looking into when 06B observations were taken to help in making any decisions regarding subsets for inclusion in DR3; SJW asked that regular updates be given to the Heads when a new batch of data is ingested (see also VDUC below); iii) SJW also asked that NJC be given time to help in assembling publicity material. NCH reported back from VDUC held yesterday. The issue of list-driven photometry arose as part of various discussions, and MJI noted that JRL is now working on the CIR toolkit implementation. NCH agreed that better communication channels concerning ingest status would help (for example) UKIDSS survey heads feed back comments on the latest process versions/data to CASU. ACTION: ETWS to progress his "ingest status" browsable pages as quickly as possible with a view to semi-automation of eyeball checks on survey data. Much useful discussion on deprecated and additional requirements in the URD took place, which should all be helpful in our development extension bids in the current Rolling Grant write-ups. Finally, PMW and NCH reported that recruitment of a new developer to displace NCH and RSC design/coding effort from VDFS into GAIA is progressing. WFCAM & VISTA updates: Nothing new to report this week. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: After a brief flash of activity, UKLight is broken. Apparently hardware supplied by UKERNA is unreliable but is being replaced asap. Normal service will be resumed (presumably) as slowly as possible; in the meantime, we continue with JANET... WSA Operations: ETWS reported that transfer of 06B data continues. NCH asked about ingest and emphasised the need to keep things moving along as quickly as possible. ETWS noted that the recent schema update (new FITS keywords) had held ingest up for a day or two, but that otherwise everything should be progressing normally. Hardware: JB noted that a blocked drain in one aircon unit in C1 resulted in things getting a little moist behind the archive servers, but that this was fixed reasonably quickly. JB reported that the LTO-3 drive is now working, and that new catalogue server hatsehpsut is in the racks but cannot be plugged into the mains. Otherwise things are continuing normally, despite the odd RAID disk replacement. Software: MAR reported generated cover image and finished off usage stats for the impending rolling grant proposal. Cover image is based on NJC's WFCAM mosaic image and a few other telescope images and screenshots. RSC reported: "* Fixed a rarely occurring obscure bug in CU7's C++ code * Made additional revisions to CU19 to update non-survey releases with changes made to survey releases * Updated schema with new attributes, and some further development to the schema updater, which JB has now applied to the WSA * Tested curation codes to ensure they work with the new software installation on djoser. Only code changes required were to handle changes to the exception types being thrown by mxODBC in the Python code." ETWS reported: "Finished installation of the software upgrade on sneferu and djoser, now it's up to the test results to install it on the other machines too. Testing, bug fixing and enhancing of CU1 and 2 after including them into the top level data transfer and ingest tool. Survey Data Release: NCH reported that a fruitful discussion with the GPS Survey Head has resulted in a significant pruning of the source table schema that will speed up both curation and end-user querying. In a nutshell, epoch 3 K, epochs 1 & 2 H2, and all extended source flux measures are to be removed from the schema for the next release. ACTION: NCH to remember to ensure the GPS schema is modified as per instructions from the GPS survey head, and Cookbook examples included to recover attributes when needed. Non-survey Data Release: JB noted that after a software fix provided by RSC, a couple more non-survey DBs are being prepared and will be released to their respective PIs next week. Astrogrid deployment: Nothing new to report this week Miscellaneous: The first announcement for ADASS XV has come out; NJC, JB and (probably) RSC will attend.