From nch@roe.ac.uk Fri May 5 17:21:34 2006 Date: Fri, 5 May 2006 15:23:34 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , John Taylor , Jim Emerson , Malcolm Stewart , Mike Irwin , Mark Holliman , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly meeting minutes: 5th May 2006 Minutes of WFCAM Science Archive meeting: 5th May 2006 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: JB, MAR, ETWS, NCH, NJC, PMW Apologies: AL, JPE, JMS, MSH, RGM, RSC DONM: 10am, Friday 12th May 2006 Actions discharged this week: ----------------------------- ACTION: JB to see that the WSA homepage is incorporated into the MyEd portal. Discharged; JB has a meeting with the relevant EUCS folks to sort this. ACTION: NCH & PMW to have a VDFS work planning session on Thurs 4th at 2pm. Discharged; see Project Management below ACTION: NJC to make a start on the DR1 UDS. Discharged Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 28/04/06 meeting: ---------------------------------------------- ACTION: JB to review Hardware/OS/DBMS design doc to note areas that need updating and itemise new sections required - CONTINUES ACTION: ETWS and JB to put a TWiki note on the intranet detailing the outcome of the UKLight meeting at NeSC. - CONTINUES ACTION: NJC, RSC, NCH, ETWS & MAR to meet Wed 3rd May, 2pm to finalise arbitrary multi-epoch survey data model. - CONTINUES; rescheduled for Wed 10th, 2pm, Plate Library. Specific points and new actions: -------------------------------- Project management: PMW presented the latest updated project plans and progress charts to the team for their perusal; one or two small additions were requested. Q2 work includes support for the survey Data Release 1, and Q3 work includes a good chunk for document preparation for the putative UK VDFS review in October. NCH reported back from the UKIDSS survey heads telecon of the previous afternoon. The two major issues relevant to the WSA team were a review of progress towards DR1, and a discussion concerning the data access policy. NCH and MJI discussed the WSA needs in terms of reprocessed 05A data and delivery of enhanced toolkit codes - see Survey Data Release below. There was a lively discussion concerning current access modes and data release policies, with CASU making a good case for faster release of the pipeline processed flat files in advance of, and more frequently than, the large prepared survey DB releases. Resource constraints necessitate delay on further consideration of this important issue until after DR1. WFCAM update: WFCAM is back on the telescope for the next observing block: May/June/July. Comments and issues arising from CASU fortnightly minutes: No new minutes as of this morning. Networking: JB reported that transfer of the reprocessed 05A data has commenced, following successful tests of ingestion of the current generation of FITS files, including automated deprecation of the previous versions (tested and debugged by ETWS and NCH). JB also reported that he is preparing a UKLight submission document, in collaboration with the NeSC folks and PSB. WSA Operations: NCH noted that a small hiccup in the ingestion procedure was identified in the course of QC, where the data for the 25th October had been ingested twice. JB reported that the offending data modification events had been successfully rolled back and that everything would be back to normal at close of play today. NCH noted that procedures had been put in place to prevent this from happening again. QC and JPEGing operations have been suspended for the last two days while these fixes were taking place. JB noted also that hiccups on the 11 Dec and 17 Jan were also fixed, and that as a result of all this activity since the most recent ingest DB backup, a full backup was scheduled for overnight next Monday. JB also noted that as this is the beginning of the month, he would be taking the opportunity to reboot the load server ahmose during the nominal "at risk" period on Monday morning. NCH noted that QC of 05B was progressing rapidly until the ingest snafu was spotted on Wednesday. Hardware: NCH noted that a request had been received from MSH concerning accomodating up to 4U's worth of rackmount chassis for new Astrogrid servers in the WFAU racks; JB gave this the all clear, noting that there should be enough room for khafre as well, but also that the UPD may not have sufficient capacity for the AG boxes (but this is no problem as the AG boxes are not expected to be "mission critical"). Software: NCH reported continuing work on CU7 procedure and code. ETWS reported writing further scripts for ingest quality control, and also for collection and visualisation of statistics in curation logs. MAR reported some work on logging of image cut-outs usage. Survey Data Release: Reiterating the DR1 preparation schedule as it currently stands: DR1 release at: 14/7 (NB: databases to be DR1 & DR1PLUS !!) Copying/transfering/ 7/7 backups etc. start UKIDSS pre-release 1/7 checks start Final CUs: 16 (incl. 23/6 SDSS DR3 hopefully), 18,19 start CUs 2,3,4,7 for DXS 20/6 and UDS start Archive QC2 starts 14/6 CU7 for wide/shallow 7/6 surveys starts The indefatigable MJI has undertaken to tweak the CASU source extractor for oversampled images (for the UDS) and hopes to have a test catalogue ready for mid-June; WFAU already have a copy of the latest stacking code that includes the necessary floating-point option for output images (needed for the DXS and UDS). A further enhancement for seeing-weighted stacking is in progress, as is the robustified sky estimation for non-integer data with noise of order 1 ADU or less, but the timescale for delivery of these tools is uncertain at present. Finally, 05A data reprocessing and transfer is progressing well: pipeline processing is complete to mid-May, and the complete reprocessed dataset should be available mid- to late-May. NCH suggested that NJC liaise directly with CASU concerning the toolkit codes. ACTION: NJC to liaise with MJI & JRL concerning obtaining the latest versions of the CASU toolkit codes. Non-survey Data Release: NCH and JB noted receiving a few enquiries (some via JAC) concerning non survey data this week. Astrogrid deployment: Nothing new to report this week. Miscellaneous: Nothing else this week.