From nch@roe.ac.uk Sun Jan 18 10:06:31 2009 Date: Fri, 16 Jan 2009 15:55:23 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 16/01/09 Minutes of WFAU VDFS Science Archive meeting: 16th January 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, RPB, NJC, RSC, PMW, MSH, LGR, RGM, MAR Apologies: JPE, JMS, AL, BCW DoNM: 10am, Friday January 23rd 2009 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: ETWS to check out availability of SDSS-DR7 split over several files/groups/volumes Discharged; no multi-volume version available, so the team has decided to work with the monolithic version which will be downloaded from UIC when the new NAS box is installed. ACTION: All to comment on the development of the VSA metadata schema as described on the relevant TWiki page Discharged ACTION: NJC to check out the incremental changes in DR7 over DR5 Superceded ACTION: NCH to fold the latest PI requirements into the VSA baseline schema TWiki notes Discharged ACTION: RPB to schedule a full WSA backup this weekend. Discharged; full backup successfully achieved (eventually) Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 12/12/08 meeting: ---------------------------------------------- ACTION: ETWS to parse that latest keyword info into the VSA metadata schema files. - continues Specific points and new actions: -------------------------------- Project management: PMW and NCH noted the necessity to review the Q1'09 plan and to complete the Q4'08 progress charts. ACTION: NCH & PMW to meet Monday 12pm for VDFS planning WFCAM & VISTA updates: As most are now aware, WFCAM will be on UKIRT permanently for the time being. This results in an increase in storage requirements which has been factored into the on-going grant negotiations with STFC. NCH and NJC took part in a VVV teleconference earlier this week with the Chileans. JPE gave a Vista update as part of that meeting, and the Q1 SV has been cancelled with a new ESO SV likely in April. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: 08B data is all up and ingested following network interuptions around Xmas. WSA Operations: RPB reported: LAS DR5 progressing: Source merging finished last night. Reseaming is due to finish Saturday morning. Non-survey releases: The first non-survey databases produced using the AutoCurate script over the Christmas break have been released. Shallow surveys U05A37, U05B7, U06A48, U06B36, and U07B36 are all done. Unfortunately the deep survey U06BH38 had problems. This may or may not get fixed depending on workload. Hardware problem with LTO3 tape drive on djoser. Thankfully John found another tape drive lying around and has attached this instead. Even more thankfully the tape drive is still under a maintenance contract, so Eclipse have been informed. Pre-Christmas problem with timestamps on NAS boxes is now fixed. Turns out all NAS boxes were querying Thoth for an ntp signal, though thoth was not running ntp. Ntp has been restarted on thoth. ETWS noted: -- Transferred and ingested reprocessed UDS 2005 data. -- Finished transfer and ingest of 08B data. -- Enhanced CU2/CU4 to process data when the date directory is split on several disks. NCH noted that he needs to assign the H2/K frame pairs from U/08B/14 to the GPS, as previously agreed with GPS folks. ACTION: NCH to assign U/08B/14 H2/K frames to the GPS in the WSA. MAR and NCH noted that they will do some last colour uniformity checks on the LAS before we launch into crossmatching and then DR5 release. The team noted that the situation over reprocessing is becoming a little complicated (UDS 05B v2, UDS requested by Nottingham, non-UDS in 05B at v2, bug fixes for deblend extended source fluxes, and now it seems GPS survey head may be requesting nebulous region reprocessing for inclusion in the GPS). The suggestion is that NCH speaks to MJI to try to understand where things stand in all this... ACTION: NCH to ring MJI pronto to wish him a Happy New Year Hardware: MSH requested a meeting to discuss priorities over relocating the archive servers from C1 to the lovel new C2 server room. ACTION: MSH, ETWS, RPB, PMW (+NCH?) to meet on Wed at 2pm to discuss the plan of attack for server relocation. PMW noted that the new NAS box should arrive towards the end of next week, and the team agreed that it should be installed in C2 from the outset rather than in C1 and then relocated. Software: RSC reported: "I've been resolving lots of minor software issues identified from the over-Christmas non-survey processing run on 100+ programmes. Almost all were successfully curated but still this run did a good job of determining the limits of the current software with so much variability between the programmes' observation strategies and with the non-survey data not receiving the same level of quality control attention as the UKIDSS survey data. As well as improving the usability and automation of the release script, I've been working with NJC to identify improvements to the curation process of the deep stacks." NJC has made various bug fixes and enhancements to the non-survey and intermediate stack recalibration software; MAR has been working with LGR on integrating new features to data analysis tools into the interface and also non-survey support. Survey Data Release: DR5 is nearly ready... LAS has source merged, but checks after this meeting had flagged up a possible problem in 08A J band fluxes at faint mags. MAR, NJC and NCH are investigating prior to running crossmatching and final release scripts in case some additional QC (or, heaven forbid, bug fixes) are needed. Non-survey Data Release: All non-surveys in 08B now have flat-file access to pipeline processed products; final tweaks to the user interface need to be made prior to release of a large tranche of non-survey databases to PIs. Astrogrid deployment & Data Analysis services: LGR reported: "Intra-UKIDSS photometry tools (interactive and list-driven) now work on LAS, GCS, GPS, DXS (thanks MAR for upgrading the interface). As in the normal catalogue, no extinction or aperture corrections are applied to the DXS. While only aperMag3 is listed in the interactive version, info on all aperMags are included in the list-driven tool. Added note on mag relations for clarity. Now trying to implement luptitudes with user-controlled softening parameter." Miscellaneous: Nothing else this week ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================