From nch@roe.ac.uk Fri Jun 20 17:03:14 2008 Date: Fri, 20 Jun 2008 16:52:39 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 20/6/08 Minutes of WFAU VDFS Science Archive meeting: 20th June 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, NJC, RPB, MAR, ETWS, PMW, MSH Apologies: JPE, JMS, AL, BCW, LGR, RGM, RSC DoNM: 10am, Friday 27th June 2008 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: RPB to backup WSA this weekend Discharged. Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 13/06/08 meeting: ---------------------------------------------- ACTION: RPB to formulate backup plan arising from meeting Continues. Specific points and new actions: -------------------------------- Project management: PMW brought along the progress charts for April/May and the team filled in the completion figures. WFCAM & VISTA updates: Nothing new this week. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: NCH thanked the team for installing and crossmatching SegueDR6 for the UKIDSS LAS but noted that backups should be secured asap. ACTION: RPB to backup SegueDR6 (SQL DB from thutmose and flat-file backup files from hatshepsut) WSA Operations: ETWS reported: - Transferred and ingested 08A April data. JPEGing is still running. - DXS data has been stacked (CU13/14) and fully ingested, including Provenance and QualityBits, so the DXS is ready for source merging (CU7). - Recalibration of GPS has been nearly done (only 05B missing), but data needs to be ingested. - Enhanced curation/operation software as needed. - RPB has ran CU7 for LAS and GCS sucessfully. CU16 (joining with external catalogues) for the LAS has started. NCH thanked the team for all the DR4 progress over the last three weeks; for schedule update, see Survey release below. NCH noted that the GPS folks have encountered some astrometric anomalies in UKIDSS DR3; MJI has been investigating the extent of the problem, and has found 206 files where there are clearly problems in the astrometric solutions (initially the GPS folks claimed at least 1% of the data was wrong). MAR and NCH have had a quick look at the distribution of number of standards versus rms to see how easily such things may be identified in QC1, but this may be too crude a method to identify problems which may only be clearly seen after band merging. In any case, MAR & NCH are taking advice from UKIDSS and MJI on how to handle this as far as GPS-DR4 is concerned. The emerging consensus seems to be that fixing up the data for DR4 shouldn't delay things too much, especially if the number of affected files is small. Hardware MSH noted that new webserver horus is now up and running with a test Debian4.0 that seems to be compatible with 64-bit Starlink plus homegrown C/C++/F codes. The plan is for MAR to finish testing the web apps on this new machine, and then (probably on a Friday in the near future) switch over from thoth. MSH noted that we need to review our hardware requirements and procurments for the medium term; the team decided to hold a separate hardware review meeting: ACTION: PMW, MSH, NCH, ETWS, RPB, RGM to get together next Thurs (26th) at 2pm to review hardware requirements. Software: This week NJC has been improving the logging and progress monitors on CU13 and 14 as well as fixing a few bugs. NJC has also checked the DXS-DR4 deep stacks and catalogues that have been produced. NJC has also been testing the CU6 framework for pipelining the whole production of deep and synoptic data sets and improving the running of this and started looking at modifying CU7 code to merge bandpasses at fixed intervals in time. ETWS noted continuing issues with 64-bit software under Debian4.0 on khafre; MSH reiterated that he has found a workaround using a test release of Debian and he and MAR find that all seems to be well (e.g. with 64-bit Starlink) on new webserver horus. ACTION: MSH to communicate his 64-bit Debian experiences to Horst in ITSG with a view to sorting out khafre. Survey Data Release: The situation is as follows: LAS: currently crossmatching with SDSS. DXS: All stacked, source extracted, jpegged and ingested; just needs the usual provenance, qbits, source merging and crossmatching (i.e. hours rather than days) GCS: source merged; just needs crossmatched. ... and then these three need some final QC/sanity checks. We anticipated phase-1 DR4 release "within a couple of weeks or so". (GPS and UDS are on hold). Non-survey Data Release: NCH reported back from discussions with JAC concerning support for "survey-like" SQL DB releases of non-survey datasets, noting that the key to better support is better automation. This will be discussed further next week ACTION: NCH to remember to discuss non-survey support at a future meeting when DR4 is better under control and RSC is back. Astrogrid deployment & Data Analysis services MSH noted that he will publish SDSS SegueDR6 via DSA as a few people have asked for this. ACTION: MSH (with metadoc help from ETWS if required) to publish SDSS SegueDR6 via AG DSA. MSH also noted that a new secure cone-search application is nearly ready to deploy for the UKIDSS proprietary datasets published via DSA (problems with the original software meant that it was switched off). Miscellaneous: NCH noted the opening of registration for the 18th annual conference on Astronomical Data Analysis Software and Systems (ADASS). The conference will be held in Quebec City at the Loews hotel during 2-5 November 2008. Zut alors. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================