From nch@roe.ac.uk Fri Oct 3 23:30:05 2008 Date: Fri, 3 Oct 2008 17:00:33 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDSF Science Archive weekly project meeting minutes, 3/10/08 Minutes of WFAU VDFS Science Archive meeting: 3rd October 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RPB, RSC, NJC, LGR, ETWS. MSH, MAR Apologies: JPE, JMS, AL, BCW, PMW, RGM DoNM: 10am, Friday 10th October 2008 in the Vista Hut Actions discharged this week: ----------------------------- None this week. Actions partly discharged but continuing: ----------------------------------------- None this week. Actions carried forward from 26/09/08 meeting: ---------------------------------------------- ACTION: NCH to break everything by deprecating the loadServer DbConstant so that curations codes can be generalised to work for both VSA and WSA... but not until the WSA non-survey SW has been rolled out. - continues Specific points and new actions: -------------------------------- Project management: NCH and PMW assembled a Q4'08 plan on Thursday, and NCH asked the team to review this and make any changes. The team also completed the progress charts for Q3. WFCAM & VISTA updates: Nothing new to report this week. Comments and issues arising from CASU fortnightly minutes: The team noted the minutes of the meeting of 24/9/08. Concerning the new keyword for summit rejected MSBs, just for clarity, will it look like this: MSBFUD = T / Observation ****ed up in some way with stuff that's OK having the same except of course F in place of T and no comment, or MSBFUD = 'Observation ****ed up in some way' / CASU added-value keyword with stuff that's OK not having the keyword present?! Either is usable (suspect the former is easier for us). To answer the query concerning an after-processing flag, we would not use this for survey data as the UKIDSS survey heads tend to make little adjustments to the accept/reject boundaries given the data obtained, rather than sticking to the requirements specified in the observing constraints. For non-surveys we could propagate into a new deprecation code and subsequently ignore the data in curation processes if folks upstream think this would be worthwhile, but again suspect this may annoy pragmatic PIs when marginal data gets excluded from, e.g. archive-end stacking or source merging... opinions welcome. With reference to naming conventions, WFAU intend to use the convention VSA=VISTA Science Archive, OSA=Omegacam Science Archive, in line with WSA=WFCAM Science Archive if that's any help... Networking: Nothing of note this week. WSA Operations: RPB reported noted that CU19 crashed when attempting to build the GPS DR4 detection table. NCH suggested that he, RPB and RSC look into this immediately after the meeting. NCH noted a conflab between Phil Lucas, Chris Davis and ETWS after the latter reminded us about H2-K difference images in the GPS. Turns out the H2 data were taken under a different project for 07B (and will be for 08B) so these will be assigned to the GPS project in the archive for DRs 5 & 6 in order that they appear (it was agreed with Phil that it's too late for DR4). RSC asked that we try to close out the non-survey rearrangements in the ingest DB today since the GPS release has stalled. ACTION: NCH to make ProgrammeFrame assignments for all non-survey projects Hardware The power down of the server room last Sunday took place, and then all hell broke out. First of all, the aircon units all failed to restart, so IT Support wisely refrained from turning on any archive servers until aircon engineers had sorted the problem mid-morning on Monday. But in the meantime, another electrical "engineer" inadvertently cut the power to the room at 10am, so everything crapped out anyway. This resulted in disruption to routine operations in that the weekly backups did not take place, and starting the building of the DR4 database with the GPS was delayed until mid-week. MSH noted that the order for a big new cluster DB server has been placed. Sun engineers have been consulted concerning the diabolical performance of their fibre-linked, ZFS gizmo lent to us (that promised so much but has so far delivered SFA). Software: This week NJC has been sorting out the synoptic tables for correlated band-pass observations. With RSC, NJC has been sorting out some errors in ProgrammeBuilder. Finally NJC has produced some catalogues for the GAMA project. RSC noted: "Updated the user documentation in the NonSurveyCuration TWiki page following changes to the software for automatic non-survey curation. Testing of automatic non-survey release continues; NJC and I have successfully squashed a few bugs that cropped up w.r.t. deep and synoptic data curation. This should be ready fairly soon pending further tests, so I've started preparing the database for non-survey release by resetting the ppErrBits values for all non-surveys and associated ppErrBitsStatus flags. I've included this capability in the general UpdateDetectionTables helper script, in case we ever need to do it again, e.g. due to algorithm changes. Further database bugs that require fixing prior to non-survey release are listed in the NonSurveyAutomation TWiki page." ETWS noted: - Implemented new CASU illumination table design into CU4. - Fixed some minor issues in the post-CU3 code and tested the illumination table transfer. Survey Data Release: MAR noted the enquiry from Phil Lucas concerning GPS mosaics. ACTION: MAR to link the existing DR3 mosaic in place of the old one on the archive pages, and to look into a DR4 version if necessary. As noted above, GPS DR4 release has hit a snag but it is being investigated as a top priority. Non-survey Data Release: Final testing is taking place presently as the GPS release has stalled. Astrogrid deployment & Data Analysis services MSH noted that a new release of the AG SW suite is available from today, and the roll-out of this release will take place at WFAU asap (i.e. when MSH returns from hols). LGR noted progress on developing intra-WFCAM image list-driven photometry with help from NJC and ETWS. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================