From nch@roe.ac.uk Fri Jan 5 18:19:54 2007 Date: Fri, 5 Jan 2007 12:02:00 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , John Taylor , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Mark Holliman , Peredur Williams , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 5/1/07 Minutes of WFAU VDFS Science Archive meeting: 5th January 2007 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, PMW, ETWS, RSC, LGR Apologies: JPE, RGM, JMS, JDT, MAR, JB, NJC, BCW, AL, MSH DONM: 10am, Friday 12th January 2007 in the Plate Library Actions discharged this week: ----------------------------- None Actions partly discharged but continuing: ----------------------------------------- The following from last week partly done but continue: ACTION: JB to rearrange external catalogues onto batch catalogue server Continues; ETWS and NCH are juggling the DR5 files during the absence of JB. ACTION: RSC to profile CU4 and investigate possible optimisation Continues; on hold until parallelisation and other refactoring is complete. Actions carried forward from 15/12/06 meeting: ---------------------------------------------- The following from last time continue: o) Add in a default row for every detector appearing in every detection table (for schema consistency when querying merged sources and individual detections) - ACTION: RSC & NCH Specific points and new actions: -------------------------------- Project management: PMW noted that the next VDMT is anticipated sometime soon (TBC in Jan); also that PPARC, encouraged by the positive FDR outcome, have actioned extending the VDFS development money (note also that we expect to be submitting the WFAU Rolling Grant renewal proposal on May 1st, but that this may slip owing to the reorganisation of the research council). PMW suggested that he and NCH meet to formalise the 2007 Q1 plan: ACTION: NCH & PMW to meet 10am Tues 9th to prepare the formal Q1 VDFS work plan. WFCAM update: NCH has received a note from AA noting that he will be visiting the ROE site on February 8th. Comments and issues arising from CASU fortnightly minutes: No new minutes this year. Networking: ETWS and NCH noted that the SDSS-DR5 backup files are now being arranged on thutmose; NCH suggested we make a tape backup before attempting to restore the DB. ACTION: NCH to ask for help from IT Support in accessing thutmose network drives from Veritas BackupExec NCH noted that he has received a message from Ed Valentijn concerning archive liaison between VDFS Science Archives and AstroWISE, from the point of view of both ingest of optical (i.e. VST) and outgest of IR (i.e. WFCAM and VISTA) data. Given uncertainties about the schedule of VST, the latter seems higher priority at the moment. In any case, NCH suggested that he, MAR and perhaps JDT should look at the AstroWISE web-service interfaces together before making a detailed response (see Astrogrid below). In the meantime, NCH will acknowledge Ed's message. Finally, NCH noted that a UKIDSS Calibration Working Group meeting has been called for the end of January, and that NJC has been delegated to attend if possible. WSA Operations: Database recalibration has been running over Christmas, with the procedure (invoked on 18th December) being only 67% complete as at Jan 4th AM. There is some evidence that it has slowed down considerably over the last week or two; NCH has rationalised the disk space on ahmose to give the transaction logs (which currently stand at a massive 1.6TB) room to expand in case that is holding things up. We will monitor the rather worrying situation closely over the next few days ... Hardware: Nothing to report this week. Software: Nothing to report owing to Xmas break. Survey Data Release: Reiterating the proto-schedule from last time: Complete transfer of 06A : Done " ingest " " : Done Photometric recal : As noted above, was expected to be complete by now but may take several more weeks ... Quality bit flagging : Done except for cross-talk; development will continue as long as photo recal runs but if not finished by recal completion then crosstalk flagging may not be implemented for DR2 ... QC1 : +1 - stalled due to photo-recal locking up the ingest DB to all other modifications; JPG eyeball deprecations still awaited for GCS (shame on NCH!), LAS and a few DXS. CU7 (source merging) : +3 (again totally dominated by GPS, and has to be done from scratch because of q-bits and recal) DXS CUs : not on the critical path, since much can be done in parallel with previous; later CUs run fast on the relatively small amount of data) CU16 : +1 QC2 : +1 Final CUs : +3 The time taken to apply the photometric recalibration DB modifications has become a major worry; there is a danger that this could put back the DR2 release date by several weeks. Non-survey Data Release: NCH noted with gratitude that two non-survey DBs were released before Xmas. Astrogrid deployment: NCH reported receiving a few emails from the VO Team (JDT/RGM) concerning "news syndication" through AG DSA access, the idea being to inform users of the status of WFAU archive services. All seems like a jolly good idea. WFAU will ask for more info and updates at the VO team meetings. With reference to the AstroWISE stuff in Networking above: ACTION: NCH to discuss and review AstroWISE interfacing with MAR and JDT. Miscellaneous: Nothing else this week.