From pmw@roe.ac.uk Fri May 30 18:15:30 2008 Date: Fri, 30 May 2008 18:10:04 +0100 From: Peredur Williams To: Nigel Hambly , Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: Andy Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 30/05/08 Minutes of WFAU VDFS Science Archive meeting: 30th May 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: RSC, NJC, MAR, ETWS, LGR, MSH, RPB, PMW Apologies: NCH (at UKIRT), JPE, JMS, AL, BCW, RGM DoNM: 10am, Monday 9th June 2008 in the Vista Hut - Note day! Actions discharged this week: ----------------------------- ACTION: RPB to convene a meeting of interested parties to discuss locations and importance of all data help on WFAU servers to organise a consistent backup and DR strategy. Done: ETWS, MAR and RPB (with e-mail input from RSC) met to survey needs (see new ACTION below) ACTION: ETWS to set up a daily cron job to synchronise amenhotep..WFCAMPROPRIETY (sic) with ahmose..WSA Done. It works. RSC commented on apparent lack of log files; agreed they were probably being squirrelled away in a different directory. ACTION: RPB (with help from NCH) to secure the GPS detection table data and ingest DB backup Done. ACTION: RSC to update total exposure times for all multiframes Done. ACTION: RSC to update WFCAMOPENTIME in line with the Board's wishes. Done. Actions partly discharged but continuing: ----------------------------------------- ACTION: NJC to test CU13/14 from newly merged SVN trunk in advance of running by ops for DXS in DR4 Python code works. Asked CASU for for bugfix or use old version of CFITSIO. ACTION: ETWS to run CU4 for individual UKIDSS surveys after successful backup of the ingest DB Underway. ACTION: MAR (with advise from NCH) to progress as much as possible of QC2 Underway. ACTION: ETWS to continue with routine CUs 1-3 to ensure timely ingest & release of 08A processed flat files for Mar/Apr etc. Underway. Specific points and new actions: -------------------------------- ACTION: RPB to formulate backup plan arising from meeting above Project management: NJC will attend the VIDEO meeting in Hertforshire on June 4th - thanks! WFCAM & VISTA updates: Nothing of note this week. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: WSA Operations: ETWS reported started the recalibration on 28/05 at 16:25 and that 16.5 out of 27 months' worth of data had been processed by Friday morning and ingesting started. He had re-uploaded reprocessed data from CASU. These are now being processed and will then be ingested, so that we are up-to-date with the CASU data for 08A. Hardware MAR reported crash of webserver, traced to problems with Thoth motherboard. It was planned to replace this and upgrade Thoth's OS to Debian4 on June 10. MSH will set up Horus meanwhile to cover for Thoth if the upgrades failed ... Software: RSC reported that he had updated and tested all software to remove dependency on the EXTNAME FITS keyword that is missing in the more recent 08A files, which was holding up ingest of the latest data from CASU (see trac ticket:116). NJC reported that he had sorted out a bug in the CU13_mosaic and totalExpTime, so we now correctly produce the totalExpTime for mosaics. With ETWS we discovered a bug in the stacking software - a new compiled version of CFITSIO does not work with the CASU stacking software. ETWS reported that he had modified CU4 so that it could filter data for recalibration by survey. He had modified the monitor status page creator to show staus of WFCAMpropriety (sic) database. Survey Data Release: We expect to start source merging, beginning with the GCS, next week. Non-survey Data Release: Astrogrid deployment & Data Analysis services LGR reported: Finally the SDSS webservers are back online, and the optimisation of the file-download process continues. Repeat runs (on the same set of coords) are now fully optimised: they do not submit any request to SDSS and take about 10 sec/position. For jobs submitting new sets of coordinates, timing is between 20-60sec/position right now. It is expected to improve later with better queries to SDSS webservices. Miscellaneous: Nothing else this week.