From nch@roe.ac.uk Sat Oct 25 10:38:44 2008 Date: Fri, 24 Oct 2008 16:40:25 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 24/10/08 Minutes of WFAU VDFS Science Archive meeting: 24th October 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RPB, RSC, NJC, LGR, MSH, MAR, PMW Apologies: JPE, JMS, AL, BCW, RGM, ETWS DoNM: 10am, Friday 31st October 2008 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: NCH to remove from the WSA website the archive downtime warning for this weekend Discharged; MAR has put a new note concerning the new date of 1st Nov (possible service outages of ~1hr duration) ACTION: LGR to register for "Science from UKIDSS II" meeting, and all to make sure they book accomodation early. Discharged; NCH noted that everyone should book travel/accom asap as well since the meeting is so close to Xmas. Actions partly discharged but continuing: ----------------------------------------- ACTION: RSC to update the WSA Multiframe schema with a new varchar(128) attribute which defaults to NONE ACTION: NJC to update the WSA schema with the new filter details Both done but not checked in/applied due to ongoing CU19/GPS issues Actions carried forward from 17/10/08 meeting: ---------------------------------------------- ACTION: NCH to break everything by deprecating the loadServer DbConstant so that curations codes can be generalised to work for both VSA and WSA... but not until the WSA non-survey SW has been rolled out. - continues ACTION: All to comment on the development of the VSA metadata schema as described on the relevant TWiki page - continues ACTION: NCH to remember to remind ETWS/MAR to reparse the DR4 schema browser. - continues Specific points and new actions: -------------------------------- Project management: Nothing of note this week. WFCAM & VISTA updates: Nothing new to report this week. Comments and issues arising from CASU fortnightly minutes: With reference to last week's query concerning file naming, NCH noted that our CASU chums had already suggested the following scheme: images v20081001_00139_??_??_??.fit conf maps v20081001_00139_??_??_??_conf.fit cats v20081001_00139_??_??_??_cat.fits where ?? ?? and ?? are optionally and in order of occurrence sf,st,tl sf - interleaved if anyone uses this mode st - stacked dither sequences tl - mosaic tile version of combined pawprints of above NCH apologises for not reading your email properly in the fist place! Networking: Nothing new this week. WSA Operations: RPB reported: "A long week wasted with CU19. For some unknown reason it refused to outgest just the final chunk of GPS data. Lots of different attempts and googling of possible problems, all to no avail. Finally managed to outgest the last chunk after NCH's suggestion to redirect stdout to a file. Still unknown why this should cause a problem, particularly when all the previous chunks outgested perfectly fine (give or take). Still, it's done now, so we can crack on with finalising DR4 for release next week (hopefully!). Stoked up the tape carousel on ahmose with plenty of blank tapes in readiness for a backup of the DR4 database over the weekend." So DR4 incl. GPS should go live sometime next week (once it's backed up and copied onto the public servers) just in time for ... Hardware ... Scottish Power are now coming to switch power cables on the 1st Nov, the resulting interuptions to archive services have been noted on the website by MAR. MSH and PMW noted that the upgrade/expansion of existing JBOD SCSI RAID is about to start, with load server ahmose first in line to have a set of 16 150GB U320s retired in favour of 300GB units. The 150GB disks will be kept as spares for other JBODs while they await similar upgrades. Software: RSC reported: "Most of the week has been spent merging NJC's CU6 enhancements/fixes into my working copy that was also substantially modified for integration into the automated non-survey release procedure. Several more bug fixes and further developments were identified, and I am now in the process of retesting the automated release procedure with the new version of CU6. When this is completed and committed to SVN, I will need to merge ETWS's changes in release_7 into trunk, to enable a new release_8 branch to be created. This will ensure the software is in a coherent state for whatever operations are necessary whilst I'm away. Additional updates to CU19 to ensure the default row of detection tables are copied when the outgest is split into chunks, and applied the bcp stdout redirect fix to all outgests. Helped RPB to manually finish off the gpsDetection part of the CU19 release. Updated WSA schema with the new Multiframe.msbRejected column. Prepared a poster on the automated non-survey data releases for ADASS. I'll be away for the next two weeks from Tuesday, and when I return I'll probably be located in R3." NJC reported: "This week I have committed my last few changes to CU6, and some scripts that produce figures for testing purposes. I have also updated the Twiki for the correct curation procedures for UDS. Finally I have produced a poster for ADASS." MAR has been working at on-demand colour JPEGs with a view to implementing their availability in the interface in the same way as currently monochrome images. There has been a flurry of emails this week between SJW and VDFS concerning some whacky Petrosian fluxes for deblended objects in the standard catalogue products. There is a consensus that a fix can be implemented relatively easily via existing correct parameters (i.e. aperture fluxes and Petrosian radii) and MJI is communicating with NCH over a fix-up code to run on the FITS catalogues; NJC noted that he was already thinking along these lines for a possible fix. NCH noted that all seem to agree that we should try to fix up the database and flat-file products for all surveys save the GPS (where it's not that important anyway) in time for DR5 release. There was some discussion concerning what should be done for the GPS (e.g. maybe set all gpsDetection Petro fluxes/mags to defaults). Survey Data Release: As noted above, there is some hope that finally GPS DR4 will be released next week. NCH noted some support requests from the GPS that have come in that should be dealt with asap - ACTION: LGR/MAR to respond and take appropriate action concerning recent GPS support requests. ACTION: NCH to remember to ask ETWS to update the GLIMPSE tables for GPS cross-matching in DR5. Non-survey Data Release: Final testing of new regime on hold until GPS DR4 is out of the door... Astrogrid deployment & Data Analysis services MSH noted that: "All WFAU hosted AstroGrid services will be upgraded on 27/10/08 to the 2008.1.2 version (DSA's, Registries, Community, VOSpace, CEA). This will include enhanced security for VOSpace and several other basic improvements and bug fixes." LGR noted further development of intra-UKIDSS image analysis tools, and asked NCH to forward support request emails concerning some possible new coverage applications. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================