From nch@roe.ac.uk Fri Oct 31 17:13:10 2008 Date: Fri, 31 Oct 2008 16:35:47 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archives weekly project meeting minutes, 31/10/08 Minutes of WFAU VDFS Science Archive meeting: 31st October 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, LGR, MSH, MAR, PMW, ETWS Apologies: JPE, JMS, AL, BCW, RGM, NJC, RSC, RPB DoNM: 10am, Friday 7th November 2008 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: RSC to update the WSA Multiframe schema with a new varchar(128) attribute which defaults to NONE Discharged ACTION: NJC to update the WSA schema with the new filter details Discharged ACTION: NCH to remember to remind ETWS/MAR to reparse the DR4 schema browser. Discharged by ETWS & MAR without any need for NCH to remind. Actions partly discharged but continuing: ----------------------------------------- ACTION: LGR/MAR to respond and take appropriate action concerning recent GPS support requests. Will be done early next week since GPS DR4 is now done. ACTION: NCH to remember to ask ETWS to update the GLIMPSE tables for GPS cross-matching in DR5. Nearly done, thanks to ETWS. NCH noted the need to poke the curation driving tables in the WSA so that CU16 automatically creates the required crossmatches. Also need to define the crossmatch tables in the schema file WSA_NeighboursSchema.sql Actions carried forward from 24/10/08 meeting: ---------------------------------------------- ACTION: NCH to break everything by deprecating the loadServer DbConstant so that curations codes can be generalised to work for both VSA and WSA... but not until the WSA non-survey SW has been rolled out. - continues ACTION: All to comment on the development of the VSA metadata schema as described on the relevant TWiki page - continues Specific points and new actions: -------------------------------- Project management: Nothing of note this week. WFCAM & VISTA updates: The autumn issue of the UKIRT Newsletter is now available: http://www.jach.hawaii.edu/UKIRT/publications/newsletter/ - lovely job. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: Nothing new this week. WSA Operations: RPB reported: "DR4 with GPS put live (finally!). DR4 was held up slightly by the length of time that it took to copy the data to amenhotep and thutmose. On closer inspection it turned out that several of the disks were set up to use "write-through" mode whereby the on-disk cache was disabled. Enabling the cache caused the write speeds to increase by a factor of 6. Fixed up the 08A blank jpgs. Eckhard provided a list of jpgs which were obviously screwed (based on file size) and I re-ran CU2 on each of them to regenerate the jpgs." ETWS noted: - Created latest set of browser pages for the DR4 GPS release. - Processed GLIMPSEI version 2.0 and GLIMPSEII version 2.0 data (mainly adding unit coordinates and HTM IDs) to be available for DR5. NCH suggested that we press ahead with CU4 catalogue ingests of 08A without prior correction of the deblended Petrosian flux bug, since the latter needs a little code development and testing and has to work post-ingest/retrospectively anyway; ETWS noted he has some ideas concerning how to do this in the most efficient way possible from the DB point of view. NCH noted that he needs to allocate the 07B H2 frames taken in director's time to the GPS so that they appear in GPS DR5. ACTION: NCH to allocate 07B H2 frames to the GPS programme and then let MAR & ETWS know when it's done. Hardware Reminder from last week: Scottish Power are now coming to switch power cables on the 1st Nov, the resulting interuptions to archive services have been noted on the website by MAR. Archive services will be offline from sometime after 5pm this evening until sometime tomorrow (hopefully mid afternoon, but if any problems things could be off until Monday morning...) ETWS noted that he intends to try a new installation of Python 2.6 on 64-bit server khafre to see if the memory leak bug is fixed, and if so he will apply the same upgrade to web server horus. Storage expansion for the DB and flat-file servers was discussed, with PMW noting the issue of the grant renewal as the VDFS ops equipment money is a little tight... MAR noted that if there is any downtime on load server ahmose we need to make sure that the flat-file synchronisation jobs with the public server don't scrub the existing databases before discovering it's not possible to update them - some uncertainty as to whether RSC had already spotted this one, so ACTION: NCH to ask RSC to check public/load-server synchronisation for robustness against load-server downtime. Software: Regarding general photometric recalibration/tweaks, ETWS suggest a new delta-mag attribute might be useful in the future, which contains all low-level zeropoint tweaks over the standard flux calibration (which is more easily expressed in SQL). ACTION: ETWS to add a delta-mag attribute to the VSA template detection table schema. As far as correcting the deblended Petrosian mags in the WSA goes, ETWS suggested we take a similar approach and after correction of the flat files to mirror CASU's changes, apply the DB fixes by keeping a note of low-level delta-mags which can be applied to the standard calibration of the corrected Petrosian fluxes, thereby avoiding the need to recompute all the low-level tweaks again. MAR noted: "Still trying to get the interface to the colour image generation working. Had problems getting the backend database query working in a reasonable time but seem to have got that acceptable now. Updated interface to reflect GPS inclusion in DR4." Survey Data Release: As noted above, the phase-2 DR4 with GPS is finally out of the door. We now proceed with DR5 ... Non-survey Data Release: The team agreed that the deblended Petrosian fluxes should be corrected before issuing any further database releases for non-surveys. We'll get there eventually... Astrogrid deployment & Data Analysis services MSH noted that the software upgrade all went smoothly save for some issues with the registry which is a little flakey. LGR noted he needs to talk to MAR concerning arbitrary file look-ups given lists of coords for his list-driven photometry implementation. Miscellaneous: Halloween tonight - mind you don't get caught by the ghoulies. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================