From nch@roe.ac.uk Sat May 8 12:05:59 2010 Date: Fri, 7 May 2010 17:13:08 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes 7/5/10 Minutes of WFAU VDFS Science Archive meeting: May 7th 2010 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RSC, RPB, MAR, NJC, ETWS, KTN, RGM Apologies: JPE, AL, MSH DoNM: 10am, Friday May 14th 2010 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: MAR to make a first-pass auto QC script for broad-brush crapulence filtering of VSA data Discharged; script needs checked into the SVN. ACTION: NJC to continue working closely with JRL at CASU on implementing the archive-end pixel processing codes. Discharged; see Software below ACTION: RSC to reconfigure CUs that use disk05 as temporary storage space to use the nifty disk34 instead. Discharged with help from ETWS and RPB ACTION: RPB to switch on world read permissions on all GPS DB objects in UKIDSSDR4PLUS. Discharged; web pages updated with release info by MAR Actions partly discharged but continuing: ----------------------------------------- None this week. Actions carried forward from 16/04/10 meeting: ---------------------------------------------- ACTION: NCH to fill out a Q2 2010 plan and continue with the progress monitoring for the time being. Continues Specific points and new actions: -------------------------------- Project management: NCH welcomed the official arrival of KTN as the new PMW. For his first official duty, KTN is assembling a report for the UKIRT Board concerning WSA operations (and will attend the meeting in London in June to represent WFAU). WFCAM & VISTA updates: Nothing of note this week. Comments and issues arising from CASU minutes: The team noted the minutes of the meeting of 21st April, in particular the fix to Vista data for the night of 20100107 (ETWS will check the transfered version at this end); the availability of v0.8 processed Vista data up to the end of March; the new developments on the tiling software; and the discussions on underexposed aural strips in tiles. Networking: Network transfers of WFCAM/VISTA data continue to run at 10 MB/s. RGM and KTN noted that there is to be a JANET upgrades meeting on June 23/24 at which we have been invited to attend as heavy network users. KTN will do the honours. WSA/VSA Operations: ETWS noted - WFCAM 10A February and March data is transferred and metadata has been ingested. Creating the JPGs is still underway. - VISTA DRYRUN data (up and including Feb 8th) has been nearly fully ingested. - Transfers have started on VISTA P85 data (from Feb 9th). - Created a new set of browser pages for the new VISTA release databases. - Enhanced monitor pages to show more detailed information for the catalogue ingests. RPB noted: "Vista databases nearly all released. "Shallow" surveys (VHS, VMC, VVV) source merged, reseamed, neighbour tables produced, and released (apart from VVV which is taking an inordinate amount of time to reseam). Decision made to cancel VVV reseaming and just release the detection tables, as has been done with all other surveys and SV projects. Vista stacking begun with the VIDEO survey. After a few technical glitches with the code, this is now running smoothly and is 70% done. New LTO4 tape drive installed in tape library. Windows system doesn't seem to recognise it yet, though. May require a reboot to rescan the SCSI bus." MAR noted that he has started UKIDSS DR8 QC, but that we're awaiting eyeball results from the LAS. MAR has also set up a more convenient jpeg eyeballing facility for GPS SH Phil Lucas, who is working on K in 09A for reprocessing requirements for the GPS DR7 release. Regarding VISTA QC, NCH noted that so far a single email from one PS PI has been received asking for one frame to be deprecated. ACTION: NCH to email PS PIs proposing a formal (and efficient!) method for user-specified QC deprecations in the VSA. RPB noted that VHS PI RGMcM has requested crossneighbour tables with SDSS Stripe 82 and the UKIDSS LAS. No problems with the former, but regarding the latter there was some worry about protecting ESO proprietary rights for UKIDSS, so ACTION: RPB to ask RGMcM if he can guarantee that the VHS survey team consists of ESO astronomers only Default joins will be made with the latest world release (currently DR4) if there are any worries over proprietorial infringements. Hardware and Systems: Last week there was an AC unit failure in server room C2. The redundant unit was not switched on, so the ambient temperature rose to ~50C - shurely shome mishtake. Anyway, IT Support are arranging for repair of the failed unit, and hopefully a more fault-tolerant arrangement of the existing infrastructure. Meanwhile, MSH investigated BIOS settings for auto-shutdown of the WFAU servers, but it seems these are hardwired to melting point so not much use. MSH suggested investigating a control unit solution that can shut down servers, while RPB suggested so high-level SW solution on each server should do the job. Doubtless we've cooked quite a few disks during this episode... expect increased RAID degredation over the next few months. KTN noted that all current maintenance contracts for archive hardware have been renewed this week. RGM noted that some bids into UoE funds ("sustainability pot") are being made for additional cluster nodes to host PanSTARRS catalogue access databases, and a standalone LTO-4 backup unit for the legacy SuperCOS datasets. Regarding LTO-4 backup units, RPB noted we have no tapes. ACTION: RPB/KTN to order several crates of LTO-4 tapes. Software: RSC noted development of the proper motion software, notionally for inclusion of pms in UKIDSS DR8, continues. NJC has modified the deep product creation software, partly to allow more flexibility, partly to handle tiles when they come and partly to work better with the changes in the CASU software. The new casutools package has been successfully integrated and the appropriate changes have been made to the automatic curation codes. We can process layers of products and can append to products more easily, since the wcsfit is now working, we get the same results processing stacks in parts or adding to existing stacks. This means that later releases can more easily build on previous releases. NJC has also made changes to ProgrammeBuilder too, so that there are no real differences between deep and shallow programmes in the VSA, so again it is much easier to append new data. Survey Data Release: Nothing to report this week. Non-survey Data Release: Nothing new this week. Astrogrid deployment & Data Analysis services: Nothing of note this week. Miscellaneous: NCH noted that ADASS for 2010 in Boston has been announced, and encouraged folks to think about attending. RGM noted that an email has been received from VST-ATLAS PI Tom Shanks regarding a VST PS readiness review meeting in Garching, September 28/29. ACTION: ALL to consider who goes to Garching in September for the VST PS readiness meeting. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================