From nch@roe.ac.uk Sat Jan 16 10:27:14 2010 Date: Fri, 15 Jan 2010 17:06:58 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting mins 15/01/10 Minutes of WFAU VDFS Science Archive meeting: January 15th 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, PMW, RPB, MSH, MAR, NJC, RGM, RSC Apologies: JPE, JMS, AL DoNM: 10am, Friday January 22nd 2010 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: PMW and NCH to meet on Monday 12pm to flesh out the 2010 Q1 plan Discharged ACTION: All interested parties (RGM, NCH, PMW, NJC, MAR, AL) to meet on Wednesday 13th at 2pm to discuss a plan of action for VSA added-value services. Discharged ACTION: PMW to book the Vista Hut meeting room for Wed 13th at 2pm. Discharged ACTION: ETWS to communicate directly with MJI over the exact contents and versioning of the latest batch of VISTA data Discharged; it's V0.6 for the time being (also confirmed at VDMT) ACTION: RPB/ETWS to check free space for mass file store and liaise with PMW/MSH if another NAS box needs ordering. Discharged; to be on the safe side, a 96TB unit is being looked at to keep things bubbling along... (worst case scenario is that we run out of space in 2 months) Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 8/01/10 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: PMW passed around the plan for 2010 Q1 for the team to adjust as necessary and otherwise endorse. NCH reported back from the VDMT which he attended with by JPE, MJI, STH, PMW, RGM and was very useful in getting us up to speed on VISTA developments. A subset of the team met on Wednesday pm to discuss "added value" archive enhancements that might be attractive to the broader community (i.e. ESO!). RGM is drafting a note which he hopes to circulate early next week. RGM noted that interviews for replacing the irreplacable PMW will take place the week after next. WFCAM & VISTA updates: NCH noted some communiques regarding VISTA "visitor programmes" where users are interested in using VDFS services. NJC noted that the generic programme builder can be used to set these up once some data is ingested; also that RequiredStacks definitions should be expunged from the archive initiation script, since the same procedure can set those up automatically. ACTION: NCH to remove any RequiredStacks definitions from VSA_InitiateArchive.sql NCH asked that attendees to the PSPIs meeting at IoA at the end of Jan draft their presentations by next Friday so that they can be reviewed and general contributions made. NCH suggested a rough model split of how the WSA works and then interation with the meeting on VSA and a prioritisated list of any changes/enhancements. ACTION: NJC, MAR and ETWS to draft PSPI VSA presentations by Fri Jan 22. Comments and issues arising from CASU minutes: None this week. Networking: MSH noted a possible JANET network outage on Feb 2nd. WSA/VSA Operations: RPB noted: "DR6 statistics generated on the copy on ramses1. Took a list of stats from previous DRs and generated them on DR6. Took a little over 48 hours though, once done, don't need to be regenerated. Will work on doing the same on the public versions on ramses5 & 6, though possibly look into how to speed things up (e.g. not doing a full scan of GPS tables). Backup onto tape of all databases continues. Still haven't got Backup Exec working across the network, but can mount the read-only databases onto ramses1 and back them up there. All UKIDSS databases now on tape." ETWS noted: - WFCAM 09A data has finished CU4 processing, ingests will finish this evening. - WFCAM 09B data for November/December has been checked by CASU and November is transferred, December will be here over the weekend. - VIRCAM November 09 data version 0.6 has been made available by CASU and is transferred. CU2 is creating JPEGs at the moment. The team discussed plans for preparation of VSA datasets prior to the PSPI meeting, and agreed to try to forward curate the V0.6 SV/dry-run data and set up access for named PIs only to each dataset to demo and kick-off discussions at the meeting. ACTION: RPB (with advice/help from RSC and ETWS) to do a VSA forward-curate test with the ingested V0.6 data. Hardware and Systems: ETWS noted - The 3rd party software update on khafre is going ahead, but there are some problems inbetween SExtractor and LAPACK, so I might need another day to fix everything. RGM asked if there were any other hardware requirements; NCH suggested upgrade of the backup library from LTO3 to 4 ACTION: RPB to check out availability/maturity of LTO4 drives and then liaise with PMW over purchase if required. Software: RSC noted: "We've agreed to restrict all future BCP outgests to one processor with a manual MAXDOP 1 option in the SQL statements to avoid the performance issues on the cluster that occurred during the last GPS release. It was decided this would be better than modifying the cluster server settings in a blanket restriction. Completed last of the shallow survey VISTA testing with a test of the Provenance table update and creation of test releases databases for all of the main surveys to aid schema browser creation of release dbs for the PI meeting. VISTA stackconf files do not contain provenance information, so they have now been excluded from the Provenance table. Further VISTA software testing awaits release of v1.0 tiled data from CASU together with the necessary stacking software updates for deep stacks." NJC has been working on implementing a galaxy classification code for the VSA. Additionally NJC has done some testing of and added some description to the CAL release. Survey Data Release: SJW has asked about the world release of UKIDSSDR4PLUS, noting that original proprietary release of July 1 2008 implies world release on Jan 1st 2010... except that the GPS was added in 3 months later. We await confirmation from GPS SH Phil Lucas as to whether it goes live now or on April 1st (snurk). Non-survey Data Release: MAR noted that WFCAMCAL08B is live and should appear in drop down DB lists on the WSA website. ETWS has produced a shiny new browser which is also live. Astrogrid deployment & Data Analysis services: MAR noted that the list-driven photometry facility has developed a few quirks in that it's crashing and/or not returning all records for a given input list. NCH noted that there is an outstanding user request to implement aperture corrections for the SDSS part of the service. RSC volunteered to check it out. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================