From nch@roe.ac.uk Fri Feb 15 15:07:03 2008 Date: Fri, 15 Feb 2008 14:09:28 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Nigel Hambly , Nicholas Cross , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Mark Holliman , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 15/2/08 Minutes of WFAU VDFS Science Archive meeting: 15th February 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, NJC, PMW, MSH, MAR, RSC Apologies: JPE, AL, JMS, LGR, BCW, RGM DONM: 10am, Friday 22nd February 2008 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: MAR to incorporate basic info on using flat files Discharged; additional links given to relevant external websites ACTION: RSC to coordinate migration of the residual CVS-tracked projects into SVN. Discharged; usual functionality has been independently checked by ETWS; MAR noted that the daily cron webpage synchronisation is also working OK. ACTION: PMW to check that IT Support have our future computing accomodation requirements factored into plans for the new room Discharged ACTION: MAR to put a note on the downtime pages, and email an interuption of service warning to wsa-announce. Discharged; note to be sent out to announce list. Actions partly discharged but continuing: ----------------------------------------- The following from last time partly done but continue: o) Add in a default row for every detector appearing in every detection table (for schema consistency when querying merged sources and individual detections) - ACTION: RSC & NCH Continues; will get sorted (honestly!) as part of the general ingest DB schema revamp to facilitate rapid generalised photometric/astrometric recalibration after DR3. ACTION: MAR to take forward the possibility of archive and/or on-the-fly colour image production. Continues; line put in the forward plan for Q1'08; MAR has been studying renaissance masterpieces in the Florence Uffizi for inspiration... Actions carried forward from 15/02/08 meeting: ---------------------------------------------- ACTION: MSH to liaise with IT Support over set up of NFS file share from hatshepsut Continues; NCH noted that although lower priority, would be good to get this sorted so as to shift SegueDR6 onto hatshepsut (plus restore DB then SQL backup) so that the transfer files can be scrubbed from disk. Specific points and new actions: -------------------------------- Project management: NCH noted that shortlisting for the ops recruitment will take place at 3:30pm this afternoon, with PMW, NCH, MSH, RGM and ETWS having perused the applications. Anticipate interviews sometime during the week beginning 25th Feb. NCH noted that AL is setting up a meeting with ESO-SAF folks to discuss interface issues with VDFS/VSA; this may be in Munich on March 6th. NCH and MAR should probably be in on the discussions. WFCAM & VISTA updates: The first 08A WFCAM night was 13th Feb, following severe weather hampering UKIRT ops at the beginning of the month. Comments and issues arising from CASU fortnightly minutes: The team noted the minutes of the meeting of 7/2/08; there were no comments. Networking: Disk tests were completed earlier in the week; thanks to MSH for tenacious efforts in identifying the major bottleneck in the WSA LAN setup; MSH noted that optimal settings have been found, and everything done is covered on the wiki at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/RaidConfigurationTests Transfers should roll along from now on at between 10-15 MB/s. WSA Operations: ETWS noted: - Finished the 07B data transfer, jpeging and ingested on the available data up to 03/11/07. - Re-installed 3rd party software on sneferu. - Updated browser page parser according to move of the SSA and webdocs from CVS to SVN. - Enhanced the DataDaemon for easier operations. NCH noted that the old SSS server is now completely backed up so the LTO-3 drive can be moved to the WSA network; the suggestion is to attach to djoser. ACTION: NCH (following advice from MSH) to ask ITSG to move LTO-3 drive to the WSA network. License keys have arrived for updated BackupExec software to enable SQL2005 backups etc. NCH suggested that a full WSA backup is performed this weekend given a lull in heavy ingest ops. ACTION: NCH to schedule a full WSA backup for the coming weekend. Hardware Linux curation servers djedefre and khufu are being upgraded to Debian 4.0 today. PMW and MSH noted that a new web server has been ordered; some discussion ensued as to the network setup of this. MSH noted the new NAS box is rebuilding it's arrays now and it is planned on being fully operational by Friday afternoon (assuming ITSG can make the necessary changes on the linux boxes by then). Software: RSC noted: * Finished migration of the CVS repository to SVN in 5 separate trac projects, and updated the UsingSVN and MigratingCVStoSVN TWiki documentation. * Updated .login and other software changes for move to SVN (see trac ticket 84), including updates for new software on the Debian 4.0 servers. * Improvements to SyncTestDb.py and Schema.py to support new FlatFileLookUp table. * Completed testing of the dev_1 software branch, and committed to SVN all my working copy changes since before Christmas consisting of changes to create the metadata mirror script SyncDb.py (see trac ticket 97) and the revised use of indices in the WSA (see trac ticket 100). Also, updated SoftwareTesting notes on TWiki. * Synchronising software branches and merging in the dev_1 branch into trunk, which includes the persistent database connection changes made back in August (see trac ticket 68). So we are ready to create the new dev_schema_1 branch, where we'll put in the schema/software changes to improve the recalibration design. * Also helped NJC with implementing a test synoptic source schema database and updates to PyWCS to aid CU6 development. NJC noted good progress on CU6, producing a fast source association script and I am ready to work on the variability statistics package. Survey Data Release: Nothing new this week. Non-survey Data Release: MAR has set up flat file access for 5 more open time PIs this week. Astrogrid deployment: ETWS and MSH have set up metadocs and DSA for BestDR1 for the BCW/RGM QSO classification application tests. MSH noted that next week (Wed-Fri) some ESAC folks will be visiting to discuss SIAP service implementation in general, and the plan is to integerate UKIDSS image services into the VO. Miscellaneous: Nothing else this week