From nch@roe.ac.uk Sat Jul 14 00:04:35 2007 Date: Fri, 13 Jul 2007 17:31:56 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , John Taylor , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Mark Holliman , Peredur Williams , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 13/7/07 Minutes of WFAU VDFS Science Archive meeting: 13th July 2007 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RSC, PMW, ETWS, MSH, LGR, RGM Apologies: JDT, BCW, JPE, AL, JMS, NJC, MAR, JB DONM: 10am, Friday 20th July 2007 in the Plate Library Actions discharged this week: ----------------------------- ACTION: NCH & PMW to meet on Mon 9th at 2pm to prepare the Q3 plan. Discharged; see Project Management Actions partly discharged but continuing: ----------------------------------------- The following from last time partly done but continue: o) Add in a default row for every detector appearing in every detection table (for schema consistency when querying merged sources and individual detections) - ACTION: RSC & NCH Continues; will get sorted (honestly!) as part of the general ingest DB schema revamp to facilitate rapid generalised photometric/astrometric recalibration. Actions carried forward from 06/07/07 meeting: ---------------------------------------------- The following from last time continue: ACTION: NCH to discuss and review AstroWISE interfacing with MAR and JDT. ACTION: NCH to start some hard-nosed negotiations with the UKIDSS PI and CSS concerning the contents and timing of DR3. - CONTINUES; see Survey Release below ACTION: AL to check on the exact wording of the survey release policy with reference to world release of proprietary survey data. - CONTINUES Specific points and new actions: -------------------------------- Project management: PMW presented the Q3 plan the he and NCH had sketched out earlier in the week. Some cuts were made to the amount of work planned to bring it into line with available effort; the plan will be fettled by PMW and then presented to VDMT (next Friday) for approval. WFCAM & VISTA updates: NCH noted that SJW is still looking for volunteers for UKIDSS observing in September/October, and has forwarded the message to team members to encourage them to think about taking part. LGR has communicated his interest. Comments and issues arising from CASU fortnightly minutes: No new minutes as of 13/07/07 am. Networking: JB reported: "CU1 has been running pretty constantly over the last week and finished transfering all the data marked OK_TO_COPY on Wednesday (this is all of April of 07A, May has yet to be marked). Discussions concerning the speed of the transfer continue with a couple of fresh ideas coming up, I am to contact Peter Bunclark about this (though the need to transfer 06B data may hold implementation up)." WSA Operations: JB reported: "CU1, CU2 and CU3 continue to the point that we have almost completed up to CU3 for April of 07A. Operations has transfered from using CVS to SVN over the week (with the usual teething issues) but it's currently looking good." Hardware: JB reported: "A fan on osorkon broke and was replaced by Eclipse fairly promptly, there were no issues with this. Data is currently being transfered to osorkon from sneferu so that sneferu can be retired as a data server (though it will remain in place to process data)." MSH noted that he and JB have been checking out the performance of the new catalogue server hatsehpsut, specifically looking at the relative performance of straight Ultra320 SCSI versis serial-attached SCSI (SAS). Early indications are that SAS always performs as well or better than straight SCSI; in some situations, SAS is considerably faster. The next benchmarking tests will be done with SQL 2005 database trawls. ACTION: MSH to put a TWiki note up concerning benchmarking the disk subsystems on the new catalogue server hatshepsut. Software: ETWS reported writing test cases for CUs 2 to 4. The test data as well as expected results can be found in /disk09/wsa/test/cu234_test/. RSC reported: "Committed various small enhancements to the software that came out of our meetings. The most useful of which is a bunch of enhancements to the CSV module including support for the standard Python library csv dialects when reading CSV files from a variety of sources. Also started upgrading the code to Python 2.5 to tidy up a few syntax hacks." JB noted having found a number of bugs in various bits of software that have been speedily fixed (for the most part - the others are not so important) by RSC and ETWS. NCH reported committing his first revision to the new SVN (pairing/seaming bug fixes/enhancements) and discharging the appropriate trac ticket (66). Survey Data Release: Good news on NDR data delivery to CASU: AA at JAC notes that provided only that there are no problems with the tape reading, two weeks have been shaved off compared to previous estimates and all the data should be on the ftp site by Friday (HST) this week. This prompted a discussion amongst the team concerning the contents of DR3. Given the delays and that users are likely to want timely data access as opposed to waiting for perfect, uniform astrometric/photometric calibrations, the WSA team suggest that we should go hell-for-leather on DR3, with calibrations as they exist at present. NCH needs to discuss this with SJW. ACTION: NCH to discuss contents and timing of DR3 with UKIDSS CSS SJW asap. Non-survey Data Release: JB reported four new non-surveys have been added to the database with CU21 having been run Thursday (though more have come in since then). MAR and JB have sorted out flat file access for test data for one of the new UKIRT Campaign Proposals (GABARIT) for the PI to peruse the data. Astrogrid deployment: MSH noted that the next 6 months or so are likely to be given over to consolidation of existing SW infrastructure rather than new developments in AG (e.g. MyDB etc.) so little or no effort will be required from the archive team on deployment of new infrastructure over that time. Miscellaneous: NCH called early doors for the Waiting Room at 6pm(ish).