From nch@roe.ac.uk Fri May 21 16:29:27 2010 Date: Fri, 21 May 2010 15:41:03 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren , Tom Kerr Subject: WFAU VDFS Science Archive weekly project meeting mins 21/05/10 Minutes of WFAU VDFS Science Archive meeting: May 21st 2010 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RSC, NJC, MSH Apologies: JPE, AL, ETWS, KTN, RGM, RPB, MAR DoNM: 10am, Friday May 28th 2010 in the VISTA Hut Actions discharged this week: ----------------------------- ACTION: NJC to Doodle poll for a kick-off PSPI co-ordination teleconf Discharged; Skype teleconf to take place on Tue June 1st at 3pm BST. Actions partly discharged but continuing: ----------------------------------------- None this week. Actions carried forward from 14/05/10 meeting: ---------------------------------------------- ACTION: NCH to fill out a Q2 2010 plan and continue with the progress monitoring for the time being. Continues ACTION: RPB/KTN to order several crates of LTO-4 tapes. Continues ACTION: ALL to consider who goes to Garching in September for the VST PS readiness meeting. Continues ACTION: ALL to review new hardware requirements/procurement over the next couple of weeks Continues Specific points and new actions: -------------------------------- Project management: NCH thanked NJC for organising a PSPIs Skype teleconf to discuss archive end things with the primary stake-holders. WFCAM & VISTA updates: Nothing of note this week. Comments and issues arising from CASU minutes: No new minutes this week. Networking: CASU have indicated that a small number of funnies in v0.8 Vista data have been corrected and the files migrated to v0.9; this will be transfered and ingested by ops next week when ETWS/RPB return from hols. It was noted that at this stage we won't backtrack on the release preparations for relatively small corrections - it is felt better to keep imperfect releases rolling out so that the survey teams have plenty to play with until such time as v1.0 products arrive. WSA/VSA Operations: NCH updated the meeting on operations for the week: WFCAM: the latest 10A transfers are ingesting VISTA-VVV: small snafu meant that the deep stack production went wonky on Tuesday; NJC and RSC diagnosed/fixed the problem and the process was restarted so VVV deep stacking/cataloguing continues VISTA-VIKING: all finished, checked by NJC and ready for release by RPB on his return VISTA-VMC: small snafu with semaphor files and multiple curation runs resulted in deep stacks/catalogues refusing to ingest. NJC and ETWS (thanks for checking in, even when on hols!) diagnosed the problem and this is now back up and running for CUs 3/2/4; then NCH will set the autocurate part-2 running. Hardware and Systems: MSH noted that the new curation server should be set up later today and hence will be ready for software installation on return of ETWS; and also that he's liaising with an engineer later this morning over certification of the LTO tape library prior to it's maintenance contract being renewed. Software: RSC noted: "Recent confusions over VSA operating instructions have demonstrated the need for consistency in the use of date ranges applied to each curation script for each programme. To make this more clear and to aid the operator we suggest that date ranges should now be mandatory arguments wherever they are relevant. The option to use mnemonic names instead of explicit dates will make this change less of a burden, and also gives us the chance to integrate the ObsCalendar module into every script that could use it. Recent changes to the software for the VSA shallow surveys now require that they are always curated using AutoCurate. This is because CU7 now requires productIDs to be set in ProgrammeFrame for all VSA surveys (and WSA non-surveys), which is performed by CU13. This also means we now need to test source table appending via AutoCurate, which is something that has not been enabled before. NJC now also plans to improve the consistency of the frame set association applied in CU7 for these surveys by making full use of associations made during the programme setup stage. All of the above changes need to be clearly documented in the operating instructions of the TWiki." ACTION: RSC & NJC to update CU codes accordingly for mandatory date range and shallow VSA curation ACTION: RSC & NJC to document new operational procedures on the TWiki NJC and RSC noted that a new version of PyFITS is available but that the change logs indicated it probably hasn't fixed the issues found during our use of the software. RSC intends to take a look and then make formal bug reports on any remaining issues. RSC noted that he intends to make a new release branch (#16) prior to the ops boys returning from hols; RSC also asked MSH if CU19 should now always build database statistics across all objects in release database to aid query exectution plans; MSH and NCH suggested that this is still a good idea despite one or two mixed messages coming from tests with SQL2008. NCH noted that we've had several instances of other poorly performing queries being drammatically speeded up by switching of parallelism in the exection plan with the T-SQL hint OPTION (MAXDOP 1). Survey Data Release: UKIDSS GPS SH Phil Lucas has been in communication regarding QC for DR7; this will be sorted out on MAR's return from hols. VISTA survey proprietary release DBs are in preparation as noted above. Non-survey Data Release: Nothing of note this week. Astrogrid deployment & Data Analysis services: Nothing of note this week. Miscellaneous: Nothing else this week. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================