From nch@roe.ac.uk Thu Jul 10 17:22:17 2008 Date: Fri, 4 Jul 2008 17:41:00 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Brian Walshe , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 4/7/08 Minutes of WFAU VDFS Science Archive meeting: 4th July 2008 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, RPB, MAR, ETWS, PMW, RSC, LGR, RGM Apologies: NJC, JPE, JMS, AL, BCW, MSH DoNM: 10am, Friday 11th July 2008 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: MSH (with metadoc help from ETWS if required) to publish SDSS SegueDR6 via AG DSA. Discharged (presumably using same metadoc as BestDR5...?) ACTION: RPB to remember to shrink the WSA transaction logs once CU19 has finished running. Discharged Actions partly discharged but continuing: ----------------------------------------- None this week Actions carried forward from 27/06/08 meeting: ---------------------------------------------- ACTION: RPB to formulate backup plan arising from meeting Continues; currently wholesale new backups running; formal policy to be documented. ACTION: NCH, RSC, MAR, ETWS, NJC & RPB to meet next Friday 4th July 2pm to sort out non-survey curation automation This afternoon! Specific points and new actions: -------------------------------- Project management: NCH noted the latest Programmatic Review shenanigans, and that on the whole VDFS operations and residual development should be OK as it falls under the "ESO deliverables" banner. PMW asked the team to fill in the progress charter for June, and noted that the Q3 plan needs to be fleshed out. ACTION: PMW & NCH to meet next Tues (8th) at 12pm to flesh out the Q3 plan. WFCAM & VISTA updates: Nothing new this week. Comments and issues arising from CASU fortnightly minutes: No new minutes this week Networking: NCH and ETWS noted that we're in discussion with CASU over possible enhancements of automation in the data flow systems as regards versioning of reprocessed data. MJI was asked for some specific examples where WFAU have got a tad confused to inform any decisions as to procedural changes. ACTION: ETWS to send MJI a specific example or two of reprocessing confusion WSA Operations: ETWS reported: - Continued the ingest of recalibrated GPS data, 2/3 are now ingested. - Finalised the browser pages for DR4. - Created metadoc of DR4 for Astrogrid. - Worked on CU1 to transfer data from lists containing fits data from a mix of dates. RPB reported: Finishing the last skinny tables in WSA for cu16 (need to document how this is done) Shrinking transaction log on WSA Turning servers on and off as the AC in the machine room went on the fritz. Replacing failed hard drive in Hatshepsut RAID array and ordering replacement. Continuing with programme of ensuring a full catalog of known good backups of databases. Hardware Computer server room C1 overheated on Tuesday afternoon due to the failure of an aircon unit; the problem was presumably exacerbated by the unseasonably warm (i.e. for Edinburgh) weather that day. Fortunately, ITSG were very kind to WFAU and did not switch off any of the public servers, so DR4 remained online throughout. Operations was affected though. NCH noted (after the meeting!) that JNTD in ITSG has some useful experience in building a high-performance, modest cost Storage Area Network, and we should look at his test-bed before making any final decisions as regards NAS v. SAN/SAS for a high capacity DB server solution. ETWS noted that ITSG have updated 64-bit khafre for glibc > 2.3 (2.5?) and that he will now try to install the WSA curation software on that machine. NCH noted that MAR and he have had no further trouble on new 64-bit web server horus since MSH had fettled the installation. Software: RSC noted: "I've started work on altering CU7 to curate the new skinny MergeSource table instead of the Source table, which will prevent the need to re-merge existing sources following a photometric recalibration. The preliminary version is working but needs to be tidied up to make the most of the optimisations that can be gained through reducing the number of columns CU7 needs to curate. Attempts to make the update of the metadata mirror, WFCAMPROPRIETY, less confusing for users are delayed whilst CU4 GPS ingests are keeping ahmose tied up. I've also helped to prepare the DR4 information on the web pages and revised the CurationTask entries in the WSA." MAR reported checking over DR4 DB and fettling the interface for release; and continued work on getting WSA stuff to run on new web server horus, (also started moving over the SSS CGIs to horus). Survey Data Release: DR4+ (phase 1: LAS, GCS & DXS) went live mid-week. MAR announced to wsa-announce; NCH suggested we also do the usual blanket email to registered UKIDSS users. ACTION: MAR to co-ordinate with SJW over DR4 announcement(s). As regards the GPS, ETWS noted (e.g. above) that the recalibrated data ingest should finish within a week (hardware permitting!) and that then the astometrically fixed files will be sorted out prior to the source merging run. Non-survey Data Release: Meeting this afternoon to thrash out automation of support for survey-like DBs. Astrogrid deployment & Data Analysis services RGM noted that AG is to be axed as a result of the recent Programmatic Review, and the immediate future was obviously the major point of discussion at the Consortium meeting earlier this week. The level of residual support (especially for data centre operations) is TBD. Miscellaneous: Preliminary sannouncement: The second 'Science from UKIDSS' workshop will be held at the Royal Astronomical Society, Piccadilly, London, 15-17 December 2008. The workshop aims to bring together, in an informal atmosphere, astronomers who are working with UKIDSS data, to hear about science in progress, to share knowledge about exploiting the UKIDSS archive, and to foster collaborations. http://www.ukidss.org/rasworkshop08/ ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================