From nch@roe.ac.uk Mon Jan 16 11:48:14 2012 Date: Mon, 16 Jan 2012 09:29:46 +0000 From: Nigel Hambly To: Eckhard Sutorius , Nicholas Cross , Rob Blake , Ross Collins , Mike Read , Mark Holliman , Clive Davenhall Cc: Stelios Voutsinas , Jim Emerson , Lorenzo Rimoldini , Keith Noddle , Mike Irwin , Norman Gray , Tom Shanks , Andy Lawrence at ROE , Tom Kerr , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive team meeting minutes, 13 Jan 2012 Minutes of WFAU VDFS Science Archive meeting: January 13th 2012 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: ETWS, RPB, NJC, ACD, MAR, MSH, STV, NCH, KTN, RSC, RGM Apologies: JPE, AL DoNM: 10am Friday January 27th 2012 in the VH (MAR in the chair) Actions discharged this time: ----------------------------- ACTION: RGM, with input from NCH, to contact VISTA PIs with suggested timescale for public releases. Discharged; see Survey Release below Actions partly discharged but continuing: ----------------------------------------- ACTION: RPB to roll out latest curation client / DB server share configuration for routine ops use. Continues; userkaf was done and RSC tested things out for that machine; all was found to be well. But RSC noted that there are issues concerning the parallelised codes and hardwired WSA/VSA functionality, so ETWS will test these out. ACTION: RPB to try monthly detection table partitions for VVV. Continues; CU4 ingest of VVV data into separate partitions implemented, but further tests required Almost there. Later CUs not yet tested and assignment of multiframeID / partition for deep products not yet implemented. ACTION: NJC/RSC to check how we are handling the replacement of deblend column with average confidence values in the standard catalogue output and update schemas as required. Discharged and all is well for the VSA, but the retrospective fix for the WSA must be added to the list of stuff to do for the final UKIDSS release - RSC volunteered to make a note on the relevant TWiki topic. Actions carried forward from 16/12/11 meeting --------------------------------------------- ACTION: RPB to set up WSA mirror/sync for flat file DBs using MS SQL replication. Continues; he presence of mergelog tables in the VSA replication caused a clash with a survey release. So replication had been turned off! No-one could remember if mergelog tables needed to be in the sync DBs so they'll be removed and then see if that breaks anything. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues Specific points and new actions: -------------------------------- Project management: RGM noted a few points of relevance to the imminent grant renewal: i) it is still unclear what the rules will be governing resources for hardware; in the meantime MSH is assessing the requirements for storage into the medium term, taking into account all WFAU's committments; ii) RGM noted that he had spoken to JAC about any plans for UKIRT operations beyond the current funding (to end of March 2013) and it seems that plans will likely be firmed up in time for inclusion in the VDFS Science Archives WSA part of the bid for resources. WFCAM, VISTA and VST updates: No news to report this time. Comments and issues arising from CASU minutes: No new minutes this time. Networking: Nothing of note this time. WSA/VSA/OSA Operations: ETWS reported: -- Installed latest 3rd-party software on shepseskaf. Having still the old kernel there but producing the same segfault for CU4 points now strongly to an incompatibility of gcc4.6 with our code, so we might need to backtrack to an older version. -- Updated the parsing script and associated files to create a GES browser. ETWS also noted that the processed VST commisioning data is being used to exercise the prototype OSA ingest DB. NCH enquired about the offline backups done over Xmas; RPB noted that all WSA/VSA DBs were backed up on LTO-4 media except the VVV which, contrary to all reasonable expectations, ran out of tapes. So this one will be done again, along with the SSA DB and the SSS flat-file legacy datasets. ACTION: RPB to complete the LTO-4 tape backups of VVV and SSA DBs and SSS flat file datasets. Finally, it was reported that all P87 VSA data have now been ingested Hardware and Systems: MSH noted that: -The new Astrogrid servers are up and running and just awaiting IP address assignments in order to host the AG services. -Forward-look requirements for WFAU storage are being assembled -The new cluster node is *still* awaiting "electrification" or plugging in to the mains in server room C2. Software: NJC and RSC have been working on lots of little big fixes and enhancements to the curation codes for handling the latest requirements of the PIs for their world-release VSA databases (see next section). Survey Data Release: UKIDSS: fix-up of an issue concerning missing/deprecated frames in the DXS DR9 was done last week; MAR noted that the processing rehash for the GPS DR8 was now all done and dusted, including retrospective QC, so the GPS is now ready for quality bit flagging and then a complete rerun of the source merging. MAR noted that some kind of cunning plan is needed to actually get the GPS DR8 into the existing release DB since the metadata for all surveys is now changed owing to the reprocessing... should be fun. VISTA surveys: RGM went over the PI responses to the recent email communication regarding world-public database releases: - VMC: PI not happy with immediate release owing to the perrenial v1.0/1.1 and ESO-SAF plus proprietary time problem (don't ask). The approach will simply be to do as she asks, which is delay until April. - VVV: DR1 DB in preparation, contents as per the recent A&A paper but with enhance DB-driven products as well. It was noted that another 5-6 weeks of processing time is needed to complete the DB, after which a 1 week checking period has been agreed with the co-PIs. RPB is in the driving seat. - VIKING: there is as yet no ESO-SAF "release" to guide us over the contents so this needs to be clarified with the PI, but otherwise he is happy for the release to go ahead simultaneously with the ESO-SAF release. GAMA fields are currently being QCed by the survey team. RGM and MAR are in communication with the PI. - VIDEO: no response to our Yuletide communication so RGM will chase; in any case, we believe the contents have been agreed already. NJC will check the current mosaics generated against the contents of the release DB, which has already been generated, using the new functionality in the release DB script CU19. Then RGM will contact the PI to schedule final checks. RGM then raised the question of publicising the releases, which prompted suggestions that ranged from the quite sensible (use existing distribution lists) to the ridiculous (Tw*tter, MyFace, SpaceBook etc). RGM volunteered to set up a new TWiki page to which all *reasonable* contributions will be welcome. Non-survey Data Release: Nothing to report this time. Astrogrid deployment & Data Analysis services: Some issues regarding the on-demand list-driven photometry tool have been reported this week, and MAR is looking into them but it could just be a case of comparing apples with oranges. Miscellaneous: Happy New Beer ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ============================================================= -- Scanned by iCritical.