From nch@roe.ac.uk Tue Sep 29 10:29:35 2009 Date: Mon, 28 Sep 2009 16:53:59 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archives weekly project meeting minutes, 28/09/09 Minutes of WFAU VDFS Science Archive meeting: September 28th 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: RSC, NJC, MSH, ETWS, RPB, NCH, MAR, RGM Apologies: JPE, JMS, PMW, AL DoNM: 10am, Friday October 2nd 2009 in the VISTA Hut Actions discharged this week: ----------------------------- None this week Actions partly discharged but continuing: ----------------------------------------- ACTION: NCH to draft an interface control document for VSA to ESO-SAF Now that the grants are all sorted out, it seems we can undertake to do this work after all. NCH noted his plan is to revise the draft in the light of the political issues raised so far; get agreement amongst VDFS as to the approach; then circulate more widely amongst the VISTA PS PIs with a view to them using it as the basis of their individual agreements with ESO-SAF. ACTION: ALL to review the VSA metadata schema in trunk/sql SQL script VSA_MultiframeSchema.sql ACTION: ETWS to check for corresponding essential FITS keys for VSA ingest against those defined for the WSA Continue; see Comments on CASU minutes below. ACTION: RPB: backup all DB-driven products after UDS DR5. Continues; this can now be done; the team agreed that all science image products (diffs, mosaics, stacks) should be backed up from /disk*/wsa/products/ and RSC asked that /disk05 is included for various bits and pieces. Migration of public DBs to the cluster servers takes priority, so this has had to be delayed. Actions carried forward from 18/09/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: RGM updated the team on the outcome of the various grant renewals. Now that the grant business is sorted out (for the time being at least), RGM noted that we should contact the survey PI stake-holders to update them on the situation (thanks to MAR for unlinking the doom-and-gloom banner on the WSA pages). ACTION: RGM to email MJI and agree on a form of words along the lines of "things ain't as bad as we thought..." The team discussed the need to start to interact more with the Vista Public Survey PIs as part of these on-going communications, and the idea of a local meeting with them early in the New Year to thrash out QC and other Vista-specific usage issues was discussed. WFCAM & VISTA updates: Good news from VISTA - JPE noted: "The remaining technical problem with VISTA (the appearance of an occasional micro oscillation of an M2 hexapod leg) was solved in July (servo parameter settings fixed). ESO then required a strip down of the top end and recoating the M1, and reassembly as an exercise for future maintenance. A good recoat was achieved - see http://www.vista.ac.uk/mirror_images.html The camera should be remounted on VISTA by the evening of 26 Sep. I am leaving for VISTA tomorrow to reset the optics and pointing after the dis/re-assembly involved in M1 recoating, and will then be spending the subsequent nights up to and including 07 Oct taking images for Press Release purposes (seeing permitting!)." The schedule for observations is now Science Verification in the second half of October followed by dry-run survey observations (as second priority during the "Paranalization" phase) up to the end of the year then formal start of survey operations early in the New Year. Comments and issues arising from CASU minutes: The team noted the minutes of the meeting of 23rd September, and fully understand the need to review the FITS header issues. Just let us know as soon as you have example processed data that are representative of how the SV data will likely appear to us... Networking: All quiet on the network interfaces. WSA Operations: RPB reported: "DR6 underway. LAS source merged and currently nearing end of reseaming. GCS can be processed once it's done, followed by neighbour tables and DXS stuff. Transferring UKIDSS and external catalogue databases to the cluster. With the final restore of SDSS DR3, they should all be present and correct, with the possible exception of extra fettling of the SSA. Hardware problems with amenhotep turned out to be a slightly dodgy power supply connector. Using a different connector solved the problem and amenhotep has been stable ever since." NCH noted that he'll check out the SSA consistency problems noted by MAR on the cluster deployment, and will let ops know when it's ready to be backed up. ETWS noted: - Completed deblend fixes on all FITS files from 05A to 08A. - Completed deblend fixes on the datbase for DXS, UDS, LAS, GCS, PTS, and COMM. Hardware and Systems: MSH noted: "In trying to solve some of the account/permissions problems on the cluster I managed to make a lot of work for myself. I installed Active Directory, which it later turned out isn't really compatible with SQL Server (which of course isn't documented, and is rather stupifying as they're both "enterprise quality" Microsoft products), so I had to then uninstall it. Being Microsoft quality, it decided to remove ALL user and system accounts from ramses1 and ramses2 without mentioning it would do this... Cluster setup is now nearly complete with only some minor issues to iron out. Restoring the databases onto cluster nodes continues and should be completed soon. Once this is finished we will be able to switch the public interface over to point to the cluster, and operations can then be switched over shortly after that." PMW noted that he has placed the order for the thoth webserver replacement. Software: NJC has been testing schema changes and associated curation changes. RSC had suggested a change to the frameset association which avoids the need to copy an intermediate stack and renaming it as a deep when there is only a single observation in one pointing. This has led to some changes in ProgrammeFrame to keep track of products - a productID and releaseNum. NJC has made other changes to the Variability tables for some new attributes too. RSC reported: "Continued working on the redesign of CU19 for use on the cluster now that the system is up and running reliably. Though I've mostly been giving CU7 some attention this week: - Investigated why we can't seem to append in latter epoch data into existing frame sets. Turns out to be due to an incompatibility between the original frame set association algorithm and our changes for the special GPS requirements. Will work with NCH to come up with a better solution for this, though this will only be important for the GPS in DR6, as the other surveys are being source merged from scratch. - Made CU7 completely schema driven to both aid the transition to dual VSA/WSA curation and to correctly handle changes to the schema following the fixes to the deblended detection columns." Survey Data Release: DR6 phase-1 should be ready (subject to checking that all bug-fixes and software enhancements are present and correct) by around 9th Oct (probly). Non-survey Data Release: Nothing to report this week. Astrogrid deployment & Data Analysis services: MSH noted that his ADASS poster is finished as well as the "VO at WFAU" website, which will be available at http://vo.roe.ac.uk shortly. Miscellaneous: ETWS noted that he created a poster for the Open Days about the very cool dwarf found by S. Warren et al. in UKIDSSDR1. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================