From nch@roe.ac.uk Thu Feb 19 18:40:33 2009 Date: Fri, 13 Feb 2009 16:46:21 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 13/02/09 Minutes of WFAU VDFS Science Archive meeting: 13th February 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, NJC, RSC, PMW, MSH, MAR, RPB, LGR Apologies: JPE, JMS, AL, RGM DoNM: 10am, Friday February 20th 2009 in the Vista Hut Actions discharged this week: ----------------------------- ACTION: RSC to fill in Starlink SW questionnaire on behalf of WFAU; ALL to fill in for personal use Discharged Actions partly discharged but continuing: ----------------------------------------- ACTION: ETWS to parse that latest keyword info into the VSA metadata schema files. Continues ACTION: NCH to assign U/08B/14 H2/K frames to the GPS in the WSA. Continues Actions carried forward from 06/02/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: The team filled in the progress chart for January. WFCAM & VISTA updates: Nothing new to report this week. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: NCH forwarded on JPE's message concerning ESO-Garching network connectivity to ETWS, RPB and MSH who expressed interest. ETWS noted starting transfers of SDSS BestDR7 from NCDM in Chicago. WSA Operations: ACTION: RPB to schedule a full backup of the WSA this weekend as it's long overdue. Operations have been severely hampered by load-server problems (see below) but these are all sorted out now so ETWS noted he will have a busy week next week. Hardware: RPB reported: "Ahmose: After the restore finished last week, the consistency check failed. It was found that the disk subsystem was dialled up to run at 320MB/s. MSH & NCH reconfigured them to run at the more reliable 160MB/s, though this involved a re-initialisation of some RAID arrays. This finished on Wednesday. Another consistency check has been run and found only two errors, both in the SDSSDR2Flags table. It was felt that this problem was minor enough not to worry about. Djoser: RAID array on djoser still degraded. Two 250Gb disks were purchased to replaced the failed unit and act as a hot spare. However, when the new disk was inserted, it was found that the existing disks were, in fact, 250.9Gb. Therefore the new disk was not large enough to fit into the RAID array. Failed disk was replaced with a 400Gb SATA disk that was spare. This will be replaced with something suitable once the two 250Gb disks have been sent back to the supplier. In the meantime, IT Support have taken over the repair of the RAID array. NJC has lodged a support ticket to monitor progress. Until that's fixed, /djoser/home and everything under it are offline, severely restricting the work of some. C2 power supply: IT Support have asked that everyone be made aware of the status of the power supplies in C2. Since rack 5 is on a different phase of the power supply to the other four racks, we have to be a bit careful. A unit with multiple power supplies should not be plugged into both rack 5 and any other rack. If it is, bad things will most definitely happen, possibly involving loss of life." ACTION: RPB to return the two 250Gb disks and order something more useful. MSH noted that the database cluster build will start next week, now that enough power strips of the same mains phase have been installed in the appropriate racks. MSH also noted that we need to complete the migration of the web server from thoth to horus so that the former can be reinstalled and brought up-to-date as a true fail-over web server. Finally, thor is being reconfigured today as new WFAU processing work horse menkaure. NCH asked about the status of the LTO-3 tape drive on djoser, and the team agreed that it may as well get moved over to horus for ease of backups given all the shenanigans with djoser. Software: RSC reported: "ETWS and I have been attempting to get the 64-bit installation on khafre working, since we're running low on functioning curation servers right now. Following last week's problems, ETWS upgraded the systems software on khafre again, which resulted in new problems in mxODBC that we eventually identified as being an incompatibility between the latest unixODBC and Python 2.6.1, which has now been fixed. After much investigation, the problem with our 64-bit installation of the CASU source extractor, cirdr, turned out to be a problem caused by CFITSIO 3.1 compressing the deep stack image FITS files using a different compression algorithm for the floating point pixel values than our other systems (3.04). By linking cirdr to CFITSIO 3.1 we were able to make it work with deep stack image files compressed by CFITSIO 3.1. However, earlier versions of CFITSIO cannot now uncompress these files, which would cause problems for our users if they don't have access to CFITSIO 3.1. We also still have major problems with the 64-bit compilations of our wfcamsrc software under the latest systems software install on khafre, which will require extensive investigation. The only other remaining problem identified so far with the systems software update is that epydoc is not compatible with Python 2.6, but that's not a major concern and the epydoc developer has been contacted. As part of the endeavour to make our software easily switch between WSA & VSA operations, I've been ensuring that our software only uses the DbSession class to access to the database." RSC volunteered to document some of the CFITSIO issues on the archive web pages in case users start to experience problems. ACTION: RSC to document CFITSIO compression issues on the archive website ETWS noted: -- Reinstalled some of the third party software to help RSC with his tests of our 64bit installation on khafre. -- Investigated problems in deprecation of compressed images of the reprocessed files for 08A/B and testing work routine on test DB. MAR noted that the User Interface for non-survey users now includes same functionality as UKIDSS access e.g. colour images, multiGetImage. NCH noted that some requests for enhancements in the survey UI have been received from RGMcM at IoA, and MAR noted these can be incorporated without too much trouble (e.g. additional keywords in FITS results files detailing DB release used and query start/end date-times). Survey Data Release: NCH noted that GPS survey head Phil Lucas has communicated his latest eyeball QC results for DR5, but requests that no further release preparations take place (some discussion needs to take place concerning posisble reprocessing of nebulous regions). No problem! Non-survey Data Release: MAR noted that users of 36 new nonSurvey survey-like release databases informed by email and additional documentation added to website. Astrogrid deployment & Data Analysis services: NCH thanked LGR for his work in working with users over the new list-driven photometry analysis tools. Regarding on-going support for these services once LGR has left for his new post in Geneva, we will endeavour to do this amongst MAR/NJC/NCH. Miscellaneous: General interest seminar in physics on parallel supercomputing - see http://www.ph.ed.ac.uk/seminars PMW also noted that JENAM registration is now open ACTION: ALL to look at JENAM programme and consider possible attendance ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================