From nch@roe.ac.uk Fri Mar 13 16:38:26 2009 Date: Fri, 13 Mar 2009 13:58:54 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Jim Emerson , Malcolm Stewart , Lorenzo Rimoldini , Mike Irwin , Peredur Williams , Bob Mann , Stephen Warren Subject: WFAU VDFS Science Archive weekly project meeting minutes, 13/03/09 Minutes of WFAU VDFS Science Archive meeting: 13th March 2009 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, NJC, RSC, PMW, MSH, RPB Apologies: JPE, JMS, AL, RGM, MAR DoNM: 10am, Friday March 27th 2009 in the Vista Hut Actions discharged this week: ----------------------------- None this week Actions partly discharged but continuing: ----------------------------------------- ACTION: ETWS to parse that latest keyword info into the VSA metadata schema files. Continues; ETWS noted further progress this week ACTION: NCH to assign U/08B/14 H2/K frames to the GPS in the WSA. Continues Actions carried forward from 06/03/09 meeting: ---------------------------------------------- None this week Specific points and new actions: -------------------------------- Project management: Nothing of note this week, other than the Scottish cheque from STFC for the Rolling Grant (i.e. a piece of paper that says "IOU one cheque"). WFCAM & VISTA updates: JPE sent around a note earlier this week to the VISTA Public Survey PIs with his take on the likely schedule. This was gratefully received, but it was not clear from this note when "Science Verification" may happen (e.g. is it likely to be before or after the formal hand-over we wonder...?) Comments and issues arising from CASU fortnightly minutes: MJI very kindly sent some draft minutes of the latest CASU meeting early this morning; unfortunately, the ROE mail server has been playing up and delivered them very late so we didn't have them in time to peruse and then discuss at today's meeting. NCH has forwarded to the team and asked for any comments to be sent directly to MJI/MR. Networking: ETWS noted: -- Transfer of the reprocessed 08B data has to wait until JANET has finished its commission of the new core lightpath infrastructure. The path from Edinburgh to Cambridge still awaits the Regional Network Connection to be made. WSA Operations: RPB noted: "Database users now fixed. Read only users are truly readonly, though had to enable exec permissions for xp_cmdshell on the master db by hand. Have written up notes in the wiki so this should be a lot easier next time. SDSS DR7. Backup files show that the database comes as one huge 4Tb MDF file. Tried to restore onto two volumes on Pami (one for the mdf file, one for the log file. Both on one volume was too large), however the restore was incredibly slow what with transporting all the data across the network twice. Spoke to MSH about restoring on the new cluster which has more space and infiniband technology! Just need to install SQL Server first. A few degraded arrays on the linux servers were revealed once our RAID checking script was rolled out across all servers. Khafre needed a disk replacing, must order a replacement so we still have one spare. Khufu had a soft failure on one disk, this was fixed by rebuilding the array." ETWS noted: -- All 08A catalogues are processed now and March till May data is ingested. June will be ingested after the weekend. -- I've looked more deeply into the unixODBC/unixODBC/freeTDS installation on khafre to find out that the given size is not in bytes (as described in the SQLServer manual) but in allowed digits. -- Re-started work on the automatic VSA schema construction. Hardware: MSH noted that the DB cluster has 2 remaining nodes to be set up out of the total of 8. An issue with Windows2008 has arisen over support for local infiniband and private LAN connections, whereby W2008 stupidly defaults to using the less efficient communication route between nodes. Investigations are on-going. Software: RSC reported: "When khafre came back on-line I was able to resume final tests of the new software installation and discovered that the values of mxODBC's cursor.description attribute had changed - no longer returning the number of bytes used by each column. Another investigation into the various ODBC layers by ETWS revealed that this attribute should never have returned such values in the first place and thus we had actually been previously relying on a bug on mxODBC that has now been "fixed" in the latest version. I've put a workaround into our software now that uses our parsed SQL schema files instead, so we no longer need to rely on what mxODBC returns. Subsequently I discovered that there still remains a 64-bit bug in mkmerge, which will be investigated now that khafre is back to full health again. In the meantime I've been working on infrastructure issues, making sure that when scripts log exceptions they also send output to stderr that ensures error message e-mails are always sent upon any exception when the script is running as a cron job, e.g. the metadata mirror update. I'm also continuing to upgrade scripts to use the new DbSession API and CLI interfaces to aid VSA development and have made progress on a mock design final changes required." NJC noted that he has got the new recalibration code working. Also had an email exchange with Jim Lewis and Mike Irwin about cir_wcsfit to update the astrometric solution in inter-intermediate stacks, but has not managed to get it working yet. Survey Data Nothing new to report this week. Non-survey Data Release: Nothing of note this week. Astrogrid deployment & Data Analysis services: Nothing new this week. Miscellaneous: NCH noted a recent communication from JHU concerning support for and porting of HTM codes in Java/C++. The team agreed that ports of HTM2 to both languages would be welcome, as the current version is old HTM1, with ad-hoc mods to get it to compile with more recent compilers, and is less richly functioned in terms of utility subroutines. ACTION: NCH to feedback comments/requests to JHU regarding ports of HTM2 to Java/C++. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. =============================================================