From nch@roe.ac.uk Mon Oct 31 09:09:35 2011 Date: Fri, 28 Oct 2011 17:11:08 +0100 From: Nigel Hambly To: VDFS Science Archive Team -- Clive Davenhall , Eckhard Sutorius , Mike Read , Mark Holliman , Nigel Hambly , Nicholas Cross , Rob Blake , Ross Collins Cc: CCs for VDFS meeting minutes distribution -- Andrew Lawrence , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Mike Irwin , Bob Mann , Stephen Warren , Stelios Voutsinas , Tom Kerr Subject: WFAU VDFS Science Archive project meeting mins, 28/10/11 Minutes of WFAU VDFS Science Archive meeting: October 28th 2011 ------------------------------------------------------------------------- ------------------------------------------------------------------------- Present: NCH, ETWS, RPB, MSH, NJC, ACD, STV, MAR Apologies: JPE, KTN, RGM, RSC, AL, C37, 89X DoNM: 10am Friday December 2nd 2011 in the VH NB: henceforth, meetings to be held on the first Friday of the month (extra operational meetings to be held as required) Actions discharged this time: ----------------------------- ACTION: RPB to set up a linux -> windows share Discharged ACTION: RPB to run cu16 on LAS/GCS/DXS against SDSS DR8. Discharged Actions partly discharged but continuing: ----------------------------------------- ACTION: RSC to test temporary ingest file writing to high bandwidth shares between curation clients and DB servers. Continues ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Work in progress (some problems with OS level permissions on r/o files - MS DOS strikes again) ACTION: RPB (or whoever communicates with the PIs) to keep up the pressure on world public release DB preparation by emphasising that the recently prepared DBs are not simply for internal consumption. Continues Actions carried forward from 30/09/11 meeting --------------------------------------------- ACTION: RPB to set up mirror/sync for flat file DBs using MS SQL replication. Working for VSA; needs to be rolled out for WSA ACTION: RPB to try a proof-of-concept on detection table monthly partitions using the VHS to start. Superseded by: ACTION: RPB to try monthly detection table partitions for VVV Specific points and new actions: -------------------------------- Project management: NCH welcomed a couple of newcomers to the team who have parachuted in to help solve the WFAU underspend/overcommit problem: Stelios Voutsinas will be working on some web-based exploration tools for billion row scale-out problems (i.e. VVV!), while our old chum Clive Davenhall returns to help out with database developments for VST (but will work on Gaia-ESO Survey archive data model in the first instance). WFCAM & VISTA updates: Nothing of note. Comments and issues arising from CASU minutes: The team noted the minutes of the meeting of 28th September; there were no major concerns. Regarding the action on EGS to check if we've got the latest version of the VIDEO data, the answer is yes we have (provided it hasn't been rereprocessed encore une fois). NCH noted that the six-monthly UKIRT Board report contribution for the WSA was due on Wednesday; MAR has chased this up. Networking: ETWS noted that WFCAM GPS reprocessed/filtered data are now all present and correct; VIRCam August data has been transfered up. WFCAM data from July (end 11A) is starting to transfer (version badge is back to normal, i.e. version 1 data). NJC took part in the latest VMC telecon yesterday; there were no major concerns, and even an outbreak of sanity concerning release timescales. Halelujah. WSA/VSA Operations: ETWS reported: -- Created browser pages for the VIKING, VVV and WFCAM DR9 releases. -- Started ingesting GPS CU4 data and transferred the reprocessed filtered GPS data. RPB reported: "VHSv20111011 released VIKINGv20111019 released VVVv20110928 released UKIDSSDR9PLUS released Slight problem with the Galex neighbour tables in DR9 now fixed and being updated." NCH thanked ops for all the releases. The DR9 release in particular is full of all sorts of new goodies (proper motions, cross neighbours with WISE, GALEX & SDSS-DR8, all of which are held locally in the WSA/VSA). RSC and NCH are working on the documentation of the new proper motion determinations. Hardware and Systems: MSH noted that newly online this week is hi-spec DB node ramses12 and also a new beefy web server for VO services. Some issues with an errant NAS box have been sorted out by replacing a motherboard. MSH also gave advanced warning that the IfA are thinking of annexing 6 racks worth of server room space for a 5 PB (8 NAS box) monster apparently for EUCLID simulation purposes, subject to confirmation that the floor can take the weight... Software: Some small fettling recently of the ESO deliverables scripts, incorporating OB level products (for VIKING) and also modifications to header keys requested in the last ESO gripe feedback. MAR noted some work in the User Interface to cope with all the various juggling of large databases (VVV, SDSS-DR8 etc). Survey Data Release: As noted above, UKIDSS DR9 is now released. MAR noted also that the first tranche of deliveries for VIKING have been shunted over to ESO - no burps so far. MAR also noted some ad-hoc large-scale dataset outgests (with a bit of help from this end) for Vasily Belokurov (infrared variability study in advance of Gaia photometric alert datasets) and Francesco La Barbera (LAS images for extended source photometry). Non-survey Data Release: Nothing of note this time Astrogrid deployment & Data Analysis services: MSH reported back from his trip to Pune for the latest (last?!) IVOA meeting. It seems many countries are killing off their VO projects, with only the UK and CDS Strasbourg maintaining the initiative (we note also that ESO and ESA have pulled the plug on their VO developments; MSH noted that AG services currently running on borrowed time down in Leicester will soon be moving north as part of a consolidation of core UK services). MSH also noted that the next big thing, "ObsCore" for catalogue services, employs TAP and PGSphere spatial indexing/geometry in an implementation hard-wired for PostgreSQL. Apparently JHU are working on a port to SQL Server, so maybe we can poach that. ETWS noted that the SDSS-DR8 UCD table is empty, which has halted publishing of that dataset via our own VO services. A quick email to JHU revealed that they were intending to upgrade to the latest UCD1+ IVOA standard, but hadn't gotten around to doing it before we slurped the DB across the Atlantic. ETWS suggested that we do this for them asap, seeing as work is being done on WFAU's archive UCDs generally at the moment to bring them into line with the latest IVOA standard. Miscellaneous: Nowt else this time. ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory Blackford Hill Edinburgh EH9 3HJ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ============================================================= -- Scanned by iCritical.