From mar@roe.ac.uk Wed Sep 25 17:40:58 2013 Date: Wed, 25 Sep 2013 17:33:21 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , Matt Jarvis , Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 20 September 2013 WFAU VDFS Science Archives progress meeting minutes, 20 September 2013 Present: RSC, ETWS, MSH, RPB, NJC, ACD, MAR, RGM Apologies: NCH, KTN, AL DoNM: 10am Friday 4 October 2013 in the Villas meeting room NCH/ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: MAR/NJC to approach VISTA PIs about their public releases. Done. VMC, VIKING and VIDEO had replied giving the go ahead for VSA public releases of the data supplied for their ESO releases. For VIKING permission was given to release all of vikingv20121205 rather than just the 151 fields supplied to ESO. No reply as yet from VHS. ACTION: ACD to circulate another request soliciting additions to the deadlines page. Done. ACTION: RPB to set up the backup jobs as superuser, so that they can be configured to preserve ownership etc. Done. ACTION: RPB to set up a routine backup of the GES data (which occupies an insignificant amount of disk space). Done via rsync. ACTION: ACD to identify and press-gang volunteers to assist with the Open Days. Done, NCH (plus student), ETWS and ACD would be wowing the public. Actions partly discharged but continuing: ---------------------------------------- ACTION: NJC to look at current and future VVV load server storage requirements so that RAMSES14 could be set-up optimally. Continues: NJC still had to calculate the total data volume required. Actions carried forward from 6/9/2013 meeting: ---------------------------------------------- ACTION: RPB to have a go at releasing other VISTA PI DBs (e.g. for JPE) Continues: Fell over during release CU19, RPB looking into it. ACTION NJC: RPB to check the logs, identify the error and forward to NJC, who would investigate and fix it. Continues: This relates to another VISTA non-survey release. RPB had forwarded the logged error to NJC. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. Specific points and new actions: ------------------------------- Project management: RGM would organise a doodle poll for the remaining PDRs on his return from Hawaii. Still not much activity on the wiki's deadlines page. ACTION: ALL to add/maintain entries on http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines WFCAM, VISTA and VST updates: * Nothing to report. Comments and issues arising from CASU minutes: * Nothing to report. Networking: Linux desktop upgrades had still not been fully rolled out so some disruption still anticipated. WSA/VSA/OSA/GES Operations: * ETWS noted: -- The 4 month of WSA data CASU had released lately are now transferred. -- VSAVVV ingests were going well, but stopped due to some SQLServer issue. RPB is investigating. -- CU6 for the VVV part 1 is still trundling along nicely. It looks like some/all of the N-E markings on the OSA library jpegs might be wrong. Fixing them could take a lot of time but it might be possible/preferable to just overlay the correct markers without going back to the FITS files. ACTION: MAR/ETWS to investigate/fix the orientation of the N-E markings on the jpegs. VSA - non-VVV v1.3 data has all been ingested (CU4). Once QC has been copied over from previous versions we can start building releases. GES - iDR1 being revised again. Now have WG10 individual node files but some issues have been discovered and ACD is investigating. OSA - Nigel Metcalfe had raised an issue with regard to unpaired sources across passbands in ATLAS. It looks like most of the genuine cases are the result of offsets between the differing passband pointings and the fact that we only source merge across a single frameset. The tiling/overlaps in WSA/VSA are such that this has not previously shown as an issue. However the tighter tiling, smaller overlaps, spurious edge detections and some incorrect field offsets in ATLAS show up in the distribution of unmatched sources. ACTION: MAR to investigate and quantify the issue with unmatched ATLAS sources. And if the issue warrants, RSC to look into the curation code with a view to pairing up the unmatched sources. The issue of backing up deep stacks re-surfaced and it was agreed to actually implement it. ACTION: RPB to implement backups of deep file products. Hardware and Systems: MSH raised the issue of available funds for new hardware eg NAS box, processing machine and infiniband upgrade. ACTION: ACD to discuss with RGM the available budget for hardware purchases. Survey Data Release: Another release GES iDR1 would be made once the WG10 node files had been ingested. VVV DR2 has almost finished the first part of CU6. P90 OSA data are all ingested so QC could be done and the release process started. Software: NJC reported further work on the MAP pipeline, running it across multiple archives. ETWS reported: -- Expanded the script to create SQL tables and metadocs from external databases for all the new views in the SDSS DBs. Unfortunately some attributes are renamed in one system table but not the others used to create the view. -- Installed the latest versions of the 3rd party software on userkaf. RSC to test it. MSH mentioned an issue that Dave Morris had hit upon with the work on MyDB. Although our release DBs are available read-only across various servers the MyDB database would need to be on a single server as it would be read-write. Non-survey Data Release: Nothing to report. Astrogrid deployment, VO & Data Analysis services: It looks like the 6dF SSAP service running under the DAL toolkit might be using the wrong units for it's cone search. MSH had discovered some more recent source code that had apparently been corrected but it was not clear how easy it would be to re-compile and integrate. SIAP services are unaffected. Miscellaneous: Potential VMC meeting to be held at ROE in spring 2014. MSH asked if we should be doing something more with the HARPS N-data we currently host. ACTION: RGM to look into making more of the HARPS N-data -- Scanned by iCritical.