From mar@roe.ac.uk Mon Aug 31 15:49:36 2015 Date: Mon, 31 Aug 2015 15:49:24 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , dmr@roe.ac.uk, Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 21 August 2015 WFAU VDFS Science Archives progress meeting minutes, 21 August 2015 Present: RSC, RGM, ETWS, RPB, MAR Apologies: STV, KTN, AL, NJC, MSH, ACD DoNM: 10am Friday 4 September 2015 in the Villas meeting room, ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION NJC: check that CU19 is outgesting a specific list from TilePawprints' tables. Done. ACTION: NJC to remove dependencies on old libraries. Done. Actions carried forward from 7/8/2015 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that that DR11 and DR12 were now available and now need to be looked at. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this action still has low priority. ACTION: ACD to obtain the details of obtaining an mxODBC licence from ETWS and progress obtaining a sufficient number of such licences. Continues. There are currently no issues in using the version of mxODBC we have but it is not desirable to do so in the long- or medium-term. ACD should liaise with MSH to ensure that cost of the mxODBC licences is included in the plans for spending the forthcoming hardware grant. ACTION: NJC noted that VISTA mosaicing crashes with normal filtering, which he should fix. Continues. NJC had implemented a fix and checked it into SVN. However, when it was run 'in anger' it still crashed. NJC to investigate. ACTION: NJC noted that he had received a report of a problem with the ESO VVV DR1 release which he should investigate. In one pointing the merged band catalogue contained fewer sources than expected. Continues. This action was low on NJC's priority list. Actions resulting from the VVV DR3 brainstorming meeting held on 24/2/15: ------------------------------------------------------------------------ ACTION: RPB to experiment with ingest via partitioned tables, along the lines of the SSIS paper. Continues (changed from 'deferred'). RPB had started looking at this but MAR and MSH were not sure if the tests to date were on properly partitioned tables. RPB and MAR to have a further look at the SSIS paper and partitioned tables. However as the VVV release was now running again, RPB noted that this was currently on hold. ACTION: RGM (was RSC, RGM): (i) RSC to write a succinct summary of the ingest and outgest problem that is the principal bottleneck with the VVV processing (briefly the ingest and outgest speed is limited to approx. 100 Mbit/sec for no apparent reason) and (ii) RGM to compose an e-mail message around this summary and send it to Ani Thakar. Continues. RSC had composed a succinct summary and sent it to RGM. RGM had emailed Ani Thakar but no reply to date. RGM had sent a reminder but had still received no response. NEW ACTION: NJC had noticed a new problem when some of the code was run on Teti (which has the more recent libraries installed). He should isolate and characterise this problem so that it could be investigated. Specific points and new actions: -------------------------------- Project management: * RGM noted that he would start going over the material collated for the grant renewal next week and would likely be contacting various individuals for further input. WFCAM, VISTA and VST updates: * Nothing to report; all the telescopes were believed to be operating nominally. Networking: * There would be a short outage at the end of August. WSA/VSA/OSA/GES Operations: * ETWS reported that 3/4 of the OSA fixed catalogues were ingested and he was now working on the un-fixed catalogues. An issue had been encountered with CFITSIO when CU4 had been run on the WSA but this was now fixed. * RPB reported that having dropped and re-attached the primary key on vvvDetection that CU19 ingests were back running. Hardware and Systems: * MSH reported: Djer's OS disk is fully fixed and working as expected. Piye had 4 dead disks, 3 of which failed silently. These have all been replaced but at the cost of emptying our stash of 2TB spares. The long term plan is to migrate all of Piye's data onto the new NAS box (to be purchased in Oct). All our servers now have PAT test labels on them (though it's unclear if they were actually tested). Software: * NJC had committed software changes to the SVN to allow a VPHAS release to be attempted. * ETWS reported that he had minor fixes to the OSA and WSA code. * MAR noted that he'd been looking into MOC map and HEALPix usage. It was agreed that HEALPix Id columns should be added into the various archives. Review of Deadlines: See: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines Survey Data Release: * GESiDR4Spectra had been released. * ESO had provided feedback on the few ATLAS band merged catalogues that had been uploaded. None of which looked unreasonable to implement. Non-survey Data Release: * A VSA non-survey release had been made for Richard McMahon. Astrogrid deployment, VO & Data Analysis services: * Nothing to report; all systems operating nominally. Miscellaneous: ACTION: MAR to contact Mike Irwin about resuming WFAU/CASU telecons.