From acd@roe.ac.uk Fri May 29 08:06:55 2015
Date: Thu, 28 May 2015 16:11:41 +0000
From: Clive Davenhall <acd@roe.ac.uk>
To: Bob Mann <rgm@roe.ac.uk>, Andy Lawrence <al@roe.ac.uk>,
"Read, Mike" <mar@roe.ac.uk>, Eckhard Sutorius <etws@roe.ac.uk>,
Mark S. Holliman <msh@roe.ac.uk>, Nicholas Cross <njc@roe.ac.uk>,
Nigel C. Hambly <nch@roe.ac.uk>, Rob Blake <rpb@roe.ac.uk>,
Ross Collins <rsc@roe.ac.uk>, Stelios Voutsinas <stv@roe.ac.uk>,
Dave Morris <dmr@roe.ac.uk>, Clive Davenhall <acd@roe.ac.uk>
Cc: "dante@astro.puc.cl" <dante@astro.puc.cl>, James Dunlop <jsd@roe.ac.uk>,
"j.p.emerson@qmul.ac.uk" <j.p.emerson@qmul.ac.uk>,
ktn keithnoddle.org <keith@keithnoddle.org>,
"Lorenzo.Rimoldini@unige.ch" <Lorenzo.Rimoldini@unige.ch>,
"mcioni@aip.de" <mcioni@aip.de>,
"matt.jarvis@astro.ox.ac.uk" <matt.jarvis@astro.ox.ac.uk>,
"mike@ast.cam.ac.uk" <mike@ast.cam.ac.uk>,
"nigel.metcalfe@durham.ac.uk" <nigel.metcalfe@durham.ac.uk>,
"norman@astro.gla.ac.uk" <norman@astro.gla.ac.uk>,
"pwl@star.herts.ac.uk" <pwl@star.herts.ac.uk>,
"rgm@ast.cam.ac.uk" <rgm@ast.cam.ac.uk>,
"Simon.Dye@nottingham.ac.uk" <Simon.Dye@nottingham.ac.uk>,
"s.j.warren@ic.ac.uk" <s.j.warren@ic.ac.uk>,
"tkerr@jach.hawaii.edu" <tkerr@jach.hawaii.edu>,
"tom.shanks@durham.ac.uk" <tom.shanks@durham.ac.uk>
Subject: WFAU VDFS Science Archives progress meeting minutes, 22 May 2015
WFAU VDFS Science Archives progress meeting minutes, 22 May 2015
Scientific
Present: RPB, MSH, MAR, RGM, ACD
Apologies: STV, KTN, AL, RSC, ETWS, NJC
DoNM: 10am Friday 5 June 2015 in the Villas meeting room,
MAR in the chair (room is booked).
Actions discharged this time:
-----------------------------
ACTION: RPB/MAR to generate the missing VVV DR2 indices and statistics.
Actions carried forward from 24/4/2015 meeting:
-----------------------------------------------
ACTION: RGM to investigate the size and content (how does it differ
from DR9?) of SDSS DR10 to see if it is worth us hosting and
whether it is necessary to keep all of the earlier releases.
Continues. The first part had been done: the difference between DR9
and DR10 is largely the presence of additional spectra. The part
of this action that continues is that RGM had not yet established
which early releases could safely be scrapped. RGM also noted that
that DR11 and DR12 were now available and now need to be looked at.
For further details see the WFAU SDSS strategy wiki page:
http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan
ACTION: ETWS, with RPB, to develop an automated helper script to
keep release DB files links updated.
Continues: this action still has low priority.
ACTION: RSC to test the new version of mxODBC (was ETWS/NCH purchase a new
licence for mxODBC).
Continues. RSC was currently testing the new version under an evaluation
licence.
ACTION: NJC noted that VISTA mosaicing crashes with normal filtering,
which he should fix.
Continues. NJC had implemented a fix and checked it into SVN. However,
when it was run 'in anger' it still crashed. NJC to investigate.
ACTION: RPB to compile a Web page listing all the completed releases.
Continues. RPB had made a start compiling this list. However, the
exercise revealed that a number of databases had been misplaced and
he was investigating their whereabouts.
ACTION: NJC to produce a roadmap for future VSA releases in order to plan
for hardware requirements.
Continues, NJC had contacted the PIs.
ACTION: NJC noted that he had received a report of a problem with the
ESO VVV DR1 release which he should investigate. In one pointing
the merged band catalogue contained fewer sources than expected.
Continues. This action was low on NJC's priority list. NJC noted that
revised VVV DR2 content had been uploaded to ESO.
Actions resulting from the VVV DR3 brainstorming meeting held on 24/2/15:
------------------------------------------------------------------------
ACTION: NJC to speak to the VVV PIs, notably Dante, about the schedule for
and expected number of remaining VVV releases.
Continues. NJC had spoken to the PIs but the results had been inconclusive;
he should pursue the matter further.
ACTION: RPB to experiment with ingest via partitioned tables, along the lines
of the SSIS paper.
Continues.
ACTION: RGM (was RSC, RGM): (i) RSC to write a succinct summary of the ingest
and outgest problem that is the principal bottleneck with the VVV
processing (briefly the ingest and outgest speed is limited to approx.
100 Mbit/sec for no apparent reason) and (ii) RGM to compose an e-mail
message around this summary and send it to Ani Thakar.
Continues. RSC had composed a succinct summary and sent it to RGM. RGM
had emailed Ani Thakar but no reply to date. RGM would send a reminder to
Ani. Still awaiting a response.
ACTION: NJC to remove dependencies on old libraries.
Continues. NJC was not present.
Specific points and new actions:
--------------------------------
Project management:
* Nothing for the minutes.
WFCAM, VISTA and VST updates:
* It seems likely that the VISTA direct camera will remain on the telescope
until 2021/22 and hence either extended versions of the current surveys or
new surveys will continue until this date.
Networking:
* Nothing to report.
WSA/VSA/OSA/GES Operations:
* ETWS was not present but was believed to have been working on the
reprocessed ATLAS OSA list-driven products.
* ACD noted that investigations of the iDR4 UVES singlespec spec were
continuing prior to ingest and were expected to complete shortly. Delivery
of the iDR4 working group astrophysical parameters was expected imminently
(scheduled for the day of the meeting), with the node parameters to follow
shortly. There would be some changes, still undecided, to the arrangement
of the node parameters.
* RPB noted that VVV DR3 processing continued.
Hardware and Systems:
* MSH reported that all systems were operating nominally.
* RGM noted that following discussions with Colin Vincent it seemed that
a further tranche of money for hardware purchases should be available
in the autumn with the possibility of an additional one in early 2016.
Software:
* Though not present at the meeting NJC had sent a note to the effect that
he had replaced calls to Scientific.Function.LeastSquares with numpy/scipy,
as requested.
* ETWS was believed to have been continuing to work on the OSA list-driven
ingest software.
* MAR had been transferring the Web pages for various of the archives to the
new Web server. He had found several bugs in the process, which he was
fixing.
Review of Deadlines:
* Progress towards forthcoming deadlines was reviewed. See:
http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines
Survey Data Release:
* The VIDEO P93 Team Release had been made on 20/5/15.
Non-survey Data Release:
* None since the last meeting. There was believed to be one still outstanding.
ACTION: MAR to compile a new list of non-surveys for release.
Astrogrid deployment, VO & Data Analysis services:
* Nothing to report.
Miscellaneous:
* RPB noted that he had booked his flight to the forthcoming VMC conference
in Naples.
* It was recalled that some of the survey groups had inquired about obtaining
credit for 'value added' products that they had provided, such as 'difference
imaging.' Ways of recording this information and presenting it to users
were discussed and some would be implemented.
ACD,
Original: 22/5/15,
Revised: 22/5/15.