From acd@roe.ac.uk Thu Dec 4 15:38:32 2014 Date: Thu, 4 Dec 2014 15:36:22 +0000 From: Clive Davenhall To: Bob Mann , Andy Lawrence , "Read, Mike" , Eckhard Sutorius , Mark S. Holliman , Nicholas Cross , Nigel C. Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris , Clive Davenhall Cc: "dante@astro.puc.cl" , James Dunlop , "j.p.emerson@qmul.ac.uk" , ktn keithnoddle.org , "Lorenzo.Rimoldini@unige.ch" , "mcioni@aip.de" , "matt.jarvis@astro.ox.ac.uk" , "mike@ast.cam.ac.uk" , "nigel.metcalfe@durham.ac.uk" , "norman@astro.gla.ac.uk" , "pwl@star.herts.ac.uk" , "rgm@ast.cam.ac.uk" , "Simon.Dye@nottingham.ac.uk" , "s.j.warren@ic.ac.uk" , "tkerr@jach.hawaii.edu" , "tom.shanks@durham.ac.uk" Subject: WFAU VDFS Science Archives progress meeting minutes, 28 November 2014 WFAU VDFS Science Archives progress meeting minutes, 28 November 2014 Present: NJC, ETWS, RGM, MSH, ACD Apologies: MAR, STV, RSC, DM, NCH, KTN, AL, RPB DoNM: 10am Friday 12 December 2014 in the Villas meeting room, MAR in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: MSH to re-configure RAID on /sahure. Done. ACTION: ACD to re-vamp the deadlines page. Done. The re-vamped deadlines page(s) were discussed during the meeting. Actions carried forward from 14/11/2014 meeting: ----------------------------------------------- ACTION: RPB (was MAR) to have a look at the status of non-survey releases for registered projects. Continues. MAR had compiled a list of non-survey projects that could be released and sent it to RPB. RPB had yet to make these data available. ACTION: RGM to investigate the size and content (how does it differ from DR9?) of SDSS DR10 to see if it is worth us hosting and whether it is necessary to keep all of the earlier releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. The part of this action that continues is that RGM had not yet established which early releases could safely be scrapped. RGM also noted that DR11 would be available in January. The main difference would be the addition of more spectra, which WFAU should archive. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this action still has low priority. ACTION: All to finish off VVV brainstorming tasks by end of January at the latest. Continues. ACTION: ACD speak to HME about the possibility of IT support making nightly backups of the SVN repository. Continues. ACTION: RPB write a chron job to make daily check-outs of all the WFAU Projects in the SVN repository in order to provide a backup snapshot of the current state. Continues. ACTION: ETWS/NCH purchase a new license for mxODBC. Continues. Specific points and new actions: ------------------------------- Project management: * RGM reported that Colin Vincent had agreed an amount for a one year extension of the WFAU staff effort. Consequently a JeS for this amount was being prepared. The forthcoming visit by Lockheed Martin had been postponed until the New Year, with the date still to be set. Also they would shortly be circulating draft documents specifying the work to be done. RGM would shortly circulate the documentation for the forthcoming annual PDRs. ACTION RGM circulate documentation for the PDRs. Finally the next WFAU Quarterly Meeting would be postponed until February 2015 and the precise date would be set to be convenient for DM and SV. WFCAM, VISTA and VST updates: * Normal operations seemed to be continuing. We were currently awaiting data from CASU from July 2014 onwards, which was a larger backlog than normal. * UKIRT had formally been handed over to a consortium of American universities. It was still undecided what contribution CASU, WFAU and, indeed, the UK would make to any new WFCAM survey. This issue would be become important if we needed to buy more storage to accommodate any such survey. Networking: * Nothing to report; everything appeared to be working satisfactorily. WSA/VSA/OSA/GES Operations: * ETWS reported that he had: - started ingesting the VVV catalogue data from P92 onwards into a separate VVVP92 database, - fixed the broken headers in the OSA data (missing keys APERCORs, etc.), - finished ingesting the OSA data, which was hampered by quite a few files containing duplicated rows, - ingested metadata and catalogues of the reprocessed v1.3.1 VVV data. * ACD reported that the GES iDR3 spectra had been successfully ingested. Preliminary checks of the iDR3 analysis files had not revealed any significant problems. The main challenge of iDR3 would be the few fields duplicated between iDR2 and iDR3 and investigation of this issue was in progress. Hardware and Systems: * MSH reminded the meeting that next Friday (5/12/14) would be 'blackout Friday' when all WFAU servers would be unavailable while essential system administration was carried out. ACTION: MSH to ensure that a public announcement about the service disruption was issued and widely advertised. The only other item to note was that a few disks had died. Software: * NJC reported that he had worked on: - a couple of fixes to the VVV software, - bringing the matched aperture pipeline branch back into sync with the trunk development branch, - the proper motion software deep surveys, - the VSA recalibration problem that arose a couple of years ago; this issue was connected with the survey tiling. * ETWS noted that he had: - extended the code for ingesting and monitoring the OSA uncorrected and/or list-driven data, - Written code to fix header problems (missing keys APERCORs, etc.) in OSA data. Review of Deadlines: * ACD noted that he split the list of deadlines into two separate twiki pages: (i) a list of the surveys currently being ingested and prepared for release and an associated list of past releases and (ii) a list of other deadlines: conference dates, registration dates, grant submission deadlines etc. He presented a draft list of the surveys currently being ingested, which was discussed and will subsequently be modified in line with these discussions. The URL for the umbrella page from which these lists can be accessed remains unchanged. ACTION ETWS/NJC: develop an automated tool to monitor progress in ingesting and curating surveys. ACTION NJC: send ACD a list of forthcoming conferences for 2015. Survey Data Release: * The discussion of survey releases was subsumed into the review of deadlines, above. Non-survey Data Release: * ETWS reported that there had been one release of VISTA non-survey data, for Richard McMahon. Astrogrid deployment, VO & Data Analysis services: * MSH reported that he had prepared several coverage maps of VISTA data. * RGM noted that Andy Connolly would like to obtain a copy of low-latitude data from the GPS and VVV surveys in order to test the current LSST stellar models in the Galactic Plane. These tests were being made because the stellar density there determines the overall computational cost of operating the LSST reduction pipeline. ACTION MSH: provide Andy Connolly with appropriate programmatic access to the WSA and VSA surveys. Miscellaneous: * ACD noted that WFAU was making a disproportionate contribution to the dispensation of Scottish justice. Staff had been summonsed for jury service as follows: - RPB on 2/12/14 (High Court), - ACD on 3/12/14 (Sheriff Court). In addition RSC had also been summonsed for jury service during October. ACD, Original: 29/11/14, Revised: 4/12/14.-- Scanned by iCritical.