From acd@roe.ac.uk Wed Oct 15 12:02:43 2014 Date: Wed, 15 Oct 2014 11:02:29 +0000 From: Clive Davenhall To: Bob Mann , Andy Lawrence , "Read, Mike" , Eckhard Sutorius , Mark S. Holliman , Nicholas Cross , Nigel C. Hambly , Rob Blake , Ross Collins , Stelios Voutsinas , Dave Morris , Clive Davenhall Cc: "dante@astro.puc.cl" , James Dunlop , "j.p.emerson@qmul.ac.uk" , ktn keithnoddle.org , "Lorenzo.Rimoldini@unige.ch" , "mcioni@aip.de" , "matt.jarvis@astro.ox.ac.uk" , "mike@ast.cam.ac.uk" , "nigel.metcalfe@durham.ac.uk" , "norman@astro.gla.ac.uk" , "pwl@star.herts.ac.uk" , "rgm@ast.cam.ac.uk" , "Simon.Dye@nottingham.ac.uk" , "s.j.warren@ic.ac.uk" , "tkerr@jach.hawaii.edu" , "tom.shanks@durham.ac.uk" Subject: WFAU VDFS Science Archives progress meeting minutes, 3 October 2014 WFAU VDFS Science Archives progress meeting minutes, 3 October 2014 Present: MAR, RPB, ETWS, RGM, MSH, ACD Apologies: NJC, RSC, STV, DM, NCH, KTN, AL DoNM: 10am Friday 24 October 2014 in the Villas meeting room, MAR in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: MAR to contact Phil Lucas about the GPS DR10 release so that he could test it before going live. Done. MAR had contacted Phil Lucas who is now testing the release. We await his confirmation that the release can be made live, ACTION: NJC to contact Phil Lucas to see if he's happy for VVV DR2 to be made public. Done. NJC was not present, but MAR reported that NJC had contacted Phil Lucas who wanted to conduct further tests. Actions partly discharged but continuing: ---------------------------------------- ACTION: RPB (was MAR) to have a look at the status of non-survey releases for registered projects. MAR had compiled a list of non-survey projects that could be released and sent it to RPB. RPB had yet to make these data available. Actions carried forward from 5/9/2014 meeting: ---------------------------------------------- ACTION: RGM to investigate the size and content (how does it differ from DR9?) of DR10 to see if it's worth us hosting and whether it is necessary to keep all of the other releases. Continues. The first part had been done: the difference between DR9 and DR10 is largely the presence of additional spectra. However, the action continues as RGM had not yet established which early releases could safely be scrapped. For further details see the WFAU SDSS strategy wiki page: http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this action still has low priority. ACTION: ACD to review VVV brainstorming Twiki page and decide who to chase about progressing their assigned actions. Continues. ACTION: RPB to write a script to backup the SVN repository. Continues. This action also spawned two new actions: NEW ACTION: ACD speak to HME about the possibility of IT support making nightly backups of the SVN repository. NEW ACTION: RPB write a chron job to make daily check-outs of all the WFAU Projects in the SVN repository in order to provide a backup snapshot of the current state. Actions from the 19/9/2014 meeting: ----------------------------------- ACTION: RPB/ETWS to see which data on /sahure were no longer required. Continues. ETWS and RPB should remove any unwanted files. Specific points and new actions: ------------------------------- Project management: * RGM noted that several members of WFAU had received stage 1 redundancy letters. These missives had been triggered by several grants finishing on 31/3/15. However, it was anticipated that replacement or extension grants would be in place by then and no redundancies would result. WFCAM, VISTA and VST updates: * ETWS noted that there had been a two month hiatus in receiving WFCAM data due to the telescope operating other instruments. RGM noted that there was some discussion over whether we would, in the future, have to archive observations from these instruments. Comments and issues arising from CASU minutes: * Mike Irwin had mentioned that CASU no longer holds progress meetings, so no more minutes would be issued. Henceforth this item would not appear in the agenda. Review of Deadlines: * The list of recently-passed and forthcoming deadlines was reviewed and the list will be updated to reflect this discussion. The list can be viewed at: http://apache.roe.ac.uk/twiki/bin/view/WFAU/ForthcomingDeadlines ACTION: RPB to check that status of the VVV DR2 ESO release scheduled for 15/9/14. Networking: * The networks for WFAU operations were working normally. ETWS noted that the re-processed VIDEO data and June VISTA data had been transferred successfully. WSA/VSA/OSA/GES Operations: * ETWS reported that for the VSA: - He had ingested the metadata and catalogues of the re-processed VIDEO data and run AutoCurate up to the stage that RPB could produce a release. - The ObjID update of the 51 billion VVV detections has finished. Now only the odd reprocessed files need to be ingested on Phil Lucas' request and then we can start with the next stage. - JPGs have been created for the VISTA May and June data and metadata ingests will finish over the weekend. * For OSA he noted: - OSA catalogue ingests continue. * ACD reported that he had been attempting to ingest the iDR4 linelist so that it could be made available to Consortium members, who would use it to obtain the atomic data they needed for their iDR4 analyses. However, some small changes to the format of the FITS files caused the ingest script to crash. Investigation was in progress and the problem was likely to be minor. Hardware and Systems: * The WFCAM archive tape drive had arrived. In fact it had been sitting in a box on its delivery pallet in the machine for a fortnight without any of the operations staff realising what it was. * MSH noted that updates had been applied to all WFAU systems to protect them against the shellshock bug. Software: * ETWS reported: - A major re-write of CU1 to transfer the WFCAM satellite data from non-standard directories and sub-directories at CASU. - Enhancements to fix the incorrect object/targets present in some VVV files and the corresponding values in the database. * MAR noted that he had made a small change to the user interface. Survey Data Release: * An earlier VMC release had been corrected by the addition of a PSF catalogue that had unfortunately been omitted. * A VIDEO release was being prepared. Non-survey Data Release: * None. Astrogrid deployment, VO & Data Analysis services: * MSH noted that he had received an extensive report from HEASARC detailing various ways in which WFAU VO services did not fully comply to the VO standards. Many of these issues were connected to the incomplete TAP implementation that WFAU uses (and would disappear when it is replaced with an improved version). MSH would investigate some of the other problems, but this work has lower priority than EUCLID. Miscellaneous: * MSH reported that the racks and equipment in C2, the machine room, were to be re-arranged to improve ventilation, following advice from a heating engineer. This reorganisation would require one or two days of down-time early in the New Year. ACD, Original: 3/10/13, Revised: 15/10/14. -- Scanned by iCritical.