From mar@roe.ac.uk Fri May 2 09:36:37 2014 Date: Fri, 2 May 2014 09:19:29 +0100 From: Mike Read To: VDFS Science Archive Team -- Clive Davenhall , Dave Morris , Eckhard Sutorius , Mark Holliman , Nicholas Cross , Nigel Hambly , Rob Blake , Ross Collins , Stelios Voutsinas Cc: CCs for VDFS -- Alastair Edge , Andrew Lawrence , Bob Mann , Dante Minniti , James Dunlop , Jim Emerson , Keith Noddle , Lorenzo Rimoldini , Maria-Rosa Cioni , matt.jarvis@astro.ox.ac.uk, Mike Irwin , Nigel Metcalfe , Norman Gray , Phil Lucas , Richard McMahon at IOA , Simon Dye , Stephen Warren , Tom Kerr , Tom Shanks Subject: WFAU VDFS Science Archives progress meeting minutes, 25 April 201 WFAU VDFS Science Archives progress meeting minutes, 25 April 2014 Present: MAR, RPB, RSC, MSH, ETWS, NJC, RGM Apologies: NCH, KTN, AL , ACD DoNM: 10am Friday 9 May 2014 in the Villas meeting room, ACD in the chair (room is booked). Actions discharged this time: ---------------------------- ACTION: NJC to sort out the magnitude limits for V1.3 data. Done, NJC had checked in the code. ACTION: RSC to investigate modifying CU7 to recover ATLAS VST unmatched sources around frameset boundaries. Discharged (for now). RGM reported that there was no real interest in this at the recent ATLAS meeting in Durham. Re-writing the source merging was a fairly major undertaking for a modest gain in recovering real matches. The potential availability of tiles at some future date would also reduce the effect (less edges). With these in mind and the fact that the matches are somewhat recoverable via the existing atlasSourceNeighbours table it was decided to provide users with additional documentation on the issue and how it can be addressed. NEW ACTION: MAR to document issue with ATLAS unpaired sources and give examples of recovering matches. NEW ACTION: MAR to email Monika Petr-Gotzens details on and examples of the ATLAS pointing offsets. Actions partly discharged but continuing: ---------------------------------------- None. Actions carried forward from 11/4/2014 meeting: ---------------------------------------------- ACTION: ACD, RPB to formulate a policy for backups and for checking that data could be recovered from the backups. Continues. Notes on a proposed policy are available on the Twiki. No decision had yet been made on backing up the SVN repository. ACTION: RPB to put all prices of additional items associated with the backup policy on the Twiki pages. Continues. ACTION: RGM to investigate the size and content (how does it differ from DR9?) of DR10 to see if it's worth us hosting and whether it is necessary to keep all of the other releases. Continues. RGM had started a wiki page for WFAU SDSS strategy (http://apache.roe.ac.uk/twiki/bin/view/WFAU/SDSSDRPlan) but was not present to report progress. ACTION: ETWS, with RPB, to develop an automated helper script to keep release DB files links updated. Continues: this one still has low priority. Specific points and new actions: ------------------------------- Project management: The WFAU Quarterly would take place Monday 28th April, starting 1:30pm. WFCAM, VISTA and VST updates: The hand over of UKIRT was still pending. Comments and issues arising from CASU minutes: No minutes had been received since the previous meeting. Networking: Working. WSA/VSA/OSA/GES Operations: ETWS reported: -- Restarted the objid update on the VVV after its hick-up. The parallelisation works well and we're already halfway through P89. Hardware and Systems: The test Nytro machine was broken and was back with Eclipse for repair. MSH reported that the new NAS box was hooked up. Survey Data Release: RPB noted that the VMC P91 release should be finished next week. GES DR2 would be attempted over the next few weeks. RGM gave some feedback from the Durham ATLAS meeting. This was generally positive. Issues that came up were that earlier access was wanted, that the Chilean U-band observations were to be included and that users wanted a way of seeing the value of the illumination corrections applied. ACTION: ETWS to liaise with STV over accessing OSAProprietary (synced DB of metadata) through the new UI ACTION: RGM to check on what is required re the Chilean U-band data. ACTION: ETWS to investigate modifying CU4 to calculate and store illumination correction values. ACTION: MAR to apply P91 ATLAS QC. CASU had applied nebulosity filtering to GPS frames wanted for DR10, we were in the process of transferring and ingesting these. Software: RSC reported: VVV performance testing with Nytro as a catalogue server had to be postponed due to the hardware failure. Instead we focused on testing optimisations to chunk sizes, which for half-sized chunks revealed an overall 50% performance improvement on outgests, with only a small improvement on ingests (for chunk sizes a tenth of the original size we only gained a further 10% in overall performance). This only brought us back to DR1 performance times, albeit with much larger variances, and essentially outgest/ingest times are now the same, with both effectively hitting the mysterious 100 Mbps limit. More advanced techniques will need to be tested to break through this limit. ACTION: ACD to arrange follow-up VVV brainstroming meeting. ETWS reported: -- Installed the 3rd party software on unas and updated the OpenSSL on menkaure for the high performance transfers. Non-survey Data Release: RPB reported that several 12B projects were currently going through the release process. Astrogrid deployment, VO & Data Analysis services: MSH would be attending the IVOA at ESAC in mid-May. Miscellaneous: VMC meeting at ROE 19-21 May 2014. -- Scanned by iCritical.