From nch@roe.ac.uk Sat Dec 3 15:15:41 2005 Date: Fri, 2 Dec 2005 12:28:28 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , John Taylor , Jim Emerson , Malcolm Stewart , Martin Hill , Mike Irwin , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly project meeting minutes: 2nd December 2005 Minutes of WFCAM Science Archive meeting: 2nd December 2005 ------------------------------------------------------------- ------------------------------------------------------------- Present: NCH, RSC, PMW, ETWS, NJC, JB, JMS, MAR Apologies: JPE, JDT, MCH, RGM, AL DONM: 10am, Friday 9th December 2005, plate library Actions discharged: ------------------- ACTION: NCH and PMW to meet next week to insert WFAU requirements in the EUCS questionaire. Discharged; NCH sent some blurb to RGM for inclusion and PMW has checked over the document. ACTION: NCH, MAR & ETWS to meet this afternoon to review the state of readiness for release of non-survey DB products. Discharged. Actions partly discharged but continuing: ----------------------------------------- ACTION: JDT to set up a Datascope Launcher for the WSA on the AG workbench pages. Continues. ACTION: JB to look at the GLIMPSE info and begin SQL implementation with help from ETWS and NCH. Continues; JB has started designing the SQL Schema and ETWS has started working on the code to produce the intermediate ingest files. PMW asked it the survey was complete, emphasising that we shouldn't ingest until the catalogue data product is in it's final form. PM volunteered to check this out. Actions carried forward from 25/11/05 meeting: ---------------------------------------------- ACTION: NCH to investigate the usefulness of a NAS solution for medium term archive mass storage. - CONTINUES ACTION: MAR to put a link to the CASU survey progress pages from the WSA. ACTION: NCH and PMW to remember to factor the following things in to to the Q1'06 plan of work: 4/ Quality control: flagging of residual cross-talk images, 128 pix either side of v. bright / saturated objects in single-passband catalogues 5/ Application of 2d photometric correction maps of systematic calibration errors (of order a few %) measured from all 2MASS standard residuals, averaged as a function of position on frames 6/ New data model required for poor-seeing time programme (multi-epoch photometry in search of planetary transists) also need to include tasks on UKLight and outside access to the continually updating database of ingested pipeline products. Specific points and new actions: -------------------------------- Project management: NCH and JMS noted that the next VDMT was scheduled for the 6th but it now seems that most people cannot attend (VEGA-GAIA PPRP and other committments) so the telecon is being rescheduled (9th was suggested but this looks impossible now as well). PMW asked that we complete the progress report for the VDMT anyway - the team did this during the meeting. PMW asked that he and NCH meet for a planning meeting early next week, to close out this and other issues ACTION: PMW & NCH to have a planning meeting, Mon 5th PM. JMS also reminded the team of the tentative UK review schedule: a formal review is scheduled for April next year. WFCAM update: See the stop press pages linked from the TWiki for latest news. Comments and issues arising from CASU fortnightly minutes: No new minutes as of 2/12 AM. Networking: JB, ETWS and NCH noted that it had become apparent yesterday (1st Dec) that the WFCAM catalogue data for the 17th April was missing from the WSA. This is a major concern, and is being investigated at the moment. It could be a problem at either end of the CASU/WFAU interface, so housekeeping and logs etc. at both ends need to be scrutinised to track down the source of the problem. ACTION: ETWS to email MR to ask if anything out of the ordinary happened at the CASU end wrt 17th April data ACTION: JB & ETWS to examine CU1 housekeeping and logs to see if there is a bug that resulted in missing catalogues for the 17th April ACTION: MAR to put a note on the WSA website noting that the catalogue data for the 17th April is missing in the current release ACTION: NCH to email SJW a warning to be circulated amongst the UKIDSS Science Verifiers that April 17th catalogues are missing. Finally, NCH noted that RGM has noted that Alex Szalay is here next week, so if we have any questions about SDSS DR3/4 availability this may be a good opportunity to ask. WSA Operations: JB reported that he has been finishing off the 05A ingest; and has secured a new full backup of the WSA. Hardware: JB noted that the new UPS has been installed without any problems on the file server khufu; otherwise all is quite on the hardware front. Software: NCH reported finishing off the SW mods for non-survey data; otherwise much work has been done on the Science Requiremens Analysis Document by recasting the old WSA one and folding in new stuff from the UK VISTA URD (in preparation for next year's review). PMW has supplied some comments on the rehashed SRAD. NCH has also been looking at UKIDSS SV data using the WSA interface and existing release. NJC reported further work on CU13 and CU14, including integration of the CASU toolkit codes. A small hiccup when using the source extractor turned out to be an incorrect input option (thanks to JRL for pointing this out). There was some debate at the meeting concerning tracking of versioning of the CASU toolkit codes; JMS suggested we voice any concerns we may have at the next VDMT. NCH suggested that now would be a good time for NJC to contact Alastair Edge and Omar Almaini as to what their requirements are for optimising the stacking for the DXS and UDS, since we can update them on where we are with the archive-end of things, and given that there is to be a release in the New Year. ACTION: NJC to contact DXS/UDS to update them on WSA stacking/ mosaicing and ask about their required optimisation of release stacks/mosaics. RSC reported continuing code maintenance of the wfcamsrc ingest applications, and also further work on the refactored super-efficient catalogue matching codes for CU16, noting that some permanent storage would be required on the linux file store client khufu to stage external catalogue info. ETWS reported: "Put the finishing touches on the non survey code. Installed more software on djoser and started work on the code for the ingestion of GLIMPSE." MAR reported: "Wrote a couple of SQL scripts to set-up and update various non-Survey issues relating to the user-interface. Adapted another form for non-survey use and altered lookup of default attribute lists to default to something that works (i.e. select table.*) when schema evolution has occurred." Survey Data Release: The SV 1+2 release continues to be used extensively: 1636 logged queries originating outside ROE (2x more than the figure for the first week). Non-Survey Data Release: SW and procedures now in place, tested and JB as been familiarised with the system. NCH asked that we endeavour to get the 05A data sets out to their respective PIs early next week (after the final loose-ends on the 05A ingest have been cleared up). Astrogrid deployment: Nothing new this week. Miscellaneous: Nothing further this week.