From nch@roe.ac.uk Mon Dec 19 17:41:30 2005 Date: Mon, 19 Dec 2005 11:42:05 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , John Taylor , Jim Emerson , Malcolm Stewart , Martin Hill , Mike Irwin , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly project meeting minutes, 16 December 2005 Minutes of WFCAM Science Archive meeting: 16th December 2005 ------------------------------------------------------------- ------------------------------------------------------------- Present: NCH, ETWS, NJC, JB, MAR Apologies: JPE, JDT, MCH, RGM, AL, JMS, PMW, RSC DONM: 10am, Friday 6th January 2006, plate library Merry Xmas to all our readers. Actions discharged: ------------------- ACTION: JB and MAR to close out non-survey release of available DBs asap. Discharged; all 05A non-survey DBs are now online and the respective PIs have been informed. Actions partly discharged but continuing: ----------------------------------------- ACTION: JDT to set up a Datascope Launcher for the WSA on the AG workbench pages. Continues. Actions carried forward from 09/12/05 meeting: ---------------------------------------------- ACTION: NCH to investigate the usefulness of a NAS solution for medium term archive mass storage. - CONTINUES ACTION: ETWS and NCH to sort out image DQC schema and ingest - CONTINUES; note also some additional things that have arisen from UKIDSS SV (see below). ACTION: NJC to send some examples of apparently inconsistent image attribute sets to JAC. - CONTINUES Specific points and new actions: -------------------------------- Project management: NCH noted that there is a Vista surveys planning meeting in London on the 20th; both he and AL will be in attendance and NCH will fold any general requirements that arise into the current analysis of Vista requirements being conducted. NCH reported back from yesterday's final UKIDSS SV meeting, and the following issues and wish list came up: Bugs: 1) Check that the merged source list pairing is producing clean lists 2) Catch incorrectly reobserved frames as NOT multiple epoch revisits and generally make frame merging a little more intelligent over choice of frame when there is more than one QC: a) Catch frames with bad channels / large spurious detection numbers b) " " " moon ghosts c) Ensure all available DQC is propagated into metadata tables (e.g. sky sub scale parsed from FITS comment). d) Implement pre-release QC in collaboration with SJW. Wish list: a) Include SQL query in results set output in appropriate format(s) and echo to the results page b) Client-end query tool for scripting c) Seeing in arcsec as a new attribute d) > 100 columns for formatted HTML summary results e) larger limit to results set size f) petro (kron/hall?) radius in merged source table g) document/implement default flux estimator as Aper3 as being generally appropriate h) Guide for simple usage to prevent "frequently made cock-ups", e.g. careful filtering when using merged source tables i) Sexagesimal format on results page and consistent orientation of thumbnails j) Dictionary translation between pipeline FITS keys and table attributes k) Reduce source pairing radius for GPS, as per WG wishes WFCAM update: See the stop press pages linked from the TWiki for latest news. Comments and issues arising from CASU fortnightly minutes: No new minutes as of 16/12 AM. Networking: A message has been received from the UKLight folks informing us of progress on the Edinburgh-end infrastructure, and asking how we would like to progress forward to testing. ETWS has received a positive response to his initial enquiry about obtaining SDSS-DR3 from Chicago; this will be revisited in the New Year if no further response has been received. WSA Operations: JB noted that general ops are progressing well: "ETWS and I have now worked out what all the problems were with the CU4 catalogue files that did not ingest were and I am about to email CASU with details. Essentially: missing keywords and keywords with NaN values in the catalogue files. The 05A Non-Survey Databases are now wrapped up (I believe). All databases (UKIDSSR2, Non-Surveys Databases and WSA) have had the JPEG data entered for all files as well as the fix for the 20050414 data due ot the files being moved. CU19 is currently running for WORLDR2 and may be complete by tomorrow morning. System backups went fine, Non-Surveys Databases have also been backed up and the WSA along with WORLDR2 will also be dumped to tape once it have finished being created. Ahmose has had a virus scan run on it and came up clean." Hardware: JB noted that all is quite on the hardware front. "No hardware problems with Djoser, Snerferu, Khufu, Ahmose or Amenhotep accepting the reboot for Djoser, Snerferu and Khufu due to a linux kernel patch." Software: RSC reported debugging/developing the new Cu16 script. MAR reported: "Finished off non-survey from the interface point of view and emailed users." ETWS reported: "Finished the refactoring of CU4. Files with keywords missing from a given list are now rejected. Included a maximal date in CU 21 for the ingestion of non-survey files and sorted the non-survey programmes in the browser pages alphabetically. Created the ingestable GLIMPSE catalogues and included GLIMPSE into the browser pages. Finally, included a date range into CU1, so that we can download specific directories from the given list of OK_TO_COPY-directories. And fixed the ls output format differences between djoser (debian woody) and khufu (debian sarge)." NCH noted a very low-level bug had been found in the pairing codes (there are 0.1% duplicate detections in the merged source table for some filters, contrary to the philosophy of each detection being propagated once and once only into the merged source table). ACTION: NCH & RSC to chase up and fix the pairing code bug. Survey Data Release: No news this week, other than the "world" readable version including a small UKIDSS taster is about to go live. Non-Survey Data Release: All 05A non-survey data have been released to their respective PIs - thanks to ETWS, MAR, JB for their hard work on this. Astrogrid deployment: MAR reported: "Attended AstroGrid AG-DAG meeting in Leceister which was mainly an introduction to AG and what it can offer." NCH noted that there is an Astrogrid Technical Workshop scheduled for the 17th to 20th January at NeSC. WSA team attendance at this will be decided at a future date when a more detailed programme becomes available. Miscellaneous: Nothing further this week.