From nch@roe.ac.uk Fri Jan 30 14:41:44 2004 Date: Fri, 30 Jan 2004 11:20:54 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes: 30th January 2004 Minutes of WFCAM Science Archive meeting: 30th January 2004 ----------------------------------------------------------- ----------------------------------------------------------- Present: JMS, NCH, ETWS, HTMG, RGM, PMW, JDT, MCH Apologies: MAR, JPE, IAB, AL, GPS DONM: 10am, Friday 6th February 2004, plate library. Actions discharged: ------------------- ACTION: RGM to progress design of a web cookbook based around the 20 Queries. ACTION: RGM to distribute the SSA User cookbook on Monday to NCH, MAR, ETWS and PMW and those people to read and get comments and suggestions back to RGM before the next weekly meeting. Done - see SSA below. ACTION: NCH & PMW to lean on Eclipse to try to resolve the current hardware difficulties. No need: Eclipse contacted us. ACTION: NCH, IAB & ETWS to meet on Wed 28th pm to further iterate the integration and testing of the ingest software. Done - see Software below. Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). ACTION: NCH to attend PPRP meeting on 4/2/04 to make ANOTHER presentation as to science archive progress and forward plan. - continuing; thanks for progressing these. Actions carried forward from 23/01/04 meeting: ----------------------------------------------- None. Specific points and new actions: -------------------------------- Project management: NCH has produced a PowerPoint presentation for the PPRP review on Feb 4th in Cambrdige; PMW has looked at it and made some suggestions; there is a dry run on Tues 3rd Feb at which the full complement of VEGA presentations will be reviewed. RGM and JMS expressed a wish to attend this. ACTION: NCH, RGM, JMS and PMW to attend the VEGA dry-run telecon on Feb 3rd, 10:30am, (AL's office?) to review the presentations for the 4th. PMW reported that he has been inconsultation with JPE and MJI as to operations budget for the WSA and beyond, and estimates will be communicated to PPARC as part of preparations for the RG renewal procedure (pipeline and archive operations funding are not being sought through the VEGA bid to e-science monies). Comments and issues arising from CASU fortnightly minutes: No new minutes as of 30/1/04. Networking: NCH & RGM reported that the DR1 SneakerNet box has arrived; NCH has set up and tried a boot; but JNTD is now needed for attachment of the unit to the WSA private area network and transfer of the data. RGM inquired as to the status of the old EDR and any new EDR dataset that might be part of DR1; he suggested we should look at this carefully before deciding whether to keep or scrub the old SkyServer EDR. Hardware: Eclipse have contacted JNTD as regards the Ultra320 problems; they have been communicating with Tyan in the US, and JNTD & IAD have come up with a number of firmware updates that may be related. Server amenhotep is still in the lab awaiting a boot-up: it turns out that since the firmware updates there are BIOS errors reporting SCSI device conflicts that may be related to our disk subsystem stability problems. JNTD and IAD are continuing to investigate; NCH is keeping a close eye on developments. Software: NCH reported that he, IAB and ETWS had spent a constructive time on Wed afternoon concerning final integration and testing of CU1-4. Things are progressing well; NCH asked that the same group meets again on Thurs 6th to push this forward to completion. ACTION: NCH, IAB and ETWS to meet on Thurs 6th Feb pm for another integration/test session for CU1-4. IAB reported: "The CVS under wfcamsrc now contains all the C++ code for schema driven image metadata and catalog data extraction from FITS files. This is designed to be able to respond to "schema evolution". Each method for calculating a particular attribute is encoded as a class which inherits from an abstract "DataMethod" class. I used a design similar to the standard "command" design pattern. If we want a new method for calculating data attributes, we simply encode that method in a new class." ETWS reported: "I've been re-writing the download test program to a version in line with the demands for cu1 including the suggestions from IAB and NCH after our meeting. Also I have contacted Mike Irvin about some questions which arose from this meeting." SSA: NCH & RGM have discussed a small revision of the SSA schema in the light of the experience of the 20 Queries cookbook: Source will now get extra attributes l,b,d and Eb-v for extragalactic work, and expediting galaxy catalogue queries. IAB and NCH have been working on incorporating the Schlegel dust map reddening measurements into the intermediate files; Source will then be reloaded and reindexed. The whole procedure should take around one week, and in the light of the continued delays on stabilising the hardware for a public service it was though best to do this now rather than revise the schema & data later in a new release. These changes will greatly improve the utility of the SSA for extragalactic work. NCH reported that indexing the SSA takes 17.5 hours; CVS script has been checked in that does this and also allows for reindexing. NCH suggested that because some of the standard 20 queries contain 3-way joins in the SSA, it will be important to point users at the personal subset PSSA so as not to swamp the catalogue server. RGM further suggested that it would be good to have an automatic load of the queries available on the web interface so that users automatically tried them out on the PSSA. Further discussions ensued concerning final iteration of the cookbook. NCH pointed out that the schemas are being revised slightly so final iteration should wait for this to be complete; in any case RGM is still awaiting comments from PMW and MAR. It was decided to delay final revision of the cookbook for one week to allow for these. ACTION: PMW to get comments on the SSA cookbook to RGM within the next few days (PMW's non-expert take on the system is highly valued). Miscellaneous: Nothing else this week.