From nch@roe.ac.uk Fri Oct 3 14:20:00 2003 Date: Fri, 3 Oct 2003 11:58:25 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Clive Davenhall , Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: Andrew Lawrence , Peredur Williams , Andy Adamson , Jim Emerson , Mike Irwin Subject: WFAU WSA weekly meeting minutes: 3rd October 2003 Minutes of WFCAM Science Archive meeting: 3rd October 2003 ----------------------------------------------------------- ----------------------------------------------------------- Present: JMS, AL, ETWS, IAB, PMW, NCH, MAR Apologies: RGM, HMG, JPE Actions discharged: ------------------- ACTION: PMW & NCH to flesh out the Q4 forward plan. (this was done, has been presented to the VDFS management group and was accepted.) ACTION: NCH to contact IBM and Compusys with a thinly-veiled request for advice on the Ultra320/Tyan server instabilities. (IBM have replied in a rather negative way - "what do you expect from home-brew software RAID solutions on 32-bit PC-type motherboards" etc; some useful feedback from Paul Hirst at JAC to whom the same info was sent; nothing from Compusys as yet. See Hardware below for the latest info.) - all done and discharged (thanks to all concerned!) Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. ACTION: RGM to progress design of a web cookbook based around the 20 Queries. (started) (still continues: 2/3 done) ACTION: NCH, ETWS to create schema-driven tags for ingest procedure, and to code up proto-curation populating scripts for the WSA for test purposes. (schema tags to drive ingest are being implemented in revised schemas in the CVS. Proto-curation populating scripts have yet to be started) - continuing; thanks for progressing these. Actions carried forward from 19/09/03 meeting: ---------------------------------------------- ACTION: PMW to liaise with AL to make sure that Compute Support are made aware, from the highest "official" level, of our requirements, re. 100 Gbyte/day transfer from CASU, and to impress on them that the 1 Gbit/s FW should be their highest priority. ACTION: JPE to design and set up centralised VDFS web pages. - CONTINUE Specific points and new actions: -------------------------------- Project management: NCH reported that he, PMW & AL had attended the quarterly face-to-face VDFS management meeting at which Q3 progress was reviewed and found to be good. PMW & NCH will attend the AGOC review meeting next Friday at Cambridge which is mainly to review management procedures but NCH and MJI have been asked to attend so that they can be on hand to give more technical details on day-to-day work. RGMcM (IoA) has set up a Wiki page at Cambridge which is intended to archive VDFS management materials; this is most welcome as a repository for the various documents etc. for VDFS management. NCH expressed the view at the quarterly meeting, and reiterated in this meeting, the view that a completely open, public website that gives a VDFS overview and "business plan" is needed in addition to any non-public Wiki pages. Hardware: Some small progress has been made on the catalogue servers, in that a highly suspect disk has been removed from the ahmose array, and that a possible suspect cabling problem on the ahmose JBOD has been identified. Work with JNTD & Eclipse is on-going. In the meantime, NCH is going ahead with full and part-loads on the two servers to exercise the disk arrays as much as possible, at the same time keeping an eye on the system logs. JNTD has indicated to PMW (& NCH after the meeting) that there is also a suspected problem with the Seagate disk firmware on the 15 krpm, 150 GB Seagate Ultra320 HDs. An update is being pursued by Eclipse... NCH reported that HME is progressing the setting up of djoser, including installation of Debian Linux, setting up of user accounts for the team (including hopefully a group "wsa" account) and the requested RAID arrays and mount-points. Software: IAB reported: "The CVS now contains working (but untested) code for generating intermediate CSV files to allow ingestion of multiframe image metadata and data associated with detected sources on these frames. A major enhancement in this code is an exception handling mechanism to make the procedures more robust. The aim here is to remove all risks of corrupting the database due to unchecked irregularities." ETWS reported: "I've included the FITS table TTYPE as a new tag in the sql Schema files. USNO B is to 90% processed, and the other catalogues have been reprocessed to include a sequence number." Networking: A constructive meeting between ATC Compute Support, IfA AVO team and the WSA team was held concerning the forward-look plan for networking and data services, including the WSA. The immediate problem of potential port blocks on the SRIF network is being pursued via EUCS, who have indicated that unless the SRIF management raise objections, the necessary ports will be opened up to enable public access of the WSA web server. SSA: For the purposes of attaching the WSA server to a working SQL Server, a work-around has been achieved by MAR that allows thoth to connect to grendel01 (containing a 0.1% SSA dataset) via grendel12. This at least allows all functionality of the SSA to be enabled (eg. for the AGOC meeting on Friday). NCH particularly welcomed the thumbnail functionality that MAR has implemented. MAR reported that interface enhancements are essentially complete. The issue of full-blown browser-launching functionality was once again raised by AL... MAR is looking into the practicalities of this using software and experience from RGM's MSc student. The feeling is that this need not delay the imminent launch of the SSA. Miscellaneous: No meeting next week due to PMW & NCH being in Cambridge for the AGOC overview of VDFS management. ACTION: NCH to type up and circulate these minutes. DONM: 10am, Friday 17rd October, plate library.