From nch@roe.ac.uk Sun Mar 28 08:38:56 2004 Date: Fri, 26 Mar 2004 14:38:04 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes, 26 March 2004 Minutes of WFCAM Science Archive meeting: 26th March 2004 ------------------------------------------------------------ ------------------------------------------------------------ Present: MAR, NCH, ETWS, MCH, PMW, IAB, JDT Apologies: JPE, RGM, HMG, JMS, GPS, AL DONM: 10am, Friday 2nd April 2004, plate library. Actions discharged: ------------------- No actions last week. Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). - continuing; thanks for progressing these. Actions carried forward from 19/03/04 meeting: ----------------------------------------------- None. Specific points and new actions: -------------------------------- Project management: Plan of work for Q2: the team discussed the draft plan in the light of completion of tasks allocated to Q1. Some residue (mainly testing) will be carried forward into Q2; otherwise, the next high priority tasks were examined and PMW will fold these into a new draft for perusal in the afternoon. ACTION: PMW to redraft the Q2 plan and discuss with NCH Friday PM. NCH noted that the VDFS URD had returned to him so he had sent a collated set of comments back to WJS. NCH & PMW both had conversations with WJS afterwards and he is most sympathetic to our suggestion of reigning in some of the more strict requirements and recasting them as less strict requirements with strict goals. MAR reminded the team that at the local VO team meeting on Wednesday it transpired that Astrogrid have convened a Deployment WG which will likely be visiting over the next few months to discuss deployment of infrastructure on our datasets; this would inevitably require some staff effort which PMW noted should be included in the Q2 plan. Comments and issues arising from CASU fortnightly minutes: No new minutes this week. Networking: Nothing new this week. Hardware: NCH has been in touch with both Eclipse and Andy Knox (IBM) concerning the U320 problem. The consensus of opinion now seems to be that the use of so many disks employing U320 (which places demands on the motherboard CPU via device drivers) and software RAID is simply saturating the processors during read/write and causing unserviced IO requests to produce errors. NCH has discussed this at length with Eclipse and agreed that the way forward should be to employ hardware RAID adaptors provided there is no significant degredation in IO performance (over the performance available to us with SW RAID and U160 speed). Eclipse have already ordered an LSI "MegaRAID" 4-channel Ultra320 adaptor that supports, amongst many others, RAID50 configurations (fault tolerant RAID 5 on each separate channel and then a spanned RAID 0 stripe across those four LVs). The hope is that moving the Ultra320 processing requirements and striping off the motherboard will alleviate the problems; at the same time we would hope to be able to employ fault tolerant RAID configurations at no performance cost, or potentially even improved performance. Eclipse are expecting delivery of the first RAID adaptor next week, and will then work closely with NCH on the configuration and benchmarking. Software: IAB reported: "Have started working on the underlying code for CU2 - the generation of compressed images. I've written some low level C code for creating a JPEG for each image HDU of a MEF file. This uses call to libjpeg. Doing it this way would be more efficient then utilities such as ImageMagick. Also recieved a dummy flat and confidence frame from JRL." ETWS noted that bug fixes have been applied to the schema parsing scripts in the light of some refinements of the SSA documentation. SSA: NCH noted that he had discovered a very small "zero hour" bug in the SSA neighbour tables. The CVS scripts have been bug fixed, and a fixing script to purge the duplicated rows from the existing tables has been written and checked into the CVS; it will run over the weekend (this saves spending another week or more recreating all the various cross-neighbour tables with DR1, 2MASS, USNOB etc). All previosuly unresolved interface and documentation issues have been resolved and we anticipate deployment of the full SSA early next week. Miscellaneous: NCH noted that JPE had communicated the announcement of the e-Science "All Hands Meeting" in Nottingham (August 31 to September 3rd). AL & RGM will both be attending; an abstract for a poster will be submitted by NCH detailing the science archive activities at WFAU. ACTION: NCH to submit a poster abstract for the e-Science "All Hands Meeting"