From j.p.emerson@qmul.ac.uk Fri Apr 25 17:12:25 2003 Date: Fri, 25 Apr 2003 16:29:25 +0100 From: Jim Emerson To: Malcolm Stewart , Mike Irwin , Peredur Williams , Richard McMahon , Andy Lawrence Subject: WFAU WSA weekly meeting minutes: 25 April 2003 VDFS Mgt team Please read these WFAU meeting minutes and feedback any comments to me. Jim --- Minutes of WFCAM Science Archive meeting: 25 Apr 2003 ----------------------------------------------------- ----------------------------------------------------- Present: NCH, HMG, IAB, MAR, RGM Apologies: PMW, ETWS, AL In Attendance: JMS Actions discharged: ------------------- - all done and discharged (thanks to all concerned!) (none this week, mainly owing to review & Easter hols). Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. - continuing; thanks for progressing these. Actions carried forward from 11/04/03 meeting: ---------------------------------------------- ACTION: JMS to review software tasks in build GANTT and comment on the plan. ACTION: PMW to liaise with AL to make sure that Compute Support are made aware, from the highest "official" level, of our requirements, re. 100 Gbyte/day transfer from CASU, and to impress on them that the 1 Gbit/s FW should be their highest priority. - CONTINUES Specific points and new actions: -------------------------------- Project management: NCH reported that the general feeling from the review was that it was successful and left the panel and attendees with a good impression. NCH thanked the team for their efforts in putting up a good doc set and a good show (PowerPoint files are on the external web site). NCH thanked RGM for keeping extensive notes, typing them up and putting them on the TWiki site. We expect formal review panel feedback within the next week or so, at which point specific actions will be allocated in conjunction with our own notes from the meeting. NCH expressed the wish that we must try to keep up momentum now that the review was over. The immediate priorities were discussed (as below): hardware, SSA, and software. Hardware: The point at which a hardware decision is needed is fast approaching. NCH & RGM had negotiated loan of an UltraSCSI RAID-adapter solution set-up from IBM; this system is being benchmarked today. NCH also reported contacts with Martin Farrow of Compusys, who is willing to offer advice on RAID configurations. HMG reported that Eclipse have set up an Ultra320 system (a la Jim Gray experiment), and would run MemSpeed and IOMeter tests on it on Monday and report the results back to us (unfortunately, they are unable to loan us this equipment for sufficiently long for an SQL Server trawl test). MAR suggested that there is a compromise solution for the immediate acquisition: we can purchase a fibre-IDE solution for the SSA, and then if optimisation fails to produce the required performance the disks can be used to populate the pixel server, the server itself can be utilised in an upgraded SCSI system, and the only redundant equipment would be the fibre-IDE RAID controller (ie. only a few K spent). HMG expressed concern over the amount of storage on the cosaxp6 RAID server. NCH echoed this, pointing out that several Tbytes of storage are needed as a staging area for intermediate files to get SSS data into SQL Server. MAR expressed concern over the web server for the SSA system; all agreed that it was likely that a specific system would be needed since cosaxp6 is over-burdened and the web server should have good performance if external users would be exercising it. RGM suggested a node of grendel could be used for this purpose; simply need an installation of linux/apache/tomcat. Software: NCH expressed concern over the way forward on the coding effort over the build phase. JMS pointed out that a compromise is necessary between formal layered, modular design and rapid prototyping (which inevitably results in monolithic code that is harder to maintain). Highly layered and abstracted modular code may not be appropriate in the WSA case, but IAB suggested a middle course where rapid prototyping experiments should be started now in conjunction with pushing forward on the formal design by identifying code modules within the existing use case analysis. NCH reported that AB from the ATC had offered to meet and advise on the way forward with formal design. ACTION: IAB to look at the highest priority use cases, and identify code modules. ACTION: NCH to meet with AB to discuss general approach to coding. RGM pointed out that one immediate issue was the question of pixel compression (lossless) for the large pixel volumes - this had arisen at the review. If a significant reduction in stored volume is available, then it would be good to know now what the gains and issues are likely to be. IAB agreed to start to look into this, using UFTI data. ACTION: IAB to obtain UFTI IR images and test out compression options: gzip, bzip2 & RICE (within CFITSIO). MAR agreed to continue working on user interface software for the SSA with regard to WSA functionality. Miscellaneous: Immediate priority was the protoype archive system, SSA with due regard to work in common with WSA implementation: NCH will revise schema/function/procedure/cross-neighbour scripts as necessary and put under version control; also to finalise bulk loading procedure. RGM agreed to continue with indexing experiments (and also pushing forward the V2.0 R&D phase). ETWS is on leave this week - NCH would liase with him on his return to agree work for next week on data transfer tasks (eg. installation and experimentation with gridFTP, and TCP/IP tuning. ACTION: NCH to liase with ETWS on network transfer work for the coming week. ACTION: NCH to type up and circulate these minutes DONM: 10am, Friday 2nd May, plate library.