From j.p.emerson@qmul.ac.uk Fri May 16 16:53:22 2003 Date: Fri, 16 May 2003 15:30:41 +0100 From: Jim Emerson To: Malcolm Stewart , Mike Irwin , Peredur Williams , Richard McMahon , Andy Lawrence Subject: FW: WFAU WSA weekly meeting minutes: 9 May 2003 VDFS Mgt team Please read these WFAU meeting minutes and feedback any comments to me. Jim -----Original Message----- From: Nigel Hambly [mailto:nch@roe.ac.uk] Sent: 09 May 2003 12:34 To: WFCAM Science Archive Team -- Clive Davenhall; Eckhard Sutorius; Harvey MacGillivray; Ian Bond; Mike Read; Nigel Hambly; Bob Mann Cc: Andrew Lawrence; Peredur Williams; Jim Emerson; Andy Adamson Subject: WFAU WSA weekly meeting minutes: 9 May 2003 Minutes of WFCAM Science Archive meeting: 09 May 2003 ----------------------------------------------------- ----------------------------------------------------- Present: NCH, ETWS, AL, IAB, MAR, PMW, HMG Apologies: RGM, JMS Actions discharged: ------------------- ACTION: IAB to look at the highest priority use cases, and identify code modules. ACTION: NCH to arrange a get-together for Thurs am with ETWS, MAR & IAB to write down the coding schedule for the next quarter, keeping in mind the existing prioritisation of the work in the review documentation. ACTION: IAB to put a small note on the TWiki about where Doxygen is on the linux file system, and how to use it. - all done and discharged (thanks to all concerned!) Actions partly discharged but continuing: ----------------------------------------- ACTION: IAB to finish analysis of common components (packages/classes/ interfaces) in the WSA by Wed 7th ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. - continuing; thanks for progressing these. Actions carried forward from 02/05/03 meeting: ---------------------------------------------- ACTION: AL to contact SJW concerning UKIDSS Implementation WG input and attendance at the strategy meeting. ACTION: PMW to liaise with AL to make sure that Compute Support are made aware, from the highest "official" level, of our requirements, re. 100 Gbyte/day transfer from CASU, and to impress on them that the 1 Gbit/s FW should be their highest priority. - CONTINUES Specific points and new actions: -------------------------------- Project management: AL reported from the recent (7th May) VDFS monthly management meeting. Agreement has been reached on monthly reporting arrangements, meeting logistics (2 telecons and 1 face-to-face per quarter) and on monthly report contents (progress against tasks lists with % complete and a SOFT-like analysis), meeting minutes (no minutes but a list of decisions and actions). Main action from this month's meeting concerns the VDFS continuation proposal to the recent PPARC e-science AO: need to agree a structure, and then WFAU needs to write the archive section ACTION: AL, PMW & NCH to meet Firday 9th 2pm to agree on structure and content from archive point of view ACTION: AL & PMW to agree with JPE and RGM (Cambs) on overall structure for the VDFS continuation proposal NCH asked about minutes of CASU weekly meetings since it seems CASU are getting the WSA ones. ACTION: NCH to ask JPE to CC CASU weekly meeting minutes to him, and to include on the TWiki with the WFAU ones. Much of the discussion of today's meeting focussed on the Review panel report from the CDR. Decision was taken to draft an "action plan" in response to all points raised, and also to go back to UKIDSS to ask for clarification from the user community on a few relevant points. It was decided that a reissue of all the review documents in the light of the review was not necessary at this stage - these will be altered and reissued at appropriate points over the build/implementation phases. ACTION: PMW to draft and action plan in the light of the review panel report. ACTION: NCH to put the report on the external website, and pull out relevant points from the UKIDSS point of view and ask SJW to distribute URL to UKIDSS for input. The main issues from the review, namely staff resources and software architecture design, will be specifically addressed in the action plan and the Software Architecture Design Doc (see below) respectively. Hardware: NCH reported some progress on catalogue server hardware: benchmarks were now at 140 Mbyte/s and 220 Mbytes/s respectively for a restored database and a specially created test database on the Ultra320 server. The disagreement with the MemSpeed results (400 Mbyte/s) is being looked into under advice from Jim Gray. Andy Knox has offered a new IBM ServeRAID4H adapter to try out on the IBM kit (which is severely underperforming at the moment). NCH asked PMW about the continuing saga of the WFAU RG and hardware money, with specific concerns over the existing RAID array which is close to full. For production of intermediate files to get the SSA data over to the catalogue servers, this is a problem. ACTION: PMW to clarify hardware money situation from old/new grants and the WFAU RG generally with PPARC. Networking: ETWS reported: " Horst and Mike has changed the tcp parameters to a higher buffer size as well as to a bigger window size. Unfortunately nothing changed for the download times i.e. the transfer rate is still 1.8GB/s. Horst suggested that this is due to the CPU, which may be too slow to handle the transfer faster. Therefore I'm now trying to download data the other way round from here to Cambridge's apm3 which is a 2GHz CPU instead of our 800MHz. The installation of GridFTP is initiated as well, so if this is ready to use, I will contact Guy Rixon again to test it." Software: MAR reported: "Worked with Java looking into. Handling all SQL data types and writing to FITS table, Coordinate manipulation using JSky and JNIAST classes. Formatting numbers (Java version on sprintf), Found out how to password protecting servlets and got it to work on grendel12." IAB reported: "A note on my personal installation of doxygen is on the Twiki. It seems like a good approach to designing the WSA software architecture is to take a component oriented view. Software components are defined and analysed in terms of their deployment, inter-dependencies, and interfaces with each other. Headway has been made and a first blueprint of the archive can be drawn up soon." NCH, MAR, IAB and ETWS had meet previous afternoon to discuss progress with the major outstanding software issue: that of overal system architecture. IAB has nearly finished the analysis of the software packages/modules/interfaces and has undertaken to draft a "Software architecture design" document. ACTION: NCH, MAR & ETWS to give feedback to IAB on lists of components and interfaces for inclusion in first draft of SW architecture design doc ACTION: IAB to draft SW architecture design doc by midday Wed 14 May SSA: Only outstanding issue concerns production of cross-neighbour and neighbour where one table has > 10^9 rows. Tests so far show that SQL scripts a la SkyServer are very slow, and there are doubts as to scalability of this particular solution. ACTION: NCH to communicate with Jim Gray and Alex Szalay as to their experience with the SkyServer SDSS DR1 neighbours. Miscellaneous: Nothing this week. ACTION: NCH to type up and circulate these minutes DONM: 10am, Friday 16th May, plate library.