From nch@roe.ac.uk Fri Jul 16 12:18:15 2004 Date: Fri, 16 Jul 2004 12:05:29 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes: 16 July 2004 Minutes of WFCAM Science Archive meeting: 16th July 2004 ------------------------------------------------------------ ------------------------------------------------------------ Present: NCH, MAR, PMW, ETWS, MCH, JDT Apologies: JPE, HMG, GPS, JMS, AL, RGM DONM: 10am, Friday 30th July, plate library (no meeting on the 23rd as NCH & RGM are away). Actions discharged: ------------------- ACTION: ETWS to set up CVS logins for MCH and JDT Discharged; not tested by JDT or MCH yet though. ACTION: MAR to liaise with HME over the possibilities of expanding the disk storage available on the http server. Discharged; see Hardware below. Actions partly discharged but continuing: ----------------------------------------- ACTION: ETWS to add in required ingest functionality for MAR's changes and check to see if the ingest code can cope with this schema evolution. ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). - continuing; thanks for progressing these. Actions carried forward from 9/07/04 meeting: ----------------------------------------------- None. Specific points and new actions: -------------------------------- Project management: NCH & PMW reported from attending the monthly VDMT meeting (this time, quarterly face-to-face) at Edinburgh (also attending: JPE, MJI, STH, AL, JMS). Main issues of relevance (so not including interminable discussions on performance metrics) are that PPARC are pushing for an oversight committee to ensure value for money from e-Sci2 allocations, and the VDFS v3 Science Archive (ie. VISTA Science Archive) PDR. Incidentally, progress in Q2 was reviewed, and found to be OK. VDMT's attitude to *another* oversight committee is that perhaps the best idea (apart from not having one at all) is to ask one or two members of VDUC to sit in on quarterly VDMT meetings. Some discussion as to the timing, scope and documentation set for the PDR resulted in general agreement that for the science archive, a reissue of updated (where necessary) docs that already exist, plus a new "VISTA Scaling Document" should do the trick, where the latter is to describe how the existing design will be scaled and modified in the light of experience and the requirements expressed in the UKVURD for VISTA. The UKVURD Issue #1 should be with us mid-August for these purposes; PDR document release deadline is now revised to Dec 15th in the light of delays in issue of UKVURD (and also delay in WFCAM delivery generally altering all schedules). The required effort to re-issue existing (and create new) document(s) will be included in the Q4 plan to be drawn up in September. Comments and issues arising from CASU fortnightly minutes: The team noted the CASU minutes of 6/7/2004; there were no comments or matters arising. Networking: NCH noted that he has been in touch with Jan Vandenberg at JHU and secured a copy of SDSS-DR2 pre-SneakerNet2. The idea, given the success of the transatlantic USNOB transfer, is to get a set of 40x20GB SQLServer backup files of "BestDR2", which constitutes 800 GB. The transfer will employ the same multi-threaded techniques that sustain > 1 Mbyte/s across the Atlantic (and > 10 Mbyte/s from CASU). The complete transfer should take of order one week onto the bulk storage server djoser. ACTION: ETWS to instigate copying of the DR2 backup files as staged by Jan on the JHU web server. ETWS reported attending an extremely interesting network meeting at UC London. Main interesting point is the assertion that the JANET backbone is underused at present, and the statement has been made that it should be possible to get transfer rates of 200 to 500 Mbit/s (ie. up to 60 Mbyte/s) given suitable end stations (which are where the bottlenecks are). This is very good news for VISTA dataflow. ACTION: ETWS to put a TWiki note up on new information concerning network bandwidth optimisation. Hardware: MAR noted that the web server thoth is capable of expansion with up to 3 more disks. ACTION: PMW to order one more SCSI disk for thoth from Eclipse (noting the caddy/connector requirements). Ian Davidson (Eclipse) has been on hols this week, so there is no more news concerning the rewiring of amenhotep's JBOD with backplanes and minimal cabling. Software: MAR reported: "Managed to install the image extraction classes/servlet on Thoth and get it to run under the security manager (set up test DB on ahmose, testwsa). Looked into creating the jpegs of extractions in Java directly but seems very slow." NCH noted that he has realised there are a few small problems in the current detection/source relational model employed in the WSA (and SSA). ACTION: NCH to send around a note to MAR, RGM & ETWS concerning potential gotchas in the detection/source relational model employed in WSA. ETWS has been attending the UCL Networking meeting; NCH has been continuing source merging software, in between attending meetings, preparing for a research conference next week, and being off sick for 2 days with a bad back (in addition to his bad attitude). SSA: No news (is still good news) this week Astrogrid deployment: MCH noted that the AG deployment hardware is now ready on the WSA network, and anticipates deployment of PAL etc. by the end of today. The final job will be to get the SSA metadata into the registry to enable client-end tools to query the SSA via web services. Miscellaneous: Nothing else this week.