From nch@roe.ac.uk Wed Jan 7 13:47:53 2004 Date: Wed, 7 Jan 2004 13:41:42 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes 7 January 2004 Minutes of WFCAM Science Archive meeting: 7th January 2004 ------------------------------------------------------------ ------------------------------------------------------------ Present: NCH, MAR, IAB, PMW Apologies: JPE, ETWS, HTMG, JMS, AL, MCH, GPS, RGM, JDT DONM: 10am, Wed 16th January 2004, plate library. Actions discharged: ------------------- ACTION: NCH to stage intermediate Source files in thoth, and email Francois Oschenbien at CDS when the first file(s) is/are ready for transfer. Done: see SSA below. ACTION: NCH to sort out write access to the linux/Windows file shares between the WSA servers. Done: see software Actions partly discharged but continuing: ----------------------------------------- ACTION: IAB to check that all necessary SW for curation client functionality is present on djoser. Continues as part of overall integration/testing of ingest procedure (see software below). ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: RGM to progress design of a web cookbook based around the 20 Queries. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). ACTION: ETWS (in consultation with NCH & HMG) to look into the djoser SATA RAID errors. - continuing; thanks for progressing these. Actions carried forward from 11/12/03 meeting: ----------------------------------------------- None. Specific points and new actions: -------------------------------- Project management: The Project Peer Review Panel meet on Wed 4th Feb at the IoA, Cambs to review the VDFS and GAIA parts of the VOProcPlus (?; formally VEGA) proposal. JPE has asked NCH to attend in order to make a "major presentation" on archive progress and the forward plan. ACTION: NCH to attend PPRP meeting on 4/2/04 to make ANOTHER presentation as to science archive progress and forward plan. Papers for the meeting are required to be at PPARC by the end of next week; NCH & PMW are sending materials to JPE who is co-ordinating assembling them into a case for continued funding of VDFS. ACTION: NCH & PMW to send materials to JPE for inclusion in the PPRP review papers. Comments and issues arising from CASU fortnightly minutes: CASU fortnightly meeting minutes for 16/12/03 were noted; NCH has linked the minutes from the TWiki report on the Astrowise visit since it contains MJI's take on the meeting. The team noted the Garching transfer speed results; see SSA below for WFAU/CDS(Strasbourg) transfer speeds. Networking: ETWS reported (just before Xmas) breaking the 10 Mbyte/s barrier for transfer speed between CASU and WFAU; a full TWiki note on the latest tests is at http://apache.roe.ac.uk/twiki/bin/view/WFAU/WSANetworkDjoser#WeekFiOn CU1 is now being coded up using multi-threaded scp. Hardware: NCH reiterated the importance of verifying that the djoser SATA RAID5 arrays are functioning with full fault-tolerance (see continuing action above). NCH also noted that Windows server ahmose (SCSI SW RAID) continues to run smoothly (eg. heavy IO jobs over the Xmas period ran fine) whereas the old problem of vanishing SW stripes seems to have resurfaced with amenhotep - JNTD is investigating the cause of this. Software: NCH pointed out that HME is moving the site CVS server in order to facilitate external access; for the next few days at least, the CVS server will be unavailable. The only point of note is that HME suggests that the web front-end ("ViewCVS") will not be immediately available. NCH & IAB both emphasised the usefulness of this facility, and when the dust has settled over the movement of the server it may be a good idea to strongly urge HME to reinstall the browser capability of ViewCVS. NCH asked IAB and ETWS (in absentia) to meet to organise a final "extreme programming" push to finally integrate/test the ingestion software. ACTION: NCH, IAB & ETWS to meet 3pm Wed afternoon to organise a block of time to finally integrate/test the ingestion software SSA: NCH reported that, barring indices and cross-neighbours, the SSA is now fully loaded on ahmose. A backup is in progress: 1.6 Tbytes takes 17 hours and 8 LTO2 tapes; a verification is in progress. The Veritas Backup Exec software is performing well and easy to use; an add-on piece of SW is needed (Veritas Agent for MS SQL Server), plus license, to make backups from the database (at the moment, flat file disk backups are made from SQLS and then these flat files backed up to the tape library). ACTION: PMW & HMG to order Veritas Backup Exec Agent for MS SQL Server from Eclipse. For the purposes of indices, NCH & RGM propose to make a basic set of indices on RA, Dec, and magnitude for the two big tables; more complicated indexing scenarios will be explored in the light of usage made of the SSA. The final issue is the one of which of the WSA external catalogues we should attach in to the SSA in the form of cross-neighbours; some discussion ensued and then it was decided to simply do the SDSS-EDR at this stage in order to prevent any further delay in attaching the full terabyte SSA to the web interface (ie. no 2MASS, USNOB, ROSAT, FIRST or IRAS at this stage although the data and scripts for these are all ready). NCH reported that the main science data of the SSA (the merged Source catalogue) has been successfully transfered to CDS. Francois Oschenbein reported a transfer rate of 2 Mbyte/s (after sorting out a major firewall bottleneck at their end). The total volume transfered was 260 Gbyte. Miscellaneous: Nothing else this week.