From nch@roe.ac.uk Fri Feb 27 20:23:11 2004 Date: Fri, 27 Feb 2004 11:47:48 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes: 27 February 2004 Minutes of WFCAM Science Archive meeting: 27th February 2004 ------------------------------------------------------------ ------------------------------------------------------------ Present: MAR, AL, JMS, NCH, ETWS, PMW, IAB Apologies: JPE, RGM, GPS, MCH, HMG, JDT DONM: 10am, Friday 5th March 2004, plate library. Actions discharged: ------------------- ACTION: IAB & NCH to meet Monday pm for another integration session on the CU3 and CU4 software. Discharged. ACTION: ETWS to switch off web access to the wiglaf SDSS SX EDR flat file system. Discharged; so far, no-one has complained. Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). - continuing; thanks for progressing these. Actions carried forward from 20/02/04 meeting: ----------------------------------------------- None. Specific points and new actions: -------------------------------- Project management: PMW noted communications with PPARC concerning the WFAU RG renewal. As has been indicated previously, the idea now is to split the RG into 3 different parts: i) E-science (VDFS, AVO etc.) ii) other WFAU services (and including research fractions) iii) Individual researchers. Item i) is now considered peer-reviewed (by GSC/PPRP etc) and money will be transfered into the existing RG via a grant revision; ii) is required to be renewed in April (hence a few weeks of work will be needed to assemble the case for the various WFAU activities that don't fall under i) and iii). Comments and issues arising from CASU fortnightly minutes: Nothing new this week. Networking: NCh noted that after detecting some error events on the MS SQL Servers concerning the DR1 data transfered from the SneakerNet box, it appears that there may be problems with the original data files thereon. Some communications with Cambridge (where the box currently resides) and Alex/Jim have lead to concerns that there is some kind of problem on the box. Anyway, our own copy of the data seems to be producing sensible results (see SSA below) despite the error messages. The SneakerNet goes to CDS after Cambridge, so there will be a chance to get it back if required (although this does not seem to be useful since the problem appears to originate from the box itself). ETWS reported another "real world" transfer test from CASU where 80 Gbytes (ie. approx 1 night's worth of WFCAM data) were transfered up to WFAU in 3 hours. Hardware: Eclipse attempted an onboard controller update (using firmware from Tyan) in the week - this failed miserably. The situation with the SCSI Ultra320 servers seems stable at the moment; NCH suggested that the thing to do now was to consider reorganising the distribution of disks and controllers to rid the servers of all IO subsystem errors. Eclipse will probably need to come in for a day or so to enable some systematic testing of different configurations. Both servers have been stable over the past few weeks, and have not required reboot. Software: IAB reported more progress on the complicated CU3/CU4 ingest codes, including coping with schema evolution and all the complexities of ingesting various data into diverse tables. IAB requested another integration session early next week. ACTION: IAB & NCH to meet Tues pm for an integration session. NCH noted that more progress has been made on CU20 concerning release of published catalogue data products from the UKIDSS surveys via SQL Server "snapshot" replication, which seems to have all the required features to implement periodic releases at arbitrary times. ETWS reported the he has installed pymssql for user scos on the curation client djoser, and written some installation notes on the Twiki. That should also be the place where other installation notes should go in the future. SSA: NCH reported that he had joined the DR1 data to the SSA, and tested the results. Astrometry between SSA and DR1 is consistent at the 0.2 arcsec (RMS) level; "reduced proper motion" diagrams have been produced to sanity check the pairing, making use of SDSS psf profile photometry and SSA proper motions. AL suggested that NCH communicate the results to Astrogrid as an example of the kind of scope of join that should be achievable in a distributed context. ACTION: NCH to email NAW and MCH the results of the SSA-DR1 join. NCH will join USNOB and 2MASS and similarly test in readiness for releasing the terabyte SSA. NCH noted that the SSA table has 10^9 rows; the DR1 main source table has just under 10^8 rows; the joining took 18.5 hours, and a query using the pre-join to produce the RPM data took 22 minutes. Miscellaneous: NCH noted that the airconditioning in C1 had iced up again. Discussion with RP, HMG and PMW indicate that some kind of planned expansion is the only way forward. NCH suggested that until we move out of C1 (ie. when the SuperCOSMOS area is decommissioned) a good estimate of the power dissipation requirements is 0.5 rack-fulls per year. ACTION: PMW to communicate the 0.5 rack/year power dissipation requirement to RP. JMS advised the team that there will be a meeting to discuss the content of documentation deliverables to ESO as part of the requirements for a PDR. This would of course mainly focus on pipeline and calibration plans. Attendance from the VDFS archive team is not vital, but anyone wanting to attend would be welcome. The likely dates for this meeting are either March 8 or March 10.