From nch@roe.ac.uk Fri Jun 4 16:28:14 2004 Date: Fri, 4 Jun 2004 12:19:56 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Clive Davenhall , Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Martin Hill , Mike Irwin , Peredur Williams Subject: WFAU WSA weekly meeting minutes, 4th June 2004 Minutes of WFCAM Science Archive meeting: 4th June 2004 ------------------------------------------------------------ ------------------------------------------------------------ Present: NCH, IAB, JDT, MAR, PMW, AL, ETWS Apologies: MCH, JPE, HMG, GPS, JMS, RGM DONM: 10am, Friday 11th June 2004, plate library. Actions discharged: ------------------- ACTION: MAR to look at Java code documenting/summarising facilities and to implement useage of the same in science archive java application code within the CVS. Discharged: JavaDoc is being used; this produces HTML documentation by default and can be configured to produce PDFs. Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. Defered until top level plan is explicitly defined. PMW tabled the VISTA risk register as an example. ACTION: JPE to design and set up centralised VDFS web pages. Progressing. (there is now a link to it from our WSA Twiki). - continuing; thanks for progressing these. Actions carried forward from 28/05/04 meeting: ----------------------------------------------- None this week. Specific points and new actions: -------------------------------- Project management: NCH noted that the VDMT meeting for review of progress in May was now scheduled for June 8th (huzzah!). NCH requested that the subject of the VDFS PDR documentation release for the end of August be discussed so that the required documentation set is agreed in advance of creating the plan of work for Q3. This is particularly important in the light of the necessary revision of short-term plans to cope with the loss of IAB. PMW reiterated the importance of having a good VDFS user requirements document for analysis as part of this PDR. PMW informed the group that he will be coming around to get updates on progress for the VDMT monthly progress report; he also suggested that he and NCH meet on Monday to go over the reports and review any other issues relevant in the immediate term while NCH is away on holiday and then in Glasgow at the SPIE meeting. ACTION: NCH & PMW to meet on Monday 7th June pm to review VDMT papers and other issues for the coming 3 weeks. PMW requested any thoughts/ideas the team may have concerning the recruitment process that now needs to make two appointments to the team as soon as possible. Some disucssion ensued as to the correct split in skills required for the two bods, and how to pitch the advertisment to attract the best candidates. NCH suggested that this be talked about some more on Monday at the "monthly" WFAU gang-of-five meeting and also in the pm get-together. Comments and issues arising from CASU fortnightly minutes: No new minutes as of today. Networking: Nothing new this week. Hardware: NCH reported the good news that Eclipse have returned the U320 RAID catalogue server "amenhotep"; and also the bad news that it doesn't work. This is most frustrating and mysterious, since Eclipse claim that it has been working fine in their labs for several weeks. JNTD is currently investigating the situation; early indications are that all the disks are fine and that there is some problem in the U320 cabling, JBOD wiring, server chassis backplanes and/or RAID card that are causing the system to malfunction. The team discussed a backup plan for the scenario that this drags on and remains unsolved close to first data ingest; the idea will be to resort to the old U160 speed SW striping configuration (which is working fine on the deployed catalogue server ahmose). This would require reinstating the old disk controllers and cabling, so a decision needs to be made as to when to admit defeat on U320 for the time being and restore the server amenhotep to it's original configuration in such a way as to be ready for first data ingest. The situation will be reviewed when NCH returns from hols; note that it is also important to look ahead to expanding the storage capacity of both load server and public server (ie. space must be left for expansion). NCH asked that somebody deputise for him while he is away for the next three weeks to liaise with JNTD and make sure that Eclipse work continuously for a solution. PMW kindly volunteered. Software: IAB reported: "Finished off the methods for calculating astrometric quantities from WCS information in the CSV generation software. Also included the new table FrameTable in the CSV builder factory functions. Fixed a small bug in the MEF file to JPEG conversion tools." IAB also informed the team that he and ETWS had spent some time meeting to go over existing code and documentation, after NCH had suggested that ETWS should "shadow" IAB while he is still here to ensure some staff continuity and understanding of the code and methods after IAB has left. IAB and ETWS assured NCH that the documentation is well in hand with Doxygen and Epydoc. MAR reported: "Worked further on FITS WCS classes. Hacked downloaded FITS to jpeg code." NCH notes (after the meeting) that these "reports" are perhaps a little too brief to justify a week's work... how's this: NCH reported that more core coding has been done on CU7 (source merging). Testing is being undertaken with simulated frame metadata. This is all proceeding smoothly; new revisions of the relevant code and utilities are being checked into the CVS. NCH also reported coming across several problems with the Python DB interface layer currently being used (PyMSSQL, a dodgy little piece of simplistic third-party middleware). The problem of exceptions being thrown when executing queries that return empty results sets (!) has been corrected by hacking the Python source (hacked DB interface is now called MyPyMSSQL.py and has been checked into the CVS under wsatools/DbConnect) but a much worse problem of SQL Server datatypes not being supported (eg. long integer and datetime types) has arisen. ETWS has made preliminary checks of the lower layers (drivers/interfaces) and there is no problem there; NCH has checked that mxODBC working locally under Windows supports these data types. It seems that the long term solution is to replace PyMSSQL with the much more professional mxODBC, but this will require some more effort to get it working cross-platform. NCH suggests that this work should wait until after IAB has left since there is enough to do during June already. ACTION: NCH to send ETWS details of the PyMSSQL problems before leaving for cuba-libres y cervezas in the Balearics. NCH & IAB had a final design meeting for the source merging code; the team are all loaded up with work to do for the next 3 weeks... SSA: Nothing new this week; quick usage update: SSA has 516 logged queries, PSSA has 377. Astrogrid deployment: JDT has reported elsewhere (see VO Team meeting minutes) that Iteration 5 is complete and ready; AL (same minutes) noted that the deployment WG have agreed to use this as the basis for their deployment and fix up as neceassary. Miscellaneous: PMW will be in the chair for the next 3 meetings (NCH on holiday and then at SPIE in Glasgow).