From nch@roe.ac.uk Fri Oct 31 13:17:00 2003 Date: Fri, 31 Oct 2003 12:34:28 +0000 (GMT) From: Nigel Hambly To: Eckhard Sutorius , Harvey MacGillivray , Ian Bond , Mike Read , Nigel Hambly , Bob Mann Cc: Andrew Lawrence , Peredur Williams , Andy Adamson , Jim Emerson , Mike Irwin Subject: WFAU WSA weekly meeting minutes: 31 October 2003 Minutes of WFCAM Science Archive meeting: 31st October 2003 ----------------------------------------------------------- ----------------------------------------------------------- Present: ETWS, RGM, JMS, IAB, PMW, NCH, MAR Apologies: AL, HMG, JPE Actions discharged: ------------------- ACTION: NCH to get HTM2 from JHU. (Gyorgy Fekete emailed; no response yet) (RGM suggests that we need to speak to Ani directly about this when he's over in a few weeks). ACTION: NCH, ETWS to create schema-driven tags for ingest procedure, and to code up proto-curation populating scripts for the WSA for test purposes. (schema tags to drive ingest are being implemented in revised schemas in the CVS. Proto-curation populating scripts have yet to be started) (NCH still hasn't started the curation scripts - see S/W below) (NCH now started; will need a day or so to complete). (completed. Proto-WSA set up on amenhotep and populated with curation data). ACTION: NCH & ETWS to finalise USNOB binary load scripts and to send these and the schema to Alex at JHU. (scripts etc. done but not checked yet; will communicate the procedure to Alex once it has been tested). (sending to Alex this morning). ACTION: NCH & PMW to meet Mon am to revisit the project plan and risk register in the light of recent AGOC advice and the WFCAM slips. (discharged, but production of required tables and GANTTs still being done) ACTION: NCH to speak with JNTD concerning the C1 power trip switches and the WSA server UPS configuration. (discharged; see Hardware below) ACTION: IAB to put a small report on the NeSC Condor seminar on the TWiki. ACTION: PMW to liaise with AL to make sure that Compute Support are made aware, from the highest "official" level, of our requirements, re. 100 Gbyte/day transfer from CASU, and to impress on them that the 1 Gbit/s FW should be their highest priority. (discharged: no longer relevant). (thanks to all concerned). Actions partly discharged but continuing: ----------------------------------------- ACTION: ALL to review risk register and think of any new internal or external risks that should be documented in the risk register. (this one is now important again in the light of recent AGOC meetings and request from JPE to revise the register. See Project Management below). (defered until top level plan is explicitly defined) ACTION: RGM to progress design of a web cookbook based around the 20 Queries. (started) (still continues: 2/3 done) (RGM on leave from 10/10/03) (resumed work this week) ACTION: JPE to design and set up centralised VDFS web pages. (nearly complete; see Project Management below). - continuing; thanks for progressing these. Actions carried forward from 24/10/03 meeting: ---------------------------------------------- ACTION: ETWS to contact Guy Rixon concerning a new GridFTP test. - CONTINUE Specific points and new actions: -------------------------------- Project management: PMW reported he and JPE had attended the VEGA consortium meeting on 28th in Cambridge: "NAW said he'd be putting a record of the Data Processing and Content Delivery meeting in Cambridge on the 28th on the Twiki http://casse.ast.cam.ac.uk/twiki/bin/view/Community/VOProcPlus20031028 so this note covers just a few points from my notes. Gerry G started, saying that having a structure / continuous group would put us in a better position to bid for ESO data processing projects than if each bid started ad hoc from scratch. Neil Geddes made point that the IT people on the GSC recognised commonalities in bids they received (for e-science 2). He was not pushing for collaboration; projects would be funded project by project and the decisions on funding would not be dependent collaboration between projects. He emphasised the last point. He also made it plain that a bid just for a layer of management on top of the existing projects would not be accepted. In response to query from Tony L. as to how the such collaboration would be funded, Neil said that he was looking to the community for proposals. Floor v L. said that there was already an ESA-funded contract at Barcelona looking into Gaia databases. Gerry said that the Germans (I don't think he said which group) were setting up programmes to bid for Gaia work. We then went through the projects on the Agenda (link above); really a varied lot with very different timescales. We then went round the table trying to identify areas of commonality: really, only at the data delivery / archive / Data Centres end. Jim E proposed a regular (?) forum for projects to exchange information, at a level of detail as in a design review. Without this, we wouldn't be able to find any synergy. Nic Walton said something about getting people to build their code in Maven but this was not taken up. Outcome was support for Jim's proposal, with the first meeting in mid-December, i.e. in time for bids to the next GSC bids in January or so." JPE has asked both NCH and PMW for a progress GANTT along the lines of the CASU one for the VDFS web pages. ACTION: PMW to produce a progress GANTT and send to JPE. PMW reminded everyone about the monthly VDFS management group meeting (telecon) on Tues next week, at which the usual materials would be being produced beforehand. NCH reported that the ICD has now been completely finalised with CASU and will be reissued. New FITS header metadata has been agreed, and must now be propagated intot he archive schemas. ACTION: NCH to release ICD Issue 3. ACTION: ETWS to propagate ICD Issue 3 changes into WSA schemas. Hardware: NCH reported that the power trips in C1 are set up as assumed; it may be possible to take the WSA hardware off this tripped supply onto their own and rely on the UPS to protect them, but then there is the possibility that the aircon could be off when the WSA hardware is on. In any case, during normal working hours, the UPS protect the servers for up to 15 minutes, provided infrastructure support manually trip the power back on. NCH suggested someone should talk to Ron about all this, and also check that djoser is now attached to a UPS (it was originally set up straight on the mains). ACTION: PMW to speak to HMG to speak to Ron concerning the power supplies in C1 to the aircon and WSA hardware, and to check that djoser is powered through a UPS. ETWS reminded everyone to keep the software wish lists up-to-date; IAB noted he has put some additional middleware requirements for djoser. NCH asked if we should again ask HME for aome kind of system access; all agreed this would be most advantageous. ACTION: ETWS to again approach HME concerning system access to djoser. Software: RGM suggested ingestion tests should now be tackled using the CASU supplied CIRSI data. NCH suggested an offline meeting to arrange this before next Wed. ACTION: NCH, IAB, ETWS & MAR to meet Tues to plan some ingestion tests using CIRSI data. PMW asked if the WSA schemas are ready to ingest non-survey as well as survey data. NCH reported that image data and metadata are catered for in the schema, but not default catalogue detections. ACTION: ETWS to produce a NonSurveyDetSchema for the WSA to curate non-survey default catalogue data. MAR reported: "Did some work on getting Java to call c-code (that issues cfitsio calls) via JNI. Might be useful for WSA FITS image extraction stuff if we offer it via Java servlets rather than cgi. Got a JDBC connection pool working under a test servlet. This might eventually be needed if connection times start to slow things down. Wrote a test Singleton pattern class that is accessible via a servlet and holds a hashtable. This is a way of storing UCDs for columns names." Networking: NCH has asked ETWS offline to continue pushing forward implementation of CU1 on djoser; ETWS has asked HME to install Python2.2 on djoser for curation purposes. SSA: A full load of the SSA is complete on ahmose; constraint checks are being attached to ensure the integrity of the data. The 10% sized dataset on amenhotep is available for continuing indexing experiments. MAR suggested that it would help in final deployment for the catalgoue servers to be moved onto the WSA private area network. NCH asked MAR to liaise with JNTD about this while NCH is on leave. ACTION: MAR to liaise with JNTD concerning the transfer of the new catalogue servers onto the WSA PAN, and to change test DB connectivity from thoth. NCH also pointed out that once this was done, the intermediate Source files for the SSA could be staged on thoth for CDS to transfer to Strasbourg. ACTION: NCH to stage intermediate Source files in thoth, and email Francois Oschenbien at CDS when the first file(s) is/are ready for transfer. RGM reported that the indexing experiments can now continue in earnest. Miscellaneous: NCH advised the group that he is away for the next three Fridays (2 hols and on UKIDSS in Oxford). NCH asked RGM to chase up the SDSS DR1 "sneaker net" boxes following on from Jim Gray's recent communication, and to make sure we're signed up (if necessary) to receive the data. ACTION: RGM to chase up the SDSS DR1 "sneaker net" boxes and check that Edinburgh is on the grand tour itinerary. ACTION: NCH to type up and circulate these minutes. DONM: 10am, Friday 7th November, plate library (PMW in the chair).