From nch@roe.ac.uk Fri Jan 27 13:55:38 2006 Date: Fri, 27 Jan 2006 13:51:23 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Johann Bryant , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , John Taylor , Jim Emerson , Malcolm Stewart , Martin Hill , Mike Irwin , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly project meeting minutes, 27th January 2006 Minutes of WFCAM Science Archive meeting: 27th January 2006 ------------------------------------------------------------- ------------------------------------------------------------- Present: NCH, ETWS, NJC, RSC, PMW, JB, RGM, JDT, MAR Apologies: JPE, JMS, AL DONM: 10am, Friday 3rd February 2006, plate library Actions discharged: ------------------- ACTION: JB to investigate the current provision of "helpdesk" facilities in the WSA, and look into off-the-shelf options for a more fancy system for future reference. Discharged; see WSA Operations below. Actions partly discharged but continuing: ----------------------------------------- ACTION: JDT to set up a Datascope Launcher for the WSA on the AG workbench pages. Continues. Actions carried forward from 20/01/06 meeting: ---------------------------------------------- ACTION: NCH to investigate the usefulness of a NAS solution for medium term archive mass storage. - CONTINUES Specific points and new actions: -------------------------------- Project management: Nothing to report this week. WFCAM update: See the stop press pages linked from the TWiki for latest news. Comments and issues arising from CASU fortnightly minutes: No new minutes as of 27/1/06 AM. Networking: ETWS reported that U.Chicago have gone quiet over SDSS-DR3 again; he will send an email nudge. ACTION: ETWS to nudge U.Chicago over SDSS-DR3 again. NCH noted that he had received an email from RGM at CASU who is worried about the back-log in WFCAM data transfers from CASU to WFAU. An agenda item at the next VDMT to discuss this would be a good idea; NCH will reply in detail once he gets a moment... NCH noted that there is a one-day meeting at NeSC concerning opportunities afforded by the UKLight fast networking project. WFAU attendance at this will be a very good thing; NCH suggested ETWS and JB to attend. WSA Operations: JB reported: "Worked with ETWS to investigate files not ingested by CU and discovered that the ones in 05B data were due to missing keywords that turned out to be non essential. CU16 has now been completed for the LAS, GCS and GPS. Disk system backups have been made as usual except that they are now stored on Amenhotep rather than Ahmose. Removed and compressed log files on /disk01 due to a lack of space there. Suggested other files that could be removed to free more space there. Investigated our use of wsa-support (none as yet) and the current settings for the list (plus noting what it is possible to do with the system). Discussed some issues with SQL query speeds on Amenhotep. Changed some of the DXS stack centers(?) in the RequiredStack table in the WSA." On the subject of the "helpdesk" (wsa-support), MAR noted that this email address now included on every archive web page. The idea is that JB, MAR, ETWS and finally NCH will (in that order) service each helpdesk request as it comes in, passing on to the next person if they don't know the answer (NCH expressed the vain hope that he would never have to respond...). All WFAU subscribers will receive all email, to cover staff absences; the helpee will be asked to send follow-up messages to the support address or any individual response will be forwarded by the helper to the list so that a full set of correspondence goes through the system. Finally, JB will enquire with ITSG as to provision of archiving of the help requests and responses for internal browsing in the first instance ACTION: JB to ask ITSG (HME) about provision of archiving of the messages coming through wsa-support. Use of this system will be monitored over the next few months; if it is deemed appropriate, some kind of user-browsable system (e.g. a web-based "knowledgebase") will be implemented, using the archive of Qs & As as a back-end database. Hardware: PMW reported that the order for the new filestore server "djedefre" has been placed now that ITSG have OKed the spec. MAR asked about the teams' opinions on acquiring a third catalogue server for large ancilliary DBs (e.g. the SSA) and a batch query server (could also be used as a fail-over server for the existing public or load servers). All agreed this was a good idea and NCH suggested that MAR start thinking about the detailed specification for this new machine. As far as the server name is concerned, the next 18th dynasty chap would be djehutymes but being a little unpronouncable, how about the Greek name tuthmosis... It seems that there may be a significant power outage sometime around the end of February. This is to install more power to the server room. NCH noted that we must keep an eye on this, and inform the WSA user communities in good time of any interuption to normal service (rather unfortunate that this may happen within a few weeks of EDR release, but it's probably unavoidable...) Software: The team has mainly been dealing with the deluge of small fixes and enhancements required by the impending data release. NJC has been working flat-out on the stack surveys (CU13 and 14 procedures) along with ETWS helping out on ingest issues; NCH has fixed up several minor bugs in the ingest DB and has closed out (hopefully) the shallow survey quality control. MAR has done the usual raft of enhancements and fixes on the interface codes. RSC additionally reported: "In between urgent database fettling, I've taken the opportunity to develop the database update scripts to make them more generic and useful for the future. Also have updated our CFITSIO installation to support larger than 2 Gb files, which will need to be tested." Survey Data Release: Things continue to progress well with UKIDSS Early Data Release preparations. With reference to the schedule sketched out previously: 05B quality control (with input from UDS and DXS folks) and the final couple of days worth of JPEGS have been completed. Source merging (CU7) completed last weekend for the shallow surveys. The slight bottleneck over the seaming of overlap sources was cured by judicious choice of a covering index for the source tables to greatly speed up the DB interrogations. DXS/UDS stacks (CU13/14): This area has progressed with great rapidity thanks to the efforts of NJC. Stacks and catalogues for the DXS have been done and will be ingested this afternoon; then NCH will complete the source merging for the same this week end. The UDS images have been received from the Nottingham folks, but there are the inevitable header issues to sort out before these can go through the standard cataloguing code and ingest. Release database preparation is anticipated to start on Monday 30th. Non-Survey Data Release: No news this week. Astrogrid deployment: Nothing new this week. Miscellaneous: Nothing else to report this week.