From nch@roe.ac.uk Fri Aug 26 12:56:51 2005 Date: Fri, 26 Aug 2005 12:25:38 +0100 (BST) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann , Ross Collins Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , John Taylor , Jim Emerson , Malcolm Stewart , Martin Hill , Mike Irwin , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly project meeting, 26th August 2005 Minutes of WFCAM Science Archive meeting: 26th August 2005 ------------------------------------------------------------- ------------------------------------------------------------- Present: NCH, MAR, RSC, NJC, JDT, RGM Apologies: MCH, PMW, ETWS, JPE, JMS, AL DONM: 10am, Friday 2nd September 2005, plate library Actions discharged: ------------------- None this week. Actions partly discharged but continuing: ----------------------------------------- ACTION: PMW to initiate removal of old hardware from C1 by communicating with all concerned parties (JNTD, HMG, RL, ...) Eclipse will move the equipment next week (see Hardware below). - continues. ACTION: NCH liase with JNTD to sort out Windows accounts, file share access and an increased remote connection limit on W2K3 DB servers. - continues (asked for remote connection limit to be raised). ACTION: MAR & RSC to assemble an advertisement-feature poster concerning the WSA for ADASS. Continues; MAR has dug out the old poster. ACTION: JDT and MAR to continue sorting out AG access option functionality for saving WSA results to "MySpace". Continues; see Astrogrid deployment below. Actions carried forward from 19/08/05 meeting: ---------------------------------------------- ACTION: NCH to investigate the usefulness of a NAS solution for medium term archive mass storage. Continues; NCH noted that he had mentioned this to MJI at CASU in the week. CASU's solution to the mass storage problem is currently identical to WFAU's (i.e. P4 servers with 3Ware RAID controllers employing SATA 400 GB disks, all mounted in 16 to 24-bay chassis). Specific points and new actions: -------------------------------- Project management: Nothing new to report this week WFCAM update: See the stop press pages linked from the TWiki for latest news. Comments and issues arising from CASU fortnightly minutes: No new minutes as of 26/08/2005 AM. Networking: ACD has made contact from NeSC, and will be visiting us on Friday 2nd September to discuss technical details and requirements. NCH & ETWS will attend; anyone else who is interested may do likewise. NCH has suggested a meeting time of 11:30am, after the usual weekly project meeting. Hardware: All hardware still functioning normally. New rack and server have been delivered from Eclipse. Installation awaits removal of the old equipment which is scheduled to take place next week. MAR asked when so that a warning of downtime on the old SSS service can be posted on the appropriate pages ACTION: MAR to communicate with Eclipse about when removal of the Alpha server cosaxp6 is to take place. RGM reported some news on the University's SAN: Sun Microsystems have built a new system that has been running reliably in their Linlithgow labs for a few months now; the schedule for deployment on UoE premises at The Bush are TBC however. Software: With reference to last week's discussions: 1) Fix ingest bug on aperture corrections [ETWS, NCH] 2) Update archive schema (e.g. merge source tables; include QC info and other missing metadata delivered through the pipeline) [NCH, ETWS] Both done. More parameter description in the schema browser [ETWS, NCH] Parsing infrastructure now in place; now needs everyone to contribute more documentation in the appropriate places. "All params" and "Default subset" buttons for attribute selection in the menu-driven catalogue search [MAR, ETWS] Continues 3) Interface upload facility for cross-ids & thumbnails [MAR] MAR has been working on this 4) Post-ingest QC [awaits implementation of 1)] Continues 5) Check merge classification consistency: e.g. p(class) and class assignment in merge tables (e.g. p(Star)=0.9999 and class != -1 ?!); Make multi-epoch frame choice more intelligent when forming frame sets; DXS & UDS cataloguing & current "best" stacks and mosaics [NJC] NJC has checked for internal consistency and everything appears to be OK. There seems to be some confusion amongst users as to a reasonable expectation on classification accuracy, and a complete lack of understanding about the trade-off between completeness and reliability (esp. when using the discrete classification codes). This obviously requires some carefully thought-out documentation and also some database Views which illustrate (via SELECT statements) how to optimise star/galaxy sample selections via the available classification attributes. ACTION: NCH to add documentation & views on image classification and associated sample selection. ETWS additionally reported: "I've expanded the dxsDetection schema a bit and included Mike's (Irwin) extended text. You can have a look at the browser at this URL: file:///djoser/etws/sandbox/WSA/src/parsing/publichtml/www/wsa_browser.html and go to the dxsDetection table" and "I've included the aperture correction into the magnitude calculation in CU4. And I've started to include more attribute information into the schemas. The parser will now produce pop-up tooltips and includes text from a given extended glossary file into the glossary pages." RSC has been working on the source association algorithm via the C++ plane-sweep implementation; NJC continues to examine frame set definition and source merging for calibration data. Data Release: Pending; awaiting retransfer & reingest of SV1 & 2 data. SSA: Nothing new this week Astrogrid deployment: JDT reported, following last week's notes about the required functionality on saving WSA query results to AG virtual storage MySpace: "Further to this - I've tried the javascript solution to this problem, and found it to be a no goer due to browser security restrictions. The browser prevents you connecting to a website which is not the originating site. Since the page will be served from the roe site, but the ACR is running on localhost, we're stuffed. So, it's back to plan A, which was the javaweb start solution. Slightly more clunky, but should work. I'll let you know when I've got a prototype done." JDT also reported: "The installer for the astrogrid-portal has now been modified to make it Tomcat 4.1-compatible. Previously it only worked with Tomcat 5+ but unfortunately Tomcat 4.1 is the default version shipped with Debian (Sarge). The modification should be straightforward to apply to the installers for the other astrogrid components and I'll try to get it done next week. I've tested the portal on Tomcat 4.1 with the Security Manager running, and determined the required security settings. Once I've tested the configuration on a Linux machine (today), I'll hand the policy file over to Horst and we should be able to get an ROE AstroGrid portal up and running next week." NCH noted that the deployment hardware will be mounted in the new archive rack once the old gear is shifted from C1. RGM noted that, looking ahead to next year's AG demos at NAM'06, it would be great to show WFCAM data being accessed through the portal. Pending solutions to security issues, RGM suggested that we look again at SIAP services on existing WFAU image data holdings with a view to gaining the necessary experience in setting up such things. ACTION: RGM to action MCH to look at SIAP services for WFAU image data Miscellaneous: Nothing else this week.