From nch@roe.ac.uk Fri Jan 21 14:35:10 2005 Date: Fri, 21 Jan 2005 12:20:14 +0000 (GMT) From: Nigel Hambly To: WFCAM Science Archive Team -- Eckhard Sutorius , Mike Read , Nigel Hambly , Nicholas Cross , Bob Mann Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence , Andy Adamson , Peter Shillan , John Taylor , Jim Emerson , Malcolm Stewart , Martin Hill , Mike Irwin , Peredur Williams , Stephen Warren Subject: WFAU WSA weekly meeting minutes, 21st January 2005 Minutes of WFCAM Science Archive meeting: 21st January 2005 ------------------------------------------------------------- ------------------------------------------------------------- Present: NCH, PMW, ETWS, JMS, NJC, RGM, AL Apologies: MCH, GPS, JPE, MAR, JDT DONM: 10am, Friday 28th January 2005, plate library Actions discharged: ------------------- ACTION: ALL to think about the possibility of NAM'05 presence (e.g. talk and/or demo). Discharged; it was generally agreed that it is a little too early to start demo-ing the WSA; NJC will look in to the science programme and may attend. Actions partly discharged but continuing: ----------------------------------------- ACTION: MAR to check out SQL Server networked server functionality for cross-server querying. Continues; NCH suggested we should do some experiments with the linked SQL Servers ahmose & amenhotep once the dust has settled on their reorganisation over the next couple of weeks. ACTION: ETWS to put (or link) ACD's network document on the WSA TWiki. Continues: awaiting the finished article from ACD. - continuing; thanks for progressing these. Actions carried forward from 14/01/04 meeting: ---------------------------------------------- ACTION: NCH to remind MAR to send JPE the VISTA ETC code that is deployed on the VSC pages. Continues: MAR still on leave. - CONTINUES. Specific points and new actions: -------------------------------- Project management: NCH noted that a start date has been agreed with the new software developer (15th March). The team had a major discussion on the various ideas being circulated on the UKIDSS (and more generally, WFCAM) data access policy. After some initial ideas about centralised, periodical registration at JAC, it was thought better to have registration available all the time, with simple criteria being applied to individuals who apply to be registered. However, AL suggested that a more generic approach should be taken that is more integrable into VO infrastructure as follows: To be ready for the VO, and to make sure we don't all go mad, we should not have a vast list of vetted individual users; we should devolve to departments or other "communities". (a) JAC/CSS approve a list of named contacts at a list of communities. Most communities are university departments or observatories, but for example, the Japanese could organise a special one called "UKIDSS-Japan", and the CSS could maintain a list called "UKIDSS-Associates". (b) Each community-contact is trusted to produce a list of users. S/he constructs a simple database in an agreed format (name, username, password, anything else ?). They provide this list to WSA and JAC. Note that this still employs individual passwords, not group passwords, to be specified and supplied by the community contacts. (c) Nobody except the community-contact is expect to do any vetting or policing. (d) WSA build this into the web-login any way they see fit (for now). (e) The contacts can issue a revised list lets say once a month. First lists can come ANYTIME NOW. (This is basically a low-tech version of the AstroGrid scheme (which pretty soon will be an agreed international standard for the VO). When you log in to the AstroGrid portal you use a chosen community name - "leicester.astro.org" or whatever. A local "community server" maintains a list which can change dynamically. Thereafter industry standard X509 certificates bounce around with the information any service (like WSA) will need to decide whether it accepts the request. (Because of prop. period etc, this can include your own "distinguished name" as well as the community name). This is available now on the AstroGrid portal but is still a dummy because the community servers don't exist). This was generally agreed to be a good approach; as far as (b) & (e) above are concerned, WFAU may be able to provide a web-form front end and database that gather the information provided by the named community contacts to enable updates at any time (rather than monthly). This will be discussed with MAR on his return to ensure practicability before we finally agree to doing the work to implement the above scheme. AL also raised the point about access to UKIDSS commissioning data, and the team agreed that provided JAC & CASU agree, and provided appropriate health warnings, there is no reason why commissioning data should not be available to be browsed and downloaded from the WSA; this was felt to be a better and more controllable scenario than ad hoc distribution of the data from arbitrary points in the data flow system. In any case, NCH suggested we should plan to demonstrate this at the next VDUC. WFCAM update: No news this week Comments and issues arising from CASU fortnightly minutes: No new minutes as of 21/01/05. Networking: Good news: CASU made available 38.6 GB of phase 1 commissioning data for transfer and other tests. The data where copied up to WFAU in 52min with a transfer speed of 12.6 Mbyte/s. md5 checksum verification at each end (guards against RAID errors as well as transfer corruption) took 60min. There are some issues with the file format (understandably, since these data really are for shaking down the system). At present, the data do not conform to the agreed ICDs between JAC/CASU and CASU/WFAU, being a) not in FITS format in the first instance, and b) having missing header values. The FITS conversions initially supplied to WFAU had blank header lines and missing "="s after keywords, and produced errors under NOFS fverify. MJI has assured us that these are really just teething problems, and will be fixed (in the pipeline if necessary and where possible) before WFAU see the data. However, some robustifying of the WSA ingest procedure against missing keyword values is probably worthwhile at this stage. ETWS and NCH are looking into this. Hardware: A number of issues arose this week: Mass-storage file server (djoser): one RAID5 set degraded and needed a replacement disk; this has been done and the system is rebuilding the set. Also, the system disk failed on Wed evening. Readable partitions have been backed up to disk and a replacement is expected this pm; NCH suggested we instigate a backup policy of the system partitions of this machine to guard against the single-point failure of the single system disk. NCH suggested that one solution might be to mirror the system partitions on one of the RAID volumes, in order to facilitate quick recovery in the event of system disk failure. ACTION: ETWS to get Horst's advice on this and to suggest a system disk mirroring policy on djoser to guard against future downtime. NCH reported that the W2K3 catalogue servers will be protected against similar problems by having their system partitions actually on a fault tolerant RAID set (not known whether this is possible/sensible under Debian Linux). A policy for system partition backup onto removable media (i.e. LTO-2 tape) should also be instigated on the catalogue servers. ACTION: NCH to instigate system disk backup policy on the W2K3 servers. Catalogue load server (ahmose): reinstallation of the system partition has not been possible owing to a RAID fileset problem that has been traced to a faulty disk backplane in the server chassis. NCH is liaising with Eclipse and JNTD to sort out these problems. Software: Some progress this week on CUs 5,13,14 (ETWS); 7 & 9 (NCH), testing of CUs 1-4 (ETWS & NCH) and implementation of range sanity checking, initially in CU3 (NJC). NJC also reported getting up to speed on the ingest codes generally. SSA: No news this week. Astrogrid deployment: NCH reported that it is not known whether the AG PAL interface access to the SSA has been re-established. Miscellaneous: NCH finally suggested early doors at 6pm this evening in the Waiting Room.