7th January 2005 Hoyle committee meeting room 10.30 - 12:50 Present: PSB, JRL, MR, RGM, MJI, DWE, STH (at dentist until ~10:55) Apologies: WJS, EGS, RWA Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report on recent VDMT meetings 4. WFCAM commissioning update 5. Status of ESO documents for FDR 6. Status of Data Centre Archives 7. INT WFS products release 8. RAID arrays update 9. APM building news 10. AOB Minutes ------- Marco Riello(MR), who started on Tuesday, was welcomed to the group. 1. Actions from last meeting MJI has made no further progress in comparing the FIRES project reduced data with ours. He said that it was sensible to carry on with this since it was the best data available - ongoing <<<< STH JAC have taken over the task of generating the MSBs for the time being. However, STH will still monitor the MSB status. JRL has not had time to work on the ZPN WCS problem in IRAF due to the commissioning. There was some discussion about giving this task a higher priority since it is still a major bugbear. JRL agreed that after the urgent WFCAM analysis is done he would work on this - ongoing <<<< STH Sandy Leggett has acquired NIR spectrophotometry to help with the photometric calibration. This will be a work package in the future. WJS got hold of a copy of the latest SDT document WJS the outcome of the issue of combining the two guide star data stream is unknown - ongoing <<<< STH has generated modified versions of JRL's diagrams EGS has released the revamped INT WFC web pages. RGM said that there were a few things missing. He will contact EGS regarding this. <<<< MJI had also raised the issue of advertising the INT WFC products database prominently on there to help satisfy requests for distributing processed WFS data. <<<< MJI raised the issue of ESO workstations and software with JMS. PSB has now acquired the necessary information to obtain the software needed to set up an ESO workstation. PSB said that he would start by trying to install this on an existing Linux PC. This will become part of workpackage to help JRL with the CPL+QFITS interface. MJI found out from a VDMT meeting that the UK VISTA URD has been signed off and released. JRL has obtained dome flats by flying out to Hawaii and helping do them himself and bringing them back on tape WJS provided STH,MJI with the latest overhead figures (eg. time to change filter etc.) for inclusion in the ETC STH raised the issue at a VDMT meeting about the level of detail, for example, the pseudo maths code (calculation forms) and other issues, that needed to go into the ETC document. After much iteration with JPE and MJI a version 0.9 ETC document was submitted on the 23rd December. Instead of calculation forms a live ETC cgi script written in Perl, fronted by a web form interface, was produced over the Christmas period. It was felt easier to debug the code and also investigate user interface issues with an active ETC in place. This will lead to further changes in the ETC document. PSB has redone the templates section of the Calibration Plan document STH has reviewed the cascade diagrams PSB did a short-term review of all the documents MJI has proof-read Section 2 of the DRLD PSB added the photometric calibration equations (~1 page) from the Calibration Plan document to the DRLD STH has added the diagrams needed for Section 3 (Functional Description) JRL has written Section 5 (DRL Data Structures) PSB has written a skeleton for Section 7 (Data Reduction CPL Plugins) MJI formulated the maths for the various quality control parameters for inclusion into Section 2. JRL has written Section 11 (Data-Processing Table) MJI raised the issue of Gavin Dalton's inclusion in the FDR process at the VDMT meeting ALL provided reports to STH RGM supplied comments for the grant bid. MJI said that he would make sure NAW that bound copies of the grant bid get produced and distributed <<<< 2. Comments on WFAU minutes It was noted that the hardware problems at WFAU had been resolved. MJI reported that he and STH had been shown the slick black hardware during the last VDMT meeting. PSB noted the report from the 17th December meeting stating that the ATC-Cambridge networking setup was resolved. It was also noted that the VDFS web pages action had been discharged. While on the subject, MJI suggested that DWE show MR around our web site. <<<< 3. Report on recent VDMT meetings There have been 2 VDMT meetings since the last CASU meeting. The previous one was a telecon and superceded by the quarterly face-to-face meeting on 6th January in Edinburgh. MJI reported on this latter while STH was making his way back from the dentist: the ESO FDR will be held on 31st January at Garching. The confirmed attendees will be MJI, PSB, JPE, JMS and Alistair McPherson. It was also suggested that it would be very useful if WJS, Steven Beard and Martin Folger could also attend in person. STH and JRL will attend via telecon; the ETC document was discussed with apparent agreement being reached on the remaining changes required for version 1.0. An updated version would be distributed today and MJI would update the live ETC appropriately over the weekend. This would then be submitted on Monday for FDR along with the ETC perl cgi script; <<<< although the DID was not officially part of the FDR it was regarded as a useful subset and that submitting it as part of the FDR (as agreed at the PDR) would stimulate some feedback. MJI asked PSB to send the existing data dictionary files to JMS for submission to ESO on Monday; <<<< MJI noted that PMW reported at the meeting that he had heard from PPARC that the CASU and WFAU rolling grant reviews may be held around the end of February. This was news to MJI; the VDMT also discussed when the UK VDFS should be held and the level of documentation required for it. JMS suggested that any extra documentation for the additional UK VDFS requirements should be at the functional specification level, ie. without too much technical detail. The submission of the documentation was set for 15th April, to be preceded by a VDUC meeting late February to sign off the URD document requirements. The review would then follow in May in a similar manner to that of the recent ESO VDFS reviews; MJI noted that there was some confusion regarding what the CASU supplied summit pipeline actually does. This has now been sorted out. However, it emphasises the fact that we should not presume too much prior knowledge when making presentations to, for example, VDUC in February; the PPARC WFCAM press release was noted. JPE suggested that we do a similar colour picture before the VDUC meeting. STH was very keen and volunteered to do this. This triggered some discussion about the web <<<< presence of CASU for WFCAM data processing. MJI felt that this would resolve itself when we present the result of the WFCAM commissioning data analysis on our web pages; WFAU asked for a sample (of good) WFCAM data for assessment. MJI said that we will select some suitable data for this when all the data has been read on, translated and MEF'd; STH noted that the quarterly task list has not been finalised. MJI and STH will draw up a list and circulate it for comment. <<<< 4. WFCAM commissioning update JRL reported on his two week trip to Hawaii in November to help with the WFCAM commissioning. Although there was the usual bad weather following JRL around, there were several good nights and lots of relevant data was obtained. Even during periods of bad weather, many calibration data files could be generated. At the moment there are many interesting features in the data that require characterising. This analysis needs to be mainly finished by the end of February before WFCAM goes back on the telescope. JRL suggested that investigations mainly focus on data from the 8th November onwards after which the system was reasonably stable (the ATC commissioning team having departed by then). The data is on 16 LTO tapes containing a total of about 2 Tbytes of data. This is first read onto RAID disks, then converted to single extension FITS using the same ORAC-DR recipe as at the summit (note that this step will not be needed for normal WFCAM operations), finally the individual FITS files are glued together into MEFs. These can be found at /data/apm6_b/wfcam/MEFs/ sorted into directories for each night. This process should be complete toward the end of next week. There followed some further discussion regarding various web pages: STH should contact Paul Hirst (and a suitable person at Edinburgh) to change the CASU link to go to our VDFS web page rather than our group home page. <<<< DWE should add a WFCAM logo and link to the bottom of our VDFS web pages. <<<< MJI needs to update the CASU group web pages entry point <<<< 5. Status of ESO documents for FDR All bar one of the required documents have been submitted. The Data Reduction Library Design (DRLD) document version 1.0 was completed and had the most effort spent on it over the last 3 months. This document was submitted for FDR on the 17th December. The other FDR documents (URD, DRS and CAL) have been updated with all the required responses to PDR RIXs and updated as far as possible to incorporate the changes in template design and other design changes necessitated by the switch from deliverable specification 1618 issue 1.0 to 1618 issue 2.0. The other work package that has taken rather more effort than planned this quarter is the ETC document. Although an interim ETC document version 0.9 was submitted on December 23rd, Version 1.0 of this, required for FDR, is still awaiting final discussion and iteration. Many of the problems with the design of the ETC are related to the overlap of the ETC requirements with the design and requirements of the Survey Definition Tool. In addition a live working ETC based on a perl cgi script was developed, accessible via a web form interface (http://www.ast.cam.ac.uk/vdfs/etc.html). STH,MJI think closure on the design of the ETC and the content of the document was reached at the VDMT meeting held on the 6th January. 6. Status of Data Centre Archives RWA was unable to attend the meeting and submitted the following report: ING archive: We now have all the DVDs from La Palma up to and including early September 2004. There are 1950 for the INT, 450 for the WHT and the 76 from the JKT. The latter stopped coming in 2003. All have been checked. The dumping of the INT CDs awaits the allocation of a suitably large disk space. With 7300 CDs each of 650 Mbyte, a fill factor of 0.9 and a Rice compression factor of say 2.5 this will need ~2 Tbyte of space. The overhead in reading this on (and the cost of extra disk space) will be more than compensated by the much quicker response to requests - jobs that used to take hours to fill will take a few minutes and can be largely automated. AAT archive: Another 112 DVDs have arrived, bringing the AAT archive up to September 2003. All 264 DVDs have been loaded in the DVD tower. There is room in the tower for perhaps another 30 to 40 disks only but that is not a problem give the upcoming retirement of the DVD tower. UKIRT archive: This is up to date with all the shipped DLTs online on DVDs in the DVD tower. 7. INT WFS products release MJI said that he had received an email from Danny Lennon regarding public availability of the processed WFS data, including catalog products. To make these really useful the main task is to cross-calibrate the large areas components. The individual data frames are already available. EGS is investigating this calibration problem and will attempt to resolve the problem in time for the Spring issue of the INT newsletter where the access will be publicised. <<<< EGS was unable to attend the meeting and submitted the following report: A quick update on progress. I have almost finished the calibration of all the WFS frames in the ELAIS regions taking as reference the calibrated released ones. I need ~1 week to check the results, see why some files do not have header keywords (e.g. airmass) and write a small report so you can have a look before applying the corrected zero points and zero errors. 8. RAID arrays update MJI reported on the various RAID arrays that we have available. Recently we have obtained 17 x 300 Gbyte IDE discs which have been distributed to the RAID arrays (with one to add to the spares). This means that there are a number of 80 and 160 Gbyte IDE discs requiring a good home. Official CASU data is spread over various machines and the intention is to rationalise other processed data onto RAID arrays on apm2 and apm3. The details are in the private minutes. STH was volunteered to make an internal web page to keep track of this. <<<< 9. APM building news The last two building liaison meetings were attended by PSB. He said that the timescale for the roof construction had slipped a little, but by working over the Christmas period the main roof assembly has now been completed. He pointed out that the main network cables for the IoA will be placed into proper cable runs rather than in a tangle under the floor of the APM building. The currently projected completion date is now mid-February. STH officially conceded the bet, blaming his youth for his optimistic folly. PSB suggested that the payment (lunch) be bought after completion. 10. AOB MJI reminded everyone to fill in their .away forms to note their expected absences. This is a very useful tool for planning meetings. <<<< MJI said that he gave a presentation of group activities at the November staff meeting and had some interesting feedback. MJI asked about the data on cass38's old discs. JRL said that he hadn't done anything with them yet. He said that RWA uses them as a staging area for FTP access so would like to keep them as is. PSB noted that cass36 and cass38 are getting a bit long in the tooth and will need replacing soon. It was also noted that the DVD tower (on cass36) would be decommissioned over the next year or so or when it goes wrong (whichever comes first) since it is now not on a maintenance contract. PSB also said that Acrobat 7.0 was out and that we get a licence discount on 10 or more. Since the secretaries are getting a batch, we should join in for one or two. PSB will organize this. <<<< RGM said that he had access to a free PDF writer. He said that he would email any interested parties. <<<< Continuing Actions ------------------ MJI continue with comparing FIRES project reduced data by further testing of a CASU seeing-weighted stacking procedure JRL finish resolving ZPN WCS problem in IRAF WJS report on two guides better than one issue New Actions ----------- RGM contact EGS regarding missing items in the revamped INT WFC web pages MJI organise production and distribution of bound copies of the grant bid EGS make sure INT WFC products database access is advertised on wfcsur web pages DWE show MR around our web site STH update and submit ETC document MJI update ETC web form interface and perl cgi script PSB send the data dictionary files to JMS so that he can submit them to ESO by Monday STH generate a pretty picture from the WFCAM data so that we can use it for advertising (by the end of February or sooner) MJI draw up a quarterly job list and circulate it for comments STH STH contact Paul Hirst (and a suitable person at Edinburgh) so that the CASU link goes to our VDFS web page rather than our group home page DWE add a WFCAM logo and link to the bottom of our VDFS web pages MJI update the CASU group web pages entry point STH make an internal web page describing our disk resources ALL fill in .away forms to note expected absences PSB sort out Acrobat 7.0 purchase RGM email interested parties about free PDF writer