10 February 2004 Observatory meeting room 10.00 - 12.30 Present: MJI, JRL, PSB, DWE, STH Apologies: WJS, RGM Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report on JAC telecon of 28th January 4. Report from PPRP meeting and VDMT of the 4th February 5. Status of ESO PDR deliverables 6. Data transfers to WFAU and Garching 7. Status of ISAAC reductions 8. AOB Minutes ------- 1. Actions from last meeting STH met with Paul Hewett at the end of last week to discuss various photometric calibration issues that concern WFCAM and VISTA. The work has been divided up between them and should be done by the end of next week - ongoing <<<< STH checking the spatial systematics in the UFTI data is an ongoing task. No new UFTI data is foreseen, so this should be completed by the next meeting. <<<< STH added that he has been in touch with a Caltech bod regarding CSO Tau data. The idea is to see what (if any) correlation exists between this and extinction. This is all part of the photometry work package. JRL has sent some dummy FITS headers to Benoit and Adam. This probably needs about another iteration. The clarifications needed could not be obtained from the SDF document. In addition they have asked for some keywords to be converted eg. RA and dec to be in decimals - ongoing <<<< MJI has been in touch with Andy Adamson and he will send us some dummy data files. JRL have worked on the advantages/disadvantages of CPL/QFITS (see item 5) PSB MJI asked AA at the telecon for any engineering rumours. According to their monthly reports some coplanarity tests have been carried out and they were about to cool the device down. A further email from Mark Casali said that they had cooled 2 science grade chips down with no bangs! Next is to put in the remaining two chips, get them coplanar and start various tests. MJI has been in touch with MC and confirmed that the data from the lab tests don't need to have the full header packets on them. Apparently the data will be sent to Cambridge via Ultrium tape and not via the Internet. STH finished the various GANTT charts and other documents needed for the PPRP and VDMT meetings. JRL produced copies of his UFTI reports for the PPRP meeting JRL JRL passed on the details of the missing ISAAC frames. It was found that STH although the files were present, they only contained the headers. STH is still in the process of tracking these down - ongoing <<<< MJI has carried out the stacking of the ISAAC data (see item 7) STH has acquired the FIRES reduced data to compare with ours. MJI said that the data looks smoothed (correlated noise) and might be difficult to compare. STH calibrating the FIRES data is an ongoing task <<<< JRL has written an outline of a report on the ISAAC reductions. This will be iterated with MJI - ongoing <<<< JRL has carried out tests on swapping sky subtraction and flat-fielding around for the ISAAC data. The result was very little difference - a 0.3 to 0.5% change in sky noise. The reason for doing this concerns faint images. If you carry out SS followed by FF you get dips where the faint images would be. Since there is very little difference caused by changing the order the recommended reduction order for this data is FF first followed by SS. MJI WJS has been invited to these meetings (but is in Edinburgh this week). ALL sent in end-of-month reports to STH STH emailed our VDUC suggestions to JPE. Tim Naylor is to be the chairman, but the rest of the committee hasn't been finalized (still waiting for some nominations to come in). It was thought unlikely that a meeting would be held before the end of March. One of the main things for VDUC to do is consider the User Requirements Document to see if there are any problems from their point of view with it. MJI have discussed dome flats for VISTA with WJS. It was pointed out to him PSB that having them 10-20% smooth would be fine. The main requirement would be to have a constant illumination. 2. Comments on WFAU minutes There was a bit more on data transfer issues (checksum stuff) - see item 6. 3. Report on JAC telecon of 28th January A number of questions were put to AA: 1. engineering chip tests: no news but found out that JAC get monthly report 2. getting SDF file for end-to-end test: AA will see to this 3. dome flats: confirmed that we don't need super uniformity 4. more service data? none is planned 5. commissioning personnel: agreed to plan but not expected to be before mid-August 6. CSV AA & AL: 3 applicants will be interviewed MJI said that AA wanted to know more about the number of reviews and was concerned about the number and their possible interference with WFCAM commissioning. The next telecon is scheduled for 18th Feb. This is delayed by about a week due to JAC being busy with Michelle recommissioning. 4. Report from PPRP meeting and VDMT on the 4th February MJI reported that the PPRP meeting went well. There were a lot of people at the meeting. Positive feedback was received after the meeting. STH said that the practice session on the previous day was useful in that all overheads were checked and that they could be pruned where necessary. WFCAM operations was not included in this bid and will have to be bid for later. VISTA operations are another matter. STH suggested that we read the notes taken from DWE's and his meeting in Leicester in order to get some further ideas regarding the running of operations. The VDMT meeting was an anticlimax by comparison. It was suggested that someone from CASU should attend the VoProc meeting in Leicester (25th February). JRL said that he would go. 5. Status of ESO PDR deliverables QFITS -v- CFITSIO JRL reported on his comparison between QFITS and CFITSIO. This was done by going through the pipeline code and seeing what additional functionality would be required in QFITS if the pipeline code had to be converted from CFITSIO. (1) tile compression (2) the ability to read a subset of columns or image pixels. (3) ability to access single extensions and image subsets within a file using the following notation: fff.fits[1] fff.fits[1][100:150,200:237] (NB: This is not a FITS standard, but it seems to be common practice in many FITS packages: CFITSIO, FUNTOOLS, IRAF, etc) (4) some, but not all routines return an error status (5) header keyword 'classes' -- allows you to exclude certain classes of of keywords when building a new file from an old one. (6) a good user's guide. The reference manual is not particularly detailed and too much is left unexplained. (7) no Perl interface (there is a restricted Python interface) (8) no WCS interface PSB said that this stuff might not be in the current version of QFITS, but perhaps some of it will be in the next version. A possibility might be to send a list of requested modifications and that would give them about a year before CASU has to start thinking about modifying its code. Another possibility is to include these as requirements in the Data Reduction Specification Document. PSB also said that CPL might have some of the required functionality. PSB at this point showed some plots of a simulated VISTA file that he had generated. 6. Data transfers to WFAU and Garching MJI reported on email exchanges he has been having with Eckhard Sutorius at ROE. WFAU want some form of external md5 checksum carried out on our files before they are transfered. This seems reasonable from the CASU point of view. A possible way of doing this would be "openssl md5 filename" (also "md5sum filename"). Routines for doing this also exist in Perl. MJI pointed out that Garching have stated that they would provide a checksum routine for carrying out a similar task. They have not done so yet. MJI said that an alternative is to rely on the checking built into normal tcpip transfers. This should confirm that the transfer has been successful. DWE said that an additional use of an external checksum would be to check that files haven't been modified since the checksum had been generated. Other matters discussed were: * There will be one directory per observing night. This is the way it will be processed and structured ie. on a nightly basis. * The issue of handling read-readiness will be done via a lock file set in the directory while the data products are accumulating in it and after the processing is complete the lock file will be removed. An alternate strategy would be to have a go file created when the data is ready to process. Another strategy suggested was to process the data in a temporary directory and rename it when the data is ready. Linked to this was the request from CASU that a receipt file be created after WFAU had copied and verified the data. * The expectation is that reprocessing will occur very infrequently and (probably) in large batches (eg. after bug fixes). This will require CASU to inform WFAU of the fact that reprocessing is happening and why. The data will go into the same directories. Given current plans are that WFAU would have to inspect the file headers to see which version of the software had being used. * no data will be officially released until CASU are happy with it. The exception will be test data. 7. Status of ISAAC reductions [the details need to be filled in or checked here] MJI showed some plots of his stacking experiments using the ISAAC data. A number of interesting features were pointed out. This data covers the Hubble Deep Field and is about 3 arcminutes square. The stack of all the data (representing n hours of observing) goes down to 2?th magnitude in K. A false colour plot using the J and K data was also shown. It is difficult to say if this reduction is better than the FIRES reduction since there is nothing to compare it against. The data will be given to the Subaru group in order for them to perform further stacking tests. There is a need to write a report on this work (see earlier action). 8. AOB --- MJI said that there would be another APM building meeting tomorrow. <<<< DWE showed the latest iteration of the documentation web page with the document numbers on it. The only two suggestions were to reduce the overall size down a bit and shrink the description column a bit. <<<< If there are any more documents that need numbers allocating to them DWE needs to know about them. <<<< DWE will then send out an email to the authors of the documents that need a number inserted. <<<< WJS's UK VDFS User Requirements Document was discussed. It was stressed that this was the UK version and not an ESO deliverable. It was decided not to discuss this in detail and that comments be sent to MJI by the end of the week and that these would be collated and passed on to WJS. <<<< (ok, so the plan has changed, but this is what was decided at the meeting). Continuing Actions ------------------ STH finish the bright star checking of the photometric standard fields (by end of next week) STH look at the spatial systematics in the UFTI standards data and new test data (by next meeting) JRL send WFCAM example FITS headers and files to Adam and Benoit STH acquire missing ISAAC data STH calibrate FIRES data using NICMOS or the published FIRES work JRL write a report on ISAAC reductions MJI New Actions ----------- MJI attend APM building meeting (Wednesday) DWE alter documentation web page (reduce the overall size and shrink the description column). ALL Inform DWE of any documents that require document numbers DWE email document authors of the numbers that should be inserted. ALL Email comments on UK VDFS User Requirements Document to MJI by end of week.