17th November 2005 Observatory meeting room 10.00 - 13:00 Present: DWE, MJI, PSB, STH, MR, JRL, RGM, WJS Apologies: NAW, EGS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report on VDMT and UKIDSS meetings - data release policy discussions (RGM) 4. Status of WFCAM processing - photom calibration (STH) - recent changes to pipeline (JRL) - progress on PSF fitting (DWE) 5. VDFS VISTA update - DRL v0.1, test data 6. Data archives update 7. Optical processing update 8. AstroGrid liaison update 9. AOB. Minutes ------- 1. Actions from last meeting WJS has not yet passed on the SADT comments to Martin Folger - ongoing <<<< JRL has not finished off the OMP database access scripts - ongoing <<<< MJI raised the issue of adding a data quality flag to the FITS header. JRL said that a 1-7 flag is set by JAC and can be added post-pipeline. <<<< MJI has not passed on feedback comments to MR on flatfield and sky properties draft report - ongoing <<<< MJI has given more URD feedback during the Edinburgh meeting. MJI continues to investigate the merged classification statistics - ongoing <<<< MJI has investigated the blended object complaints and has found that they are not present in the current version of the pipeline. MJI contacted Steve Warren to see who needed to attend the UKIDSS meeting. MJI, STH and RGM attended (see item 3). WJS said that visiting RAL during AIT3a would be good. This will be in the second half of December. WJS said that he would check up on this. <<<< MR have finished setting up the test dataset harness. This is on apm24_b MJI and consists of about half of the data from 8 April. RGM asked if there was any repeat data within this. MR has finished setting up the pipeline version log. PSB contacted the Computer Service regarding UKLight. We deferred this until later, but then forgot about it! - ongoing <<<< MJI have contacted Paul Aslin about possible lobbying of the IT Syndicate PSB about the Internet charging model. They will keep us posted if there is any news. MJI arranged a meeting to discuss design of DQC archive DWE will report of the progress of PSF fitting of 05A data later (item 4) WJS has not provided updated measurements of the VISTA filter characteristics - ongoing <<<< ALL sent in monthly reports EGS have not produced a set of options for group web pages and circulate DWE - ongoing <<<< 2. Comments on WFAU minutes RGM said that he had noticed that there was a draft outline of an archive paper and asked if there was a need for an equivalent pipeline one. MJI said that he was thinking about it. RGM raised the issue of communication between UKIDSS and CASU and was concerned that if there were problems with the data, was the information coming back to CASU? Specifically, there was a recent UKIDSS Survey Heads (?) telecon - was this minuted? MJI thought that CASU were generally represented at most of these meetings. The problems regarding the WFCAM guide stars was discussed briefly. 3. Report on VDMT and UKIDSS meetings - data release policy discussions (RGM) MJI said that at the UKIDSS meeting presentations were given describing the current pipeline, the SV1 results and from each of the surveys, the science being carried out. RGM's impression was that the pipeline is very well regarded since papers are being prepared already. He also pointed out that the results from SV1 were being used for PATT proposals. No new major issues were raised, but there was much discussion regarding cross-talk and whether some kind of flagging of suspected images can be carried out. MJI said that most of these comments referred to the old data and that no-one has yet commented on the data from the latest version of the pipeline which treats cross-talk. MJI agreed that he (?) would generate a diagram to explain the way cross-talk affects the data across the channels in order to help people write their own algorithms if they wanted to carry out any flagging. <<<< RGM said that the data release policy was in disarray. There was much discussion at Edinburgh, but no real consensus. The DXS and UDS groups want faster access to the data. However, Steve Warren is concerned about how the data will be used. There are also Science Verification issues in that he felt that the data should be formally passed through the SV process before being released. This is still being discussed by email. MJI said that from the CASU point of view, the products are ready for release now. STH said that the issue of star/galaxy separation was also raised. MJI reported that there was nothing major from the VDMT meeting held on 8th November and that there were no decisions made. 4. Status of WFCAM processing - recent changes to pipeline JRL said that main changes that have been carried out to the pipeline recently are to the sky subtraction process. This has now been reorganized so that it is more robust. Also, the skies are now based on exposure time. An additional additive component has been identified and the best way to correct for this is to have the skys based on both filter and exposure time. MJI said that this was linked to the illumination dependant pedestal. Currently, JRL, MJI, MR and RWA are processing the 05B data. - photom calibration STH said that there is pressure to change the observing strategy for standards. It is felt that too many standards are being observed. This takes about 7 minutes every 1 (or 2) hours. The UDS are unhappy about this, since they don't need this. Current investigations show that using 2MASS for the calibrations may meet the specification, so it may not be necessary to observe standards at all. There are possible problems with the UKIRT faint standards which are being looked into. 14 nights have been identified as having very good photometric conditions and are suitable for investigating this. By the end of the month a recommendation is needed for the observing strategy for standards. <<<< MJI said that at one of the Edinburgh talks Roc Cutri had described the 2MASS photometry as being good to 2%, so it should be good enough for us. STH pointed out that for many nights the photometric zero point varies quite a bit and at timescales shorter than the frequency at which the standards are being observed. Thus there is a further need for 2MASS to be used for the calibration. The systematic radial term is also being investigated using stacked data and comparing this to 2MASS. The calibration of the Y and Z data is another problem which will probably require some system to be bootstrapped. MJI asked if we still needed to generate a set of secondary photometric standards. It was felt that this was the case and that these would be useful for VISTA. STH said that one of the limitations he had found was that there were not enough stars in the fields that he was using to characterize some of these features and that he will have to select a more crowded field in which to carry out these tests. <<<< - progress on PSF fitting DWE reported on the recent work on PSF fitting. The current model is based on the adaptive kernel one from Alard and Lupton. Due to having 36 coefficients there are robustness issues, but recent modifications may have dealt with most of these. It was suggested that a circular aperture (rather than the current postage stamp) be tested to see if that helped with the <<<< robustness. DWE said that this would make next to no difference to the fit itself due to the weighting used. Tests have been carried out to automatically process all the catalogue files for complete nights (8 & 9 April) and it takes about 1.5 hours per night to carry out the PSF measuring and 2.5 hours for the PSF fitting. These tests were carried out on apm3 (2GHz AMD machine). Tests show that the astrometric accuracy in the mid-magnitude range is between 50% and 100% better, however at the bright end this is limited by the atmosphere so pipeline and PSF accuracies are the same. For the photometry, PSF fitting shows no significant improvement over what is produced by the pipeline. It was suggested that plotting a colour-colour diagram of a crowded field (eg. M17) using first pipeline and then PSF photometry would also reflect the photometric accuracy with the advantage of not requiring repeat observations. DWE said he would look into this. <<<< A very limited amount of UDS test data was recently taken and this shows that there is next to no difference between 2x2 and 3x3 interleaving. A third PSF report is being written. MR reported on WFCAM data reduction issues. The survey status web pages now have the date added to them. 8 new tapes arrived last Friday that cover up to 5 November. Processing has been completed up to mid-October. So far only 3 imcomplete images have been found in 05B, so the 05A problem has been solved. The data for 05B has been read from tape with no problems showing that the tape reader is very reliable. 5. VDFS VISTA update - DRL v0.1, test data PSB showed an image taken by the VISTA camera. Obtaining such data was very useful since problems get identified early on. For example the negative numbers of the image were a bit of a surprise. Work on the TCS is making rapid progress. Last week useful discussions were held between PSB and Steven Beard. WJS reported the latest VISTA news. The AIT2 (test?) went well. Cooling and vacuum performed well. There did seem to be a problem with the filter wheel being very noisy. WJS then gave his impression of the noise. Although very good, it is unlikely he will get very far on talent shows with this. JRL said that the DRL tests are ongoing. The main work consists of using code checking utilities such as lint and splint. There was some comment regarding Doyxgen. JRL reminded us that the code written so far can be used in a parallel processing manner since the extensions can be processed individually or in one go. It is up to the people at Paranal whether they use the parallel option for their summit pipeline. (I don't think you want to mention that they have no money to buy any computers.) PSB made some comment regarding deliverables and recipes. There was an expressed desire to provisionally deliver these recipes early for comments before the formal delivery date. (This is a bit sketchy, but VISTA stuff tends to make me lose concentration) 6. Data archives update Nothing new to report 7. Optical processing update Much processing is being done (IPHAS, Subaru, ...) 8. AstroGrid liaison update STH said that he had been invited to join the AstroGrid Deployment Advisory Group. Floor is also part of this group representing Planck. RGM said that the next AstroGrid Consortium meeting will be held at Jodrell Bank on 5/6 December. 9. AOB. JRL raised the issue of secretarial support, which will be needed for helping to organize ADASS 2007. We were reminded that CASU have 0.8 of a secretary and Di Sword was the designated person. Continuing Actions ------------------ WJS pass on SADT comments to Martin Folger JRL finish off the OMP database access scripts MJI feedback comments to MR on flatfield and sky properties draft report MJI finish investigating merged classification statistics WJS finish sorting out the CASU visit to RAL, in particular check that having it coincide with AIT3a is OK PSB report on his findings regarding UKLight WJS provide updated measurements of the VISTA filter characteristics EGS produce set of options for group web pages and circulate DWE New Actions ----------- JRL add JAC observing quality flag to the FITS headers MJI produce a diagram to explain the cross-talk STH recommend a strategy for observing standards STH select a more crowded field in which to carry out photometric tests DWE test circular apertures for PSF work DWE generate some colour-colour plots from a crowded field to test the PSF photometry