WFCAM report from CASU ====================== Data transfers -------------- 08B transfers are working very smoothly with minimal manual intervention or feedback required from this end. It usually takes a few hours to transfer a night of data which means the data is in Cambridge less than 24 hours after it was taken. Another cron job running here then automatically checks, verifies and reformats the 4 separate data channels to single multi-extension FITS files (MEFs), and moves them from the data transfer disk to their final resting place. This step now also includes inserting a flag in the FITS headers to record the MSB status manually recorded at the summit and logged in the OMP. This is accomplished via mirrored access here to the relevant OMP DB. Shortly thereafter the MEFs are ingested into the raw WFCAM data archive here and are ready for processing. When enough nights have accumulated to fill an LTO2 tape they are also backed up and saved for off-line storage (in another building). All raw UKIDSS and calibration data available at CASU for 08A has also been transferred to ESO and 08B transfers are on schedule. To check the status of transfers and processing at anytime see http://casu.ast.cam.ac.uk/surveys-projects/wfcam/data-processing/ WFCAM processing ---------------- Due to the variable level channel "offset" problem first noted during 07B and the additional photometric calibration procedure to compute monthly illumination corrections, pipeline processing now proceeds in three stages. Step 1. is the usual processing on a nightly basis. By arranging with JAC to have a full set of twilight flats taken during the 1st week of observing for 08B this has enabled us to udpate the master calibration files and start processing a night of data within about 48 hours of it being taken. This gap has been maintained throughout the semester and augers well if and when WFCAM is on 100% of the time. Step 2. involves fixing the parquet floor pattern in those programmes where it is still noticeable after standard processing. This also involves restacking and recataloging as outlined in our previous Board report. Step 3. is the "monthly" detector level calibration and generation of the illumination correction tables. The new data transfer setup from JAC now allows us to offer a (limited) fast track processing for urgent data requests where residual channel offsets are unimportant and for which photometric calibration to 2% accuracy is sufficient. This was (unexpectedly) trialled successfully for project U/08B/33 - followup of high Z QSO candidates - where the processed calibrated data was with the PI within 48 hours of the data being taken and only 24 hours after the PI contacted us. We also continue to provide the usual direct rapid access to any stage 1. flat file data products to any PI who requests it. After some discussion with JAC, and after a few trial iterations, we have now implemented a faster turnaround quality assurance of MSBs via an MSBTid log accessible on the processing status web pages. On completion of stage 1. processing for a night, each science product component of an MSB is compared to the observational constraints specified in the OMP and recorded in the FITS headers. All MSB components are listed and those that fail to meet the set criteria (cloud cover, seeing, J-band sky brightness) are flagged. This also logs and flags the MSB accept/reject information recorded at the summit. Early 08B data also saw a few more tests of processing of different observing modes, and relevant header keywords, to automatically make use of offset skies for large extended objects. This seems to have worked rather well, although still needs further testing of the products to verify this. Over the summer, 08A processing was completed on schedule i.e. processed, fully calibrated, checked and flagged ready for transfer around a month after the last data was taken. WFCAM reprocessing status ------------------------- See final item. WFCAM calibration ----------------- The paper describing the end-to-end WFCAM photometric calibration was submitted to MNRAS over the summer. Referees comments have been received and answered and although nothing serious arose required a few more tests to be run. This has now been resubmitted and hopefully should be in press and on astro-ph shortly. As noted earlier, individual detector magnitude zero-point corrections are computed approximately monthly and are adjusted in all FITS headers before files are flagged as read to copy to the WSA. Post-processing application of the illumination correction will be applied at the science archive end. Funding ------- In spite of our best efforts there has still been no progress on the rolling grant application submitted nearly 18 months ago. Current funding runs out for several CASU activities, including WFCAM processing, on 31st March 2009. However, in order to extend our existing grant that far we have had to hire out 50% of the time of one of our expert software personnel to the Planck project over the current financial year. In addition, another key team member is retiring through ill health at the end of this year. Although neither are heavily involved in WFCAM support the knock-on effect of this from our other projects, and our inability to recruit replacements, or the requested new hires, has now become critical. So far we have successfully minimised the impact on 08B WFCAM processing, which we have given the highest priority. However, due to pressures from other projects we have had to delay, or indefinitely postpone, a series of reprocessing requests which would normally have been carried out in parallel. These include: updating catalogues to fix a bug in Petrosian magnitudes for deblended objects - this affects all processed WFCAM data; reprocessing UDS data from 05B to try and improve the rejection rate; and re-analysing GPS data in regions of high nebulosity to improve source detection and photometric accuracy.