Report from CASU ================ Data transfers -------------- These are still running smoothly through completion of 09B and the first few months of 10A. As usual the raw WFCAM data automatically arrives in Cambridge and is checked, MEF'd, OMP'd and ingested within 24 hours of it being taken. The nightly processing typically runs a few days behind this. Checking the products, and bundling up the monthly photometry updates takes the rest of the time. The raw WFCAM data archive in Cambridge is up to date, as are the transfers of all raw UKIDSS and calibration data to the ESO archive. The current state-of-play including transfers and processing status can be found at http://casu.ast.cam.ac.uk/surveys-projects/wfcam/data-processing/ The total of raw WFCAM data received and processed is now around 200 Tbytes; processed data products are pushing 320 Tbytes. An interesting statistic from the first quarter of 2010 was that WFCAM generated 16 Tbytes of raw data, whilst over the same period, VISTA raw survey data came to 17 Tbytes (VISTA users tend to do more coadds and use longer integrations). One minor transfer hiccup had occurred on July 12,13 toward the end of 09A, when only 999 files out of 3186 and 3329 respectively had been transferred. This was only discovered a few months ago by an eager PI who spotted some missing files from his observing run. How this happened is unknown but quick checks at JAC and CASU revealed this to be a one-off event. The data was retrieved at the JAC end, transferred over and the nights' incomplete MSBs and missing data were (re)processed. Most of the missing data was for a PI programme but there were a few odds and ends of GPS and DXS data too. JAC have now initiated an extra check on numbers of files held at JAC compared to those received here as recorded on the above web pages. Cassegrain UKIRT archive ------------------------ The update of the Cassegrain UKIRT archive, to correct some missing FITS headers and some missing parts of datasets, has finally been finished and is now the de-facto default archive here. After completion a new offline backup on LTO4 tapes was made. WFCAM processing ---------------- Semester 09B processing is complete and the first two months of 10A processed data (February, March) have now passed their final QC checks and monthly photometry updates and have been flagged as ready to copy for WFAU. The usual occasional small problems are fixed as they occur, e.g. a few MSBs have incorrect RECIPE keywords but advance warning from JAC enables these to be corrected before processing begins. At long last some new linearity measures using the dome screen were taken, transferred over and analysed. There is a small non-linearity amounting to ~0.5% at 500 ADU and a similar level of non-linearity at 30K ADU, neither of which is worth correcting for. The new waveforms introduced on WFCAM around February 20th have not lead to any noticeable changes in the science products and as noted above the new linearity data taken with them also suggest no change in the pipeline is required either. Pipeline updates ---------------- There have been no significant changes recently. Fast-track releases ------------------- The agreed deal with WFCAM processed data has always been to release pipeline processed data directly from CASU when asked by the PI for more urgent access than normal. In exceptional circumstances this can be within 48 hours of the data being taken but is more typically one or a few weeks downstream. There have been various fast-track flat file release requests including: several parallax runs from U/10A/D1 within hours of the night being flagged as "processed"; others where weeks pass e.g. U/UKIDSS/B2 Ben Burningham; U/09B/K2 Myungshin Im; U/10A/J2 Yumiko Oasa; and even a few fast processing (and analysis) requests from JAC (within days) during the new waveform testing in January/Feburary. WFCAM reprocessing ------------------ The reprocessing queue has been whittled down some more by completing the UDS J-band reprocessing for 05B-07B. Given the large amount of effort involved the H-band data from the same period is on hold until confirmation of the gain, or otherwise, for the J-band data. The queue then got whittled up by a successful Autumn test on attempting to improve the sky subtraction for some earlier "cloudshine" data for projects U/06B/30, U/07B/6 and U/SERV/1763. A few frames of ancient WFCAM GPS data had their astrometry improved. These must have escaped the earlier trawl since although satisfying the standard checks they still needed some more TLC. Running the toolkit astrometry update software cured the problem without further manual intervention. A few small fixes have been required with a particular sample of stacked products for a PI programme U/09B/6 (Smith) which showed the well-known (but fortunately rare) astrometric registration problem during dither stacking. The latest pipeline version is more immune to these and worked well without any further intervention. UKIRT newsletter items ---------------------- After being bombarded with several queries about the workings of the nebulosity filter, the details were written up for the upcoming UKIRT newsletter. This is quite a general purpose algorithm and in addition to optical data has been used successfully on Planck and Herschel data too. The previous newletter also contained CASU contributed articles on: "Dribbling" (or correcting the PSF mismatches that occur in interleaved data) "Saturated WFCAM stars" (or extending the dynamic range of WFCAM data) to alert a wider audience to these issues and potential solutions. Funding for WFCAM processing ---------------------------- As noted in the last Board report, the CASU rolling grant application, submitted in May 2007, that covers WFCAM processing, reprocessing, troubleshooting, updates, user support etc.., was finally resolved in September 2009. Providing there are no further significant changes we expect to be able to continue to provide support at the current exhorbitant level of 1 FTE for all WFCAM matters at least up to the end of March 2012.