29th April 2008 APM comfy chairs 11:30-12:40 Present: STH, NAW, RGM, PSB, MJI, DWE, JRL, MR Apologies: RWA, EGS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Recent meetings/visits - VMC meeting in Herts - upcoming Video meeting - NAM 4. Data archives update - AAT, ING, WFCAM, UKIRT 5. Other processing update - VST news - IDR IPHAS paper - HawkI 6. WFCAM update - illum corrs tests/checks - photometric calibration paper - ftp transfers - flippin' jitters - processing status 08A - upcoming UKIRT board meeting 7. VISTA update - general news - new SADT release - pipeline progress - data connections to ESO; ftp tests 8. ADASS 2007 - done ? 9. AOB Minutes ------- 1. Actions from last meeting MANY have provided comments on the latest draft of the photometric calibration paper. STH said that comments from DWE, NCH and Paul Hewett had been incorporated. Two more items need to be dealt with and the paper will then be submitted. JRL has finished off checking why the pipeline catalogue software and toolkit give different results in a few specific cases. They now agree to within rounding errors. STH has not finished checking if the information on the Plone astrometry technical pages is up to date - ongoing <<<< MJI is still waiting for the transmission curves for VIRCAM Z and N118 filters from JPE - ongoing <<<< MJI has found out what SDSS do about elliptical apertures for galaxy photometry. The short answer is that the standard pipeline analysis eg. Petrosian magnitudes, is based on circular apertures. The slightly longer answer is that from DR5 an alternative method is used (and recommended), known as Cmodel magnitudes. These are model fitted fluxes based on a linear combination of exponential and de Vaucouleurs model fits. Each component is elliptical. This is similar to the planned future upgrade for elliptical model fits based on Sersic profiles. MJI still needs a final check that the SMPs are up-to-date - ongoing <<<< RGM JRL has not revived the changes log web page for WFCAM pipeline - ongoing <<<< JRL has held lengthy and detailed email discussions with JAC staff about enabling the pipeline to cope with offset sky strategies STH has not yet looked at the new SADT - ongoing <<<< MJI has acquired a copy of the latest VISTA integration schedule but has not yet got his mitts on the commissioning plans - ongoing <<<< JRL has negotiated and fixed a date for the CASU barbeque - see later 2. Comments on WFAU minutes MJI noted from recent WFAU minutes that WFAU thought our commenting on the IAC request for access to all the WFCAM processed image data was an April fool. Sadly: a.) we didn't think of it and b.) the request was genuine. We have not heard any further on this topic but await Eduardo's return from La Palma with interest. MJI also suggested that STH get in touch with NJC and ETWS about the tests he carried out using the LAS to confirm the efficacy of the illumination correction. MJI independently confirmed the detector level improvement a while back using his U/05B/7 data on M33. <<<< There was a brief discussion about the radial photometric distortion correction and whether or not it had already been applied in DR3. RGM supplied a graphic showing the size of the affect for different passbands and asked if it, or something similar, could be included in the Plone technical pages. <<<< 3. Recent meetings/visits Not a lot is happening on the external meetings front. MJI noted that he had attended the VMC planning meeting at U. of Herts. on 8th April and gave the usual CASU VDFS spiel. RSC was also there and did the same for WFAU. A fair bit of science was discussed but no new surprises about processing came up. MJI noted there was an upcoming Video survey team meeting end of May/early June but no date or venue was available yet. He wondered if VHS, Viking and VVV were going to do the same at some point, but all had been quiet in the Hobyar House. He also asked if anything significant, other than the zero-cost model for ground-based survey astronomy, had been aired at NAM - apparently not. MJI pointed out the upcoming Euro-VO Workshop on how to publish data to the Virtual Observatory, 23-27 June 2008 at ESO, Garching. The registration deadline is 14th May 2008 (http://www.euro-vo.org/dcaworkshop2008/index.html) 4. Data archives update AAT: JRL commented that we had recently received more AAT data which had been ingested into the archive, which was now up to date. He also reported that we have "A satisfied customer. We ought to have him stuffed." - Basil ING: The raw archive is up to date to end of March. April data is awaiting the 30th to arrive before it too gets the ingestion treatment. UKIRT: JRL said the Cass archive was up to date in terms of available data and that he was awaiting the next batch. WFCAM: MR reported that the raw archive was tracking the newly arrived WFCAM data a few days in arrears due to the various checks, MEFing and fixes (see later) that occur before ingest. Tape backups of 08A raw WFCAM MEF'd and checked data are tracking about a week behind that. 5. Other processing update VST news: MJI relayed the following snippet, "I hear consistent reports that the VST should be operational starting from 2009 (the M1 cell should be sent to Paranal end 2008)". VISTA is still the front runner but watch this space. IDR IPHAS paper: This has carefully negotiated its way through the refereeing process and is now "in press". HawkI: RGM reported on a comparison of manually driven IRAF-based reduction of science verification data from HawkI with the same data processed by JRL using a variant of the CASU NIR pipeline. The CASU version goes 0.2 mag deeper. 6. WFCAM update Data transfers: FTP'ing of raw WFCAM data from JAC continues to work seamlessly with data typically arriving here within 24 hours of it being taken. A new, but fortunately minor, problem related to scaling of coadded exposures was noticed whilst processing the first few nights of 08A data. The cause of this was traced to the FITS compression conversion due to a software upgrade of the external cfitsio library, which we had supplied prior to 08A data taking. A patch has been issued to JAC to fix it. Already transferred data is fixed here in situ prior to ingestion to the raw WFCAM archive and subsequent transmission (UKIDSS and CAL data) to ESO, and of course pipeline processing. Processing status: In spite of the scaling problem and a few others relating to missing darks and missing RECIPE keywords, processing is rapidly catching up with data taking, though final checks and monthly zero-point updates remain to be done. As usual the status pages for 08A show the current state of play http://casu.ast.cam.ac.uk/surveys-projects/wfcam/data-processing/ and derived quality control diagnostics. All of Febuary 08A is now checked and flagged as read to copy to Edinburgh. Some requests for urgent early access to various bits of processed service or PI data for 08A have been received (20080401 Patel U/07B/18; 20080331 Tedds U/SERV/1779; 20080331 McMahon U/SERV/1783; 20080310,20080320 Haines U/08A/32) and were dealt with by directly transferring processed data to the PIs. JRL expounded briefly on the discussions relating to automating offset sky data taking, and subsequent pipeline processing, for observations involving large extended objects. In a nutshell two modus operandi have emerged: flipping large extended objects between pairs of detectors either as 1,4 <-> 2,3 and 1,2 <-> 3,4; or for bigger objects genuine offset sky alternating with on target exposures. The former is a cunning plan as it provides an efficient way to be always integrating on both target and sky, simultaneously even. Photometric calibration paper: The status of this is as noted earlier. The remaining "todos" are confirming the overall spatial coherence and accuracy of the absolute calibration. Other: MJI noted that there was an upcoming UKIRT board meeting in Oxford, May 15-16. A copy of the submitted (to JAC) CASU and WFAU reports was circulated with the draft agenda (of this CASU meeting). He also commented that he had not seen any UKIDSS report yet. MR reported that he had been contacted by Tiziano Valentinuzzi from Padova about stacking and mosaicing some U/05A/2 project data on Abell Clusters. A preliminary comparison somewhat surprisingly revealed that the CASU stacked and mosaic'd image using the CASU toolkit software on a trial cluster (starting from the normal pipeline science products) gave results that went 0.75 mag deeper than the SWARP'd MONTAGE'd image created using the same data in Padova. MJI also noted that he had seen a bunch of recent emails relating to putting the UofH proposed 50% share of time for joining the UKIDSS consortium back on the negotiating table again. 7. VISTA update We all noted the encouraging news about safe mirror delivery to Paranal and the subsequent press releases. The phrase "Full scientific operations are due to start early next year" caught the eye. ESO have released CPL 4.1.0 which incorporates JRL's CPL interfaces to WCSLIB. Ironically this will now require a compatible release of the VIRCAM Data Reduction Library, which incorporates a private version of the interface, which should be announced soon. We are still waiting to receive the digital versions of the Z and N118 filters and likewise feedback on the current Calibration Plan draft. A repeated request for the "CASU wishlist" for VIRCAM data was responded to with a repeated reply. It seems unlikely that any more lab data will be produced much before camera first light. Some further analysis of the July 2007 data is being undertaken, however the high-order effects that need investigating are really too correlated in the existing data and there is a danger of over-interpretion. Networking tests to ESO are underway sporadically. Florian Heissenhuber, "Network Specialist" has been working with PSB. Simple ftp's seem to achieve ~5MByte/s which is easily fast enough for the transfer of raw VISTA data from Garching to Cambridge (an average 250Gbyte night should Rice-compress to ~60Gbyte which at 5Mbyte/s would take about 3.5 hours to transfer). The preliminary conclusion is that the marginally adequate transfer speed of raw UKIDSS WFCAM data is, as previously suspected, due to the rate-limiting bottleneck of the ESO database ingestion process - that would not be a problem for transferring VISTA data to Cambridge. Further tests are stalled pending ESO IT support installing the "iperf" tool. The RECIPE keyword and allowed values have been in the news recently. These are still very much a work in progress and final agreement on them will probably not be possible until after on-sky science verification has been completed. 8. ADASS 2007 The proceeding are outta the door. 9. AOB Infrastructure: A 3COM 5500G network switch has been installed in the APM building as part of the IoA's upgrade to 10Gigabit backbone and the University's upgrade of the phone system to VOIP; the switchover of machines will happen later (there may well be a glitch in the UKLight VLAN at that time (and the APM WIFI!)). CACC: There will be a meeting of this august body on Thursday. MJI and PSB will attempt to find out what the current University network usage costing model is in light of all the lovely FTP'd incoming data. <<<< Meeting area: The partition isolating the computer rack noise was completed on Friday hence it is time to consider setting up a group meeting area. MJI will investigate meeting table options and DWE will liaise with Andy Batey <<<< about wall-mounted display screen options. <<<< The new fridge-like quality of the main room area (not surprising given the location of the sensor) is already being taken care of. BBQ: There was a brief discussion about arrangements and preferences for the CASU summer? BBQ, but as you don't need to know about that ..................... Continuing actions ------------------ STH check that Plone astrometry technical pages are correct MJI acquire transmission curves for VIRCAM Z and N118 filters MJI finish off checking and acquiring the latest SMPs RGM JRL revive changes log web page for WFCAM pipeline STH download, install and try out new SADT New actions ----------- STH contact NJC,ETWS about details of LAS tests on illumination correction STH include graphic showing the size of the photometric radial distortion correction in the Plone technical pages. MJI find out at CACC meeting what the network costing model currently is MJI investigate meeting table options for the APM building DWE liaise with Andy Batey about an APM wall display screen