16th September 2005 Hoyle meeting room 14.00 - 16:00 Present: MJI, PSB, STH, MR, JRL, RGM, WJS (RGM & WJS partially) Apologies: DWE, NAW, EGS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Review of VDUC and VDMT meetings 4. VDFS VISTA update 5. Data archives update - UKIRT, WFCAM, AAO, ING 6. WFCAM news and JAC telecons 7. Upcoming UKIDSS SV meeting 8. Optical processing update 9. AstroGrid liaison update 10. AOB - hardware - ADASS - PD33s Minutes ------- 1. Actions from last meeting STH has yet to trial test version of SADT, PSB said he would help out PSB by contacting Martin Folger re: the software and installing it here - ongoing <<<< JRL has still not finished simple access scripts to mirrored OMP database tables - ongoing <<<< ALL assorted presentations were placed in the appropriate web pages MR is still writing up the flat field properties and sky frames characteristics report - ongoing <<<< MR has put a "known features" page on the WFCAM status pages (note EGS to check that this is publically accessible since both MJI and RGM had not been able to access it remotely) <<<< WJS has checked the VISTA template definitions and pointed out to SMB a few areas where enhancements are desirable (eg. filter loop and different exposure times) PSB has contacted JPE about the status of the SADT with respect to the RIXs and is still awaiting a response EGS has written a test script to remove duplicate entries in the ING DVD and CD databases, he and RWA will test this out shortly MJI produced a PDF version of the catalogue parameter document and DWE linked it off the web pages MJI produced a draft UK VISTA deliverables document and distributed it STH for comment (see also item 3.) ALL examine new catalogues for bugs, errors on the errors and so on. Several tests were made and a few interesting features noted that lead to several minor enhancements MJI is still waiting to confirm AAO plans for AAOmega data processing which he will progress at the AAT Board meeting next week - ongoing <<<< 2. Comments on WFAU minutes There were several comments on the comments about the issue of how to deal with transferring new processed data versions. MJI had been out of the loop on this due to travelling back from observing but said he would investigate the recent emails with WFAU and sort it out. He also noted that he had thought that an agreed policy was already in place. <<<< 3. Review of VDUC and VDMT meetings MJI, STH, PSB and WJS gave their impressions of the recent VDUC meeting and the substantive issues that were raised. WJS noted that he had sent in a list of actions from the meeting to the Chair but noted that there was still some internal debate on the exact wording of the actions. Regarding data releases: the consensus was that a release to the UKIDSS community (NB. also ESO) of Semester 05A data ought to take place by the end of the year since it would have then been 6 months after the last data for that semester was taken. Version 1 of the pipeline processing for this semester is has been complete since the end of July. Reprocessing for Version 2 of the 2D images is complete and a test Version 2 of the catalogues for SV1 has been available for the last 3 weeks. However, since there seemed to be no SV impetus for a release of this to the SV1 testers at the last SV meeting we have taken advantage of the hiatus to carry out further tests on these catalogues. A few minor improvements have subsequently been made and the SV1 catalogues will be reprocessed to generate the official Version 2 by the end of next week (~22nd September). This will then be extended to the whole of 05A over the following two weeks. <<<< Simon Dye again presented the puzzling differences in photometry he was finding, but during the meeting the problem that he was having was resolved by STH and his photometry is now in agreement with the CASU pipeline version. A further substantive point discussed, but not resolved, was the transfer of data for VISTA from Garching to Cambridge. The only sensible long term (ie. economic) option is to use internet transfers. However, this raises the tricky issue of the bandwidth bottleneck into and out of ESO Garching. This will be followed up at VPO level. There was some discussion about the details of the URD and MJI is to provide some further feedback on several items to the next VDMT meeting <<<< The day after the VDUC meeting there was a VDMT meeting held via telecon. STH reported on the actions from that meeting. There was considerable overlap on topics with the previous days VDUC meeting but the main items to resolve from CASU's point of view are the diverging dates and version contents of the planned deliverable releases. 4. VDFS VISTA update PSB reported on the progress with VDFS deliverables. A prototype DRL recipe for dark frame combining has been produced and used as a trial to shakedown our understanding of some of the interface issues. Delivery of DRL v0.1 is on track but will soon require delivery to CASU of simulated FITS files with full (pseudo) header information present to fully end-to-end test the various components. JRL noted that he had almost finished porting the CASU image detection and cataloguing code to run within the CPL environment. PSB noted that the HOWFS FITS specification was looking good and had welcomed the chance to feedback comments. 5. Data archives update - UKIRT, WFCAM, AAO, ING UKIRT - up to date no further changes: WFCAM - MR noted that the first batch of tapes for semester 05B covering the period 25th August - 4th September had arrived and had been ingested into the raw archive system. He also noted that there were no missing files. MR then gave an impressive demonstration of the DQC products database and query system that he and EGS have been developing. This was well received and provides a lot of the missing functionality in chasing up progress, feedback from external users and so on. MJI noted that a survey progress web page run off the backend of this database could provide automatic updates of the progress of the UKIDSS WFCAM surveys and would be an excellent prototype for the required VISTA survey progress databases. MR and EGS will investigate this extra appendage. <<<< AAO - JRL reported that he had almost finished setting up the data archive access and would attempt to complete the remaining scripts by next week. <<<< He also noted that the AAO are planning to changeover from Sybase to MySQL sometime in the near future which will require some reworking of the archive content mirroring scripts. ING - MJI reported on a fruitful discussion that he and RWA had held with their ING counterparts during last week's final CMT management meeting in La Palma. At this meeting it was agreed that internet transfers would now be the sole means of shipping data as of the end of June 2005. We currently have in Cambridge DVDs covering the period up to mid-February 2005 and the remaining period from February to June, plus some overlap, will be shipped on hard (UBS2) disk. ING have the last ~15 months of data on RAID disks and prior to that local tapes going back to January 2000. RWA will finish off the ingest of the DVDs (the exact period covered contingent on the hard disk delivery) for the INT on one half of the new disk server which will also be the final resting place for the internet transferred data, currently buffered through a RAID disk on another PC. The final step in placing all of the ING archive online will be the ingest of the remaining CDs for the INT, although alternative strategies for the latter will also be considered. 6. WFCAM news and JAC telecons As noted earlier, the first batch of WFCAM data for 05B has been received and ingested. The image quality for WFCAM was reportedly much improved with the whole focal plane now in alignment. We do not currently have actual average figures but MJI noted that the data taken for the UDS 2x2 -v- 3x3 tests averaged 0.7 arcsec image quality across the entire array. The data for one detector (#3) was transfer directly from JAC via internet and is being used to assess the merits of doing deep surveys with alternative MSB strategies. Initial results on image quality, seeing, enclosed energy, detection, suggest (unsurprisingly) little difference in the final data products at this level of average seeing. MJI noted that he had also used this data to test drizzling the interleaved 3x3 data, at 0.133 arcsec/pixel, onto a 0.267 arcsec grid and had obtained good results. Further tests are still ongoing in conjunction with Omar Almaini the UDS PI. 7. Upcoming UKIDSS SV meeting STH noted that in addition to the SV meeting in Cambridge on the 21st, there would also be a WFCAM calibration working group meeting in Cambridge on the 20th. STH presented the draft report that will form the basis for discussion at the calibration meeting (and will no doubt also be discussed at the SV meeting). He noted that generally the agreement between the default 2MASS calibration and the UKIRT FS standard system was excellent but that there was still a puzzling offset, particularly in the H-band of ~3.6% between the calibrations. He showed that colour equations from WFCAM to MKO could not account for this since they are essentially zero and he also noted that the nightly extinction on "good" nights (eg. 8th and 18th April where many standards were taken) was very close to the default values of 0.05 mags being used in the pipeline. PSB suggested directly comparing the mags of UKIRT FS stars in the 2MASS catalogue, thereby eliminating WFCAM from the equation, as a direct check on the integrity of the different systems and the colour equations adopted. <<<< STH noted that he had done some more work on the residual photometric spatial systematics and was only finding an overall effect at the +/-1% level over the whole array. MJI noted that this is already within the systematic photometric error budget. STH said that of order 20 external people would be coming to the SV meeting and MJI asked if other CASU members could attend to help answer questions about WFCAM data processing since he would not be there. 8. Optical processing update No further news on this. 9. AstroGrid liaison update Nothing to report. 10. AOB - hardware MJI noted that 2 more disk servers have been delivered and the first of these has been installed. One of these 4 Tbyte partitions will be used for ING archive storage, the other will be for raw WFCAM data storage. - ADASS MJI asked for any further news of the upcoming ADASS meeting and if any preparations were required it. JRL noted that his spot had been upgraded to a talk and that he had plenty of WFCAM material to base it on. - PD33s PSB asked if MJI could read the draft PD33s before they were sent off to IoA admin for ingestion into the system. MJI said, yes, if you are quick, <<<< but noted the Paul Aslin had indicated that the drafts were generally fine and covered what was wanted. - Other STH raised issue of Plone and Solaris web pages for discussion at next CASU meeting. <<<< Continuing Actions ------------------ STH trial test version of SADT with PSB's help to install software PSB JRL finish access scripts to mirrored OMP database tables MR finish writing up the flat field properties and sky frames report MR ensure features web link is publically accessible EGS MJI find out about AAO plans for AAOmega data processing New Actions ----------- MJI investigate problems regarding new processed data versions with WFAU MR reprocess the catalogues for V2 release JRL MJI append more details to the URD feedback for the next VDMT MR investigate adding survey progress web pages to DQC database EGS JRL finish AAO archive access scripts STH do a direct comparison of UKIRT FS and 2MASS photometry MJI read and comment on the PD33s ASAP ALL think about issue of Plone and Solaris web pages for discussion at next CASU meeting