26th January 2006 Hoyle committee room 10.00 - 12:40 Present: DWE, MJI, PSB, STH, MR, JRL (RGM after 11:30) Apologies: NAW, EGS, WJS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. VDFS VISTA update 4. WFCAM processing 5. Upcoming VDMT meeting and this quarter WPs 6. Data archives update 7. Optical processing update 8. AstroGrid liaison update 9. AOB Minutes ------- 1. Actions from last meeting JRL has finished off the OMP database access scripts. There was a bit of confusion regarding this item. MJI asked about the OMP database. updates. JRL said that the main problem with the mirror is related to host OMP database changes eg. IP address. The mirror is broke, and he is looking into it and will reflect on the problem. JRL has not provided an MSB quality script which MJI will use to update the FITS header during pipeline operations - ongoing <<<< MJI continues to investigate the merged classification statistics - ongoing <<<< WJS was not present so it is unclear what was happening about AIT3a and a possibile visit - ongoing <<<< DWE had a meeting to discuss the future design of VDFS web pages. The main EGS result from the questionnaire was a majority in favour of an all Plone MJI system. DWE and EGS are currently working on a trial system. EGS has generated some stacked DXS data and catalogues. It is likely that this will be part of the Early Data Release (EDR). Though there was some confusion about how this fitted in with what WFAU were doing. MJI produced colour-colour and colour-magnitude diagrams of IC4665 and M17 and an LAS tile, along with a brief howto. This can be found at http://www.ast.cam.ac.uk/~wfcam/docs/reports/sv/#science MR was asked to make this visible from the Plone system as well. <<<< MJI asked Konrad Kuijken about the ordering of OmegaCam extensions within a FITS MEF file. OmegaCam delivers a fixed order until system meltdown. MJI clarified the deliverable dates for DRL versions. He noted that from 1618 the minimum next step deliverable is v0.5 which is now due September 2006, the date is set relative to the date of commissioning. He also noted that CASU intend to produce interim versions with corresponding updated calibration and DRLD documents. PSB have not suggested a suitable strategy for jittering VISTA JRL twilight flats. This is not a priority at the moment - ongoing <<<< MJI DWE released the latest PSF report http://www.ast.cam.ac.uk/~wfcam/docs/reports/psf3/ JRL has not finished the mods to AAT archive. He has asked if there is any more AAT data that needs ingesting, but has not had a reply - ongoing <<<< JRL has not asked if there is any more UKIRT data that needs ingesting - ongoing <<<< EGS have sorted out an automatic and reliable reboot for apm14 and apm15. MR MR said that PSB has the root password is for both machines. MR also noted that the current Plone system is accessed directly via their machine names, so it would not be easy to switch over in the event of a hardware failure. DWE said that this would be sorted out with the new Plone system. 2. Comments on WFAU minutes PSB noted that the UKLight options are still open and will be interested in the outcome of investigations. RGM noted that the transfers to Edinburgh are now 2 months behind and that the ESO transfers have overtaken this. STH asked what data was being released as part of the EDR. (This was discussed later - item 4) 3. VDFS VISTA update Telescope status - MJI said that the progress as reported at various recent meetings was very promising and that WJS gave a report at the VISTA HS Science Surveys meeting. VIRCAM - PSB said that Gavin Dalton had reported that they had traced the source of the stray light entering the camera. The AIT3a tests are currently in progress. PSB was uncertain what the plans for the next few months were. VDFS Documentation - PSB said that he had identified 9 documents that were pending in that they had not been signed or released yet. He announced that he was making it his personal campaign (PPC) to get these documents sorted out. He said that he would also like these documents freely available on the web rather than sitting on our internal system. MJI echoed this sentiment and in particular would like the ETC document available. DRL status - It was pointed out that v1.2 of the above documents are those that are regarded as the final FDR documents. It was also noted that as JRL advances the DRL, the appropriate documentation is being updated. JMS has suggested Sandra Castro writes a document reviewing the VIRCAM recipes and then have a telecon to discuss this. It was felt that it would be better to get technical feedback and then decide what to do based on whether only a few minor tweaks are needed or a major redesign would be required. ETC - This new CASU requirement occupied most of PSB's (& some of MJI's) time during January. It was released on 15 January. The main difficulties were the short timescale for this unscheduled software project and that some of the characterisation data were in diagram form only and had to be manually digitized. Although help pages were generated, this was handicapped by the ETC document not being available to users even though the ETC was. Workstations at Paranal - Although there has been some movement on this front, it is still not clear if more than one will be available. VDFS UK Review - MJI distributed the Terms of Reference for this review, which however is open-ended about which documents are needed. It was noted that the UK User Requirements Document has still not been signed off and was now a bit behind events. A likely date for the review would be the end of April. It was not known who would be on the review panel. These issues should be raised at the next VDMT meeting. <<<< MJI pointed out that there are 3 FTE working on VDFS development until the end of September 2007 and that at the current rate of progress some retrenchment may be required. VISTA Surveys - There have been a number of meetings recently: VISTA Hemisphere Survey (Betamax) in Cambridge - PSB, MJI and STH went to it, GPS in Hatfield and also a VPHAS++ meeting on GPS in Warwick, and a SuperDXS (which will be held tomorrow at Oxford). These surveys in general will match the VST optical public survey programmes. VISTA operations - It was unclear, to MJI at least, if we were processing all VISTA data or just UK-led parts.. 4. WFCAM processing MR said that the processing was progressing well with data up to 10 January either pipelined or in the pipline. We have now reached the steady state stage of waiting for tapes to arrive. In a nutshell, data up to 31 December has been processed and checked, up to 20 October has been transfered by WFAU and up to 25 October has been transfered to ESO. JRL reported on the improved (in an rms sense) sky subtraction algorithm. He said that he would wait until the current batch of data has been processed and then test the algorithm on some real data (about end of January). This modification is not expected to results in noticeable changes for most end-users but is part of a long term development toward VISTA data processing. DWE reported on persistence. A quick look indicates that any pixel that gets saturated will have a persistence effect. This can be up to about 100 counts and decays over about 60 seconds. It is unclear if there is a filter dependency. The difficulty is in identifying saturated pixels since they can have any value eg. saturated stars look like doughnuts. MJI said that using the catalogue data would help here. It is necessary to decide what to do about persistence before the reprocessing of 05A starts (in late February). DWE was asked to provide some further characterisation information. <<<< There was some discussion regarding how much of the data was checked by eye by the pipeliners (MR,RWA,MJI,JRL). MJI said in general that a small fraction of supposedly good frames were randomly selected and visually checked, though how many depended on which pipeliner. All problem frames (eg. bad astrometry) were inspected. JRL said that most problems with processed data were caused by problems with the sky subtraction part. MJI noted (yet again) that around 2 million 2kx2k science frames have been processed through the pipeline and oddly enough most have not been visually inspected. RGM said that for the EDR UKIDSS data from 05A will be released. This will not include the SV data. The exception from this is that the 05B September data for DXS and UDS would also be released. Additionally there are some data quality checks further restricting this selection. Authorship of the EDR paper being drawn up by Simon Dye was discussed. STH and MJI had contributed a couple of paragraphs each. MJI drew up a list of CASU personnel who will be co-authors on the paper and will forward to Simon Dye. <<<< DWE reported on the PSF work. The split PSF measurer is working well and a baseline PSF fitting programme using this has been written. This latter programme is not in an ideal condition and more work is still needed on this to approach optimum result stage. Adding PSF measures to catalogue data is delayed until this software meets these criteria. MJI said that the BBC had visited WFCAM on the 12 January and some frames for publicity had been taken. This data will be on the next batch of tapes to arrive. JRL said that he would be processing this data (needed by 2nd February?). <<<< There was some discussion about whether there was a need for a JAC telecon. This does not seem urgent given that WFCAM will come off the telescope shortly and would be better deferred until later. RGM asked if there was any news from the UKIRT board meeting. MJI said that he would try and find out. <<<< STH said that he, Paul Hewett, PSB and JPE had a meeting with Martin Cohen to discuss absolute photometric calibration. He has a network of absolute standards (down to 14 magnitude) which could be used for the Y and Z calibration. A misunderstanding regarding the use of 2MASS for WFCAM calibration was clarified and he now agrees that the calibration can be done to the 1-2% level. 5. Upcoming VDMT meeting and this quarter WPs STH said that an extra work package has been added to account for the extra work on the VISTA ETC. He reminded us that he wanted the monthly reports (covering December and January) in by the end of today so that MJI could review the results on Friday. <<<< The work for the UK Review will come in during this quarter, so some clarification regarding the review will be necessary from the VDMT meeting. 6. Data archives update The INT data has now been all loaded by RWA. He and EGS are sorting out the organizational changes that are required. The data was ordered by tape, CCD, DVD number and needs rearranging to date order to match the new internet transfer storage and access. MR reported that the raw WFCAM archive was up to date. No news is good? news regarding the rest of the archives. 7. Optical processing update Most of the optical processing that is being carried out by the group at the moment is on IPHAS data. STH said that we will be processing some 200Gb of INT data for an Oxford group. This data was taken between July 2004 and January 2006. RGM said that he and a student are using DR4 and DR5 SDSS data to calibrate INT data taken under non-photometric conditions and deriving relevant colour transformation equations. 8. AstroGrid liaison update RGM said that the EU have awarded money for an extra person to work on AstroGrid in Cambridge for 2 years. 9. AOB NAM - This will be in Leicester (Monday 3 to Friday 7 April) and it was felt that a CASU presence was required to highlight the quality of the pipeline processing and show off some example diagnostic and survey progress tracking web pages. RGM said that he would look further into this. <<<< ADASS - JRL said that the venue has been booked for the week starting 23 September 2007. Di Sword will be helping organize the conference dinner and the reception. Science with WFCAM data was discussed. MJI was asked to clarify with Andy Lawrence (CC Andy Adamson) CASU direct data access rights for doing science with the processed data. <<<< Continuing Actions ------------------ JRL provide a script to add JAC MSB quality flag to the FITS headers MJI finish investigating merged classification statistics WJS find out what is happening in AIT3a and see if a CASU visit is feasible PSB suggest a suitable strategy for jittering VISTA twilight flats JRL MJI JRL finish mods to AAT archive and sort out ingesting more AAT data if more needs ingesting JRL find out if there is any more UKIRT (non-WFCAM) data that needs ingesting New Actions ----------- MR make the science plots of IC4665 and M17 and the LAS tile visible from the Plone reduction status web pages STH raise various UK Review issues at VDMT meeting DWE provide some further results on persistence MJI forward EDR paper CASU coauthor list to Simon Dye JRL process the publicity frames for the BBC (by 2 February if possible) MJI acquire news from the UKIRT board meeting ALL send in 2 months' worth of reports to STH by end of today (26th) RGM investigate possible CASU presence at NAM 2006 MJI clarify with Andy Lawrence (CC Andy Adamson) CASU direct access data rights on the processed data