26th April 2007 Hoyle Committee Room 11:00 - 13:25 Present: DWE, STH, JRL, PSB, MR, MJI, EGS, NAW Apologies: RGM Guest appearance: Robert Greimel (RS) Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes - UKLight saga update 3. NAM news 4. Report from Second Generation VLT Instruments meeting 5. Data archives update - AAT, ING, WFCAM, UKIRT 6. Optical processing update - IPHAS release 7. WFCAM news - raw data status - processing status - photometric calibration update 8. VISTA news - pipeline progress 9. ADASS 2007 - latest 10. AOB - upcoming VDUC meeting - cass38 and cass36 - rolling grant application Minutes ------- 1. Actions from last meeting JRL has not repaired the AAT archive mirroring (PostgreSQL problems), however some progress has been made since Stuart Rider has been in touch with us. Since this is issue seems very much out of our hands, it has been decided to drop this action. MJI did not email Matthew Colless regarding the above problem. This action has now been superceded by the above. RGM has not reported back on the classification aspect of cross-talk weeding. MJI said that it should be very easy to spot misclassifications since stars and galaxies have very different colours in the infrared. EGS and MJI should now be in on this action - ongoing <<<< EGS prepared the ADASS 2007 preliminary announcement web page - MR http://www.adass.org:8080/Conferences/2007/Venue/ MR continues to investigate credit card payments for ADASS 2007 - ongoing <<<< PSB is waiting for a quote for wireless provision for ADASS 2007 from a local (Cambridge area) person. WJS has not added field rotation flat field tests to the VISTA integration tests - ongoing <<<< MJI noted that WJS has now officially left the IoA as of 31 March. He will raise this issue at the VDMT meeting tomorrow. <<<< JRL has not found out more about the previous providers of wireless services at Kensington Town Hall. This action has been superceded by a previous action. RGM has not emailed Tim Naylor, JPE and WJS to ensure that discussing the updated UK User Requirements Document is part of the VDUC agenda - ongoing <<<< MJI said that he would also raise this at the VDMT meeting. <<<< The VDUC meeting will be held in Edinburgh on 10th May. STH said that there will be a UKIDSS Survey Heads meeting in Oxford on 9th May. There will also be a calibration meeting tagged onto this (before or after). NAW said that he would email Tim Naylor to obtain the agenda for the VDUC meeting and the minutes from the previous meeting. <<<< Many said that this would be useful since they could see a CASU presence would be needed. MR has not contacted the Durham group about raw WFCAM data processing progress. It was intended that this not be done until the end of April - ongoing <<<< MJI distributed drafts of conference proceedings of ESO calibration JRL workshop MR ALL commented on above drafts MJI will ask about obtaining more VISTA data at next VDMT meeting - ongoing <<<< 2. Comments on WFAU minutes PSB reported that the UKLight connection is "up and running" between Edinburgh and Cambridge. 5 problems still had to be sorted out from the original announcement and there is still no SSH connectivity. Ping works ok, but physical tests have shown that the signal is at the low end of the spec. Many of the delays stem from the lack of access to the root password at WFAU. NAW suggested that our Internet costs should be billed to Edinburgh. This might provide the right incentive. There then followed a discussion on UKLight links to ESO and ESAC. Getting a link to ESO would probably be too complex. NAW and PSB agreed to discuss the ESAC link offline to see how they might progress this. <<<< STH noted that WSA had installed a new toolkit and asked if this included list driven photometry. MJI said that it did not. It was also noted that DR2 for GPS had now been released. 3. NAM news EGS, RS and NAW attended NAM last week. EGS reported that Michael Rowan-Robinson had said that photometric redshifts don't work for the WFCAM filters in the DXS, but that it did for 2MASS data. This was regarded as odd since WFCAM photometry is calibrated from 2MASS and that the colour terms are small. EGS said that he will look into this. <<<< Also mentioned at the meeting was that the funding priorities for STFC (nee PPARC) will be for science-based projects that are UK-led. 4. Report from Second Generation VLT Instruments meeting JRL and PSB reported on the 2nd Generation Instrumentation Pipelines meeting held at the Max-Planck-Institut für extraterrestrische Physikon 18 April (http://www.mpe.mpg.de/ir/meetings/vlt2gi.php). Many instrument groups were present and they almost outnumbered the ESO personnel present. Many CPL issues were discussed. One of the main criticisms of CPL is that you can't update a file - you have to completely rewrite the file. QFITS will be replaced by CFITSIO in version 4.0 of CPL. This will not make much difference to our software since QFITS underlies CPL. The welcome exception is that compressed FITS files will become available. NAW mentioned the Reflex system which is a modular system (unlike CPL). This is only funded until November. (not sure about this Reflex stuff) 5. Data archives update AAT: JRL (or MJI?) said that we have recently received some more data. Note that this archive is now offline due to the problems with mirroring the AAT end of the data base (which contains the headers). UKIRT: JRL said that more data has just been received and that it is in the process of being ingested. ING: MJI reported that data is being received over the Internet. RWA has gone to La Palma to try reading in the last 300 tapes that are causing problems. WFCAM: MR said that all the raw data (all chips) has been read in up to the end of 5 April except for 6 nights in January. JRL mentioned that there is still a group keyword problem with some of the data. PSB is carrying out FTP tests with Brad at JAC and obtaining 64kBits/s. This seems to be a software limit, since multiple FTP sessions can be set up, each getting the same data rate. It is likely that a solution can be found to increase this. Even with an increase to the "wire limit", this will be much slower than UKLight and would only be useful for occasional transfers eg. missing files. PSB was asked to email a copy of his report to Brad (CC'd to Andy Adamson). <<<< ESO transfers have been carried out up to 24th March. JRL said that there was a press release about the Kognitio data base system that he has tested with 2MASS data. This system uses multiple processors and doesn't require indexing and it thus very fast. They are keen to carry out some tests with the IPHAS data. This is definitely worth pursuing. 6. Optical processing update - IPHAS release EGS reported that the data from 2003 to 2005 has been ingested and that the aim is to have a public release on 1 June. The documentation for this is being written by EGS, NAW and RS and that a paper will be placed on astroph soon. NAW said that there should also be a press release. There was then some discussion about how the Kognitio system would cope with a merged data base of IPHAS, 2MASS and GPS(?). JRL said that he would get in touch with Kognitio to see if they would be interested in doing some further tests. <<<< 7. WFCAM news MJI said he is starting to think about getting some more disk space. There was also some discussion about what to do with the spare 750 Gb disks. EGS said that he could use these as backups for the IPHAS data base. <<<< MR said that, ignoring data before 20 December, only 2 more nights of 06B (up to 31 March) data remains to be processed. MJI made a comment about twilight spacing. MR noted that there were many incomplete groups. There was a need to put night logs into the appropriate directories. (I didn't follow the above 3 comments - they are as they appear in my notes) STH gave us an update on the photometric calibration. The Y band has been done to 1%, but the Z band has not been done yet. When asked about the consequences of not using SDSS selection, he said that the same numbers were obtained. A meeting in early May is planned. MJI said that there will be a UKIRT board meeting in late May and that it would be good to have an updated web page containing the relevant information. <<<< (not sure if you intended an action on this) STH said that investigations have been carried out into the Eddington bias. This is apparent when you compare deeply stacked data with shallow surveys. Since we use a 10 sigma detection threshold, the bias is only 1%. More work needs to be done on this. MR was congratulated for his article in the UKIRT newsletter. EGS asked if the WFCAM photometry has been corrected to allow for the astrometric distortion? MJI said that this has not been applied to the data in the catalogues since it has been agreed that the fluxes should just be the sum of the relevant pixel contents. There is enough information in the headers to work out the correction. MJI said that he would email NCH about this. <<<< 8. VISTA news JRL said that he had received comments back from Sandra Castro regarding the 0.5 release. Path files stuff discussed (I have no idea what this was about - hence no notes) There was some discussion about whether there would be a final list of remaining issues to be sorted out. This was thought to be unlikely. PSB raised the issue of what is meant by the exposure time value contained within the FITS headers. MJI said that he would also raise this issue at the VDMT. <<<< It was noted that PSB has written a web page explaining how to use WCS within FITS tables as opposed to FITS images. http://casu.ast.cam.ac.uk/documents/vista/astrometry/tables-wcs/view It was noted that from 07B, catalogues will have this format within their headers. MJI said that a proposal is being written by ESO to improve the data link between ? and ? in Chile. JPE is currently at ESO discussing this. There was no news regarding hardware procurement for the summit. NAW mentioned that STFC (PPARC/SERC/SRC) funding of data centres is being reviewed. It would be useful if we had some numbers for CASU. It was not clear how this could be input. NAW said that he would look into this. <<<< 9. ADASS 2007 JRL said that 3 proposals for tutorials have been submitted (Data mining, 3d visualization of data, Relational data bases). Thus, there will be a tutorial. Should there be a charge for this? The answer was yes. This has been done in the past and that this would encourage people who have signed up to turn up. The next milestone is 1st May, when the registration page needs to be ready. 10. AOB MJI said that cass38 has now been retired properly and that cass36 was next on the list. There was some discussion as to what functionality from cass36 needs to be preserved. MJI said that the rolling grant application will be submitted on 20 May and that input was required for the research and archive activities. <<<< Office space was discussed. PSB has to transfer to JRL's office by September. It was seen as useful for this to happen earlier since the process of clearing out the SPO will probably occupy PSB quite a bit at that time. Further discussion was had concerning moving the cabinets out of the storage room to make room for the computers. This should make the central area a lot quieter and a more pleasant working environment (where visitors can be dumped). Continuing Actions ------------------ RGM report back on classification aspect of cross-talk weeding EGS MJI MR sort out how ADASS 2007 credit card payments are to be made WJS consider adding field rotation flat field tests to integration tests RGM email Tim Naylor, JPE and WJS to ensure that discussing the updated UK User Requirements Document is part of the VDUC agenda MR contact Durham group about raw WFCAM data progress (~end of April) MJI ask about obtaining more VISTA data at next VDMT meeting New Actions ----------- MJI raise issue at VDMT meeting of adding field rotation flat field tests to VISTA integration tests MJI raise issue at VDMT meeting that an updated UK User Requirements Document should be part of VDUC agenda NAW email Tim Naylor to obtain agenda for VDUC meeting and minutes from previous meeting NAW investigate possibility of UKLight link to ESAC PSB EGS investigate photometric redshifts in DXS PSB send FTP tests report to Brad (CC'd to Andy Adamson) JRL contact Kognitio regarding further tests involving merged data base containing IPHAS, 2MASS and GPS(?) EGS sort out use of spare 750 Gb disks as backups for the IPHAS data base MJI STH prepare an web page on the recent photometric calibration developments in time for the UKIRT board meeting (late May) MJI email NCH to remind him about the photometric corrections needed to account for the astrometric distortion MJI raise issue at VDMT meeting of what is meant by exposure time value contained within VISTA FITS headers NAW find out how to provide input to STFC review of data centre funding ALL provide input on research and archive activities for rolling grant application (by 20 May)