Wednesday 23rd September 2009 APM Meeting tables 11:40-12:50 Present: MJI, RWA, EGS, RGM, JRL (via email) Apologies: DWE, MR, SC, STH, NAW Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Recent and upcoming meetings 4. Data archives update - AAT, ING, WFCAM, UKIRT 5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST ..... 6. WFCAM update 7. VISTA update 8. AOB - grant status etc.. Minutes ------- 1. Actions from the last meeting MJI stage I software release on hold due to other pressures - ongoing <<<< EGS MJI ditto technical paper - ongoing <<<< MJI ditto utility to convert binary tables plus corrections - ongoing <<<< STH guide to whereabouts of on-line data particularly for ING archive has been shelved since all you have to do is ask the right person STH have revisited the linearity, zero-points and colour equations for VISTA MJI and the information is now on the public access technical pages on the VISTA website MJI asked but did not receive perls of wisdom regarding the occasional w/e hiccups in the WFCAM raw data transfers 2. Comments on WFAU minutes JRL noted that Eckhard contacted him regarding some of the keywords in the VISTA data products and their meanings relative to WFCAM. This is still in a state of flux since there are likely to be quite a few changes in the next few weeks as parts of the Cambridge science pipeline are rewritten (see below). MJI took the opportunity when attending the UKIRT workshop to discuss assorted queries arising in recent minutes regarding: occasional real nos. in areal profile; getting the Petrosian fix out of the door; relating the saga about why VISTA fits headers are still changing; updating them on stacking routines; and double checking that 09A is not going to be in DR6. He also swapped stories on the grant sagas. 3. Recent and upcoming meetings The VISTA IOT on September 1st was fascinating as always and the minutes are in the usual place, if anyone is interested. Therein lies some useful information about schedules which in addition to a copy of the recent letter to public survey PIs should satiate sentient souls. MJI and STH attended the UKIRT workshop, September 14th - 16th, to celebrate 30 years of astronomy with UKIRT. MJI spoke on Local Group galaxy properties in the NIR and STH on results from the WTS. Apart from these two talks the workshop managed to be both quaint and scientifically interesting, though not always at the same time. Gary Davies finished off the meeting by discussing the future of UKIRT. Suffice to say that completing as much as possible of the UKIDSS surveys by mid-2012 is a good idea. In addition to all the other ongoing reviews, NAW and MJI on behalf of CASU filled in the tedious STFC proforma as input for the "STFC Prioritisation of your project/area". The proforma was split into five sections representing the main priority areas of STFC: Excellence, Economic Impact, Societal Impact, Leadership and Synergies etc... you can imagine what fun filling this in was. MJI also contributed to a similar proforma for VISTA that JPE coordinated. JRL is off to ADASS in a week's time but promised to return before VISTA science verification data sputters in. He also promised to get off his butt and write his ADASS talk, back permitting <<<< 4. Data archives AAT -- JRL reports that all is smooth apart from one hiccup with a missing file needed for the web site, got a copy from Chris Ramage and all is well. UKIRT -- almost all the original sdf data are now here. About 20 nights of data are still missing and Andy Adamson is investigating these. Conversion to FITS and MEF'ing has started. Once complete the whole archive will be backed up on to tape, encore une fois. ING and WFCAM archives trucking along nicely. WFCAM and UKIRT are getting some TLC from September 15th for two and a half weeks so we can safely predict the WFCAM archive is future proof for at least a week. Due to the recent meltdown of apm2 and its associated ING data transfer disk there was a slight hiatus in ING archive transfers. But normal service was resumed quickly and a replacement transfer system put in place. 5. Optical/NIR processing : HAWKI, INT WFC, MegaCam, Subaru, VST ..... EGS is processing some archival CFHT MegaCam data in the Spitzer FLS and also investigating processing SERVS Subaru data as part of a long term research project. As this includes both new and old SuprimeCam data MJI commented that the current Hamamatsu detectors have a somewhat different (vastly improved) QE performance, particularly in the red passbands and that may lead to some interesting effects, e.g. photometric redshift estimation. MJI noted yet another time series team was at it, The Palomar Transient Factory and tasked STH (in absentia) to find out a bit more about it and report back at the next meeting. <<<< EGS is calibrating previous INT WFC data of 2XMM fields using SDSS DR7. A re-release of the optical data through CASU is planned. JRL in his copious spare time fixed a problem with his processed HawkI SV data in sparsely populated high latitude fields (not enough 2MASS stars per detector to generate a reliable solution). MJI noted he needs to use the ZPN distortion model derived from other SV data in Carina if he plans to accurately mosaic it. 6. WFCAM update. Semester 09A is all signed sealed and delivered. 09B processing is now finished up to WFCAM standdown (from Sep 15th viz warm-up for basic maintenance, pump to flush the cryo-system, etc... ) and just requires the usual monthly photometric calibration udpate. RGM pointed out a possible problem with the survey progress web page being out of date. EGS said he would look into this <<<< Re-reprocessing trundles along with a yet another visit to the good old Petrosian Identity after NJC up at WFAU spotted a small gotcha in the supplied fix code. This is finally finally all completed at this end. Still next in the queue are the DR6 GPS nebulosity enhanced (re)processing request to be followed by UDS J and H-band data from 05B-07B. MJI is in communication with Watson Varricatt regarding the low light level non-linearity tests and also a revisit of the normal non-linearity tests. Apparently low light level tests have been done and the data delivered to Derek Ives for analysis. Watson and MJI will also be comparing dome flats with twilight sky flats. <<<< After Alastair Edge's visit and a bit of email discussion, a couple of 08A DXS datasets were select to test the pawsky method against the previous standard tilesky processing. Conclusions so far are that using pawsky on this data makes little difference to the visual appearance of the pipeline produced skies/science frames nor to statistical measures of pixel noise properties. After stacking one deep DSX K-band set using both methods conclusions are: there is very little depth difference between using pawsky and tilesky methods on DXS data. The only significant difference is a slightly blotchier background appearance (which you can optionally process out using e.g. the nebulosity filter) and which has no discernible effect on the local rms noise and hence limiting depth attainable; the individual MSB outputs and the deep DXS stacks are very close to photon noise limited as judged both from direct rms pixel noise measurements and from randomly placing 2 arcsec diameter apertures over the frames and comparing noise in aperture with that expected from rms pixel noise - they are the same, in apparent contrast to results for the deeper UDS stacks. MJI noted that he had submitted a couple of articles for the upcoming UKIRT Newsletter viz:- "Dribbling" (or correcting the PSF mismatches that occur in interleaved data) "Saturated WFCAM stars" (or extending the dynamic range of WFCAM data) 7. VISTA update At present VISTA is in stand-down mode which will also include re-aluminizing. JPE is going out for an extended period which will include telescope re-alignment, checking, and taking some data for publicity purposes prior to SV starting mid-October. Quite how the publicity images taken to be ready for handover and the SV data itself will get back to Cambridge faster than normal is still a moot point. We have had no new data since 20090819. All nights with on-sky data have been fully reduced and placed on line for the brave and privileged of the project to peruse. JRL noticed that some data in July suffered from shadowing due to the filter wheel being out of alignment and suggested that re-datuming, together with recording the drift, should be done as part of the daytime checks. He also spotted a faux pas in the header information in some unusual (but allowed) observing modes and liaised with Steven Beard to get it fixed. There is a singular lack of data which could be used for PR exercises. This is due in part to the problems with the filter wheel sticking in earlier times, which made it impossible to get good background correction which these types of images depend upon. There is also currently no 'official' way of taking offset sky observations for extended objects. Fortunately on 20090809 a large amount of observations were done near the galactic centre which do not require offset sky observations. These have provided some stunning colour images. MJI and JRL are liaising with Richard Hook and the ESO PR team to see if these and other data on M55 are good enough to form part of the PR blurb. With the science verification runs looming large and with one of the SV projects being observations of M42 it now looks as though we will get a template to do offset sky observations. The summit/ESO pipelines have been modified to accept these data and do the right stuff. Version 0.9.6 of the ESO pipeline has been submitted to JPE for distribution to ESO. No further news on the official ESO VISTA ETC, but as noted in earlier minutes, the CASU version (1.4) is up to date with the latest info and has even been known to give plausible answers. The science pipeline still has a small way to go. Originally it was to be based partly on what we have written for ESO (for reasons of ease of maintenance). However it has become clear that once the pipeline software is delivered ESO plan to 'modify' the software to a point where APIs may be changed. Since we had planned to share the same code base this would have a knock-on effect of possibly breaking code that we prepare here for the Cambrdige science pipeline. So the decision has been made to have a science pipeline completely outside of CPL. This has meant a lot of extra work, but it will guarantee local control over our pipeline software. This will inevitiably mean that the extra data product headers produced in the pipeline will be a little different from what has already been delivered to WFAU. The good news is that this aspect will look more like WFCAM data. Indulgence is requested from our friends up north...... In addition to this hiatus, assorted different tiling strategies are still being tested since none of the bespoke solutions, including our own, are currently good enough. When that lot gets sorted the current wish-list of jobs for upgrading the VISTA cataloguing software still awaits viz: average confidence in aper3 encoded in #55; half-light radius analysis to replace Hall measures; elliptical apertures for all galaxian measures; PSF analysis; Sersic fits. All we need are people in post to do it. RGM suggested a slightly clearer access route to the VISTA web pages from the top level plone entry point would not go amiss. Something along the lines of the WFCAM direct access would suffice. EGS said he would investigate and make it so. <<<< The location of VISTA data has now been completely rationalized. The raw data is stored on apm35_a and the reduced data is stored on apm29_a. All the other pooled storage partitions on apm29 have been nuked. MJI noted that the technical performance figures sent out by Magda Arnaboldi, together with a copy of the letter to survey PIs, are on the internal web pages. RWA suggested that he include the performance figures for VISTA on the technical webpages. <<<< 8. AOB MJI relayed the latest news in the grant saga wherein one rolling grant, submitted two and half years ago, has now been finally issued as one slimmed down rolling grant together with a so-called VISTA exploitation grant to cover the slimming costs. We still await news, more in hope than expectation, of the research element of the rolling grant which has been resubmitted as part of the current grant round. However, we are now, finally in a position where we can begin recruitment to fill the vacant VISTA position. The extra rack, switched power supplies and the remaining CPU/disk storage units for Gaia have been installed, as have the two extra 10kW AC units - our contribution to global warming proceeds apace. Prior to this several of the CD/DVD cabinets were removed to another part of the IoA to make enough space. Still no sign of the high temperature monitor. SC will chase up as its a bit late when the alternative smoke detector goes off <<<< The raid array on apm2 finally expired permanently and has been nailed to its perch together with its host. This kick-started a slight reshuffle with the windows machine apm1 being rejigged as a backup clone of apm3 and another quieter PC becoming the windows jobbo. Continuing actions ------------------ EGS add catalogue software and binary table conversion software to software MJI release pages MJI finish off technical paper updates MJI write utility to convert binary tables with all corrections New actions ----------- JRL debutt and write ADASS talk, back permitting EGS check out and update the WFCAM survey progress web pages MJI compare WFCAM dome flats with twilight sky flats STH regale us with the PTF saga at next meeting EGS make top level VISTA pages similar access to WFCAM pages on plone RWA put performance info on VISTA web pages SC progress the high temperature monitor installation