From sth@ast.cam.ac.uk Mon May 2 09:28:09 2011 Date: Fri, 29 Apr 2011 13:18:27 +0100 From: Simon Hodgkin To: Mike Irwin Subject: drAFTY Minutes CASU meeting Thursday April 28th 11:30-13:00 CASU meeting area. Present: MJI, AKY, EGS, JRL, STH, MR Apologies: NAW, SC, JPE, RGM, JCR Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. Optical/NIR processing 7. WFCAM update 8. VISTA update 9. AOB Minutes ======= 1. Actions from the last meeting -------------------------------- AKY - is still examining the UltraVista image from HJM, with particular emphasis on 1) comparison between artefacts in both images, and 2) comparison between the fluxes of objects. <<<< STH - has built a plone web page, but has not released it yet. Action finish it off and make the page public. <<<< AKY - showed us the results of her flatfield behaviour investigation. It looks like a monthly timescale is about right - we see generally linear trends with small changes from month to month, apart from occasional spikes in the median flatfield values. We also looked at the linearity on monthly timescales, and will have a think about whether to start using a rolling master linearity. AKY will write all this up and share it. <<<< MJI - put some info and a picture on the web, concerning the MJD variation across a VISTA tile. MJI - Added a footnote about tile QC info. Basically, it's a suggestion to look at pawprints. EGS suggested that the tile could contain more info in the headers about basic info such as seeing, ellipticity, skybrightness, magzpt of the component pawprints. This needs some more thought. <<<< MJI - The apm44 kernel got some TLC and it now communicates with the card and therefore the disk. AKY - MJI hasn't noticed the homespace filling up - therefore it's all good. JRL - Missing sempahore flags in various web pages have been found. EGS - inquired about the 8 thread uplink limit for ESO PHASE III delivery - with the positive result that the number has been increased to 16. MJI - an extra column was required and added for the VISTA illumination tables. 2. Comments on WFAU Minutes --------------------------- No comments 3. Hardware Update ------------------ The Supermicro JBOD expansion now has 3 new disk arrays (c,d,e) which hold roughly 70Tbytes in total (split as: 36,18,18). At some point we'll shuffle data around a bit - but they are currently empty. One will be for WFCAM, the other 2 for VISTA. The new fibre-channel raid works - but is not yet connected up. MJI noted that the monitoring pages have stopped working. And spotted that the gaia one also doesn't work at all. Various accusations were thrown around. ** Discussion about whether to put new link in these minutes **. We need to think about upgrading to Debian for all the blades at some point in the future (currently Fedora). It's not urgent, but then again - we do have a gap in the VISTA dataflow. 4. Meetings and Telecons ------------------------ The next VISTA IOT is not expected before middle of May. MJI Skyped into a VMC meeting and realised that we have just switched from ESO Cycle 86 to 87. So we need to get ready with period 87 web page (JRL will set the directories up for April). <<<< *** for edit: In-kind software contributions.. ESO+CASU have agreed. In a nutshell - we are going to provide a bunch of upgrades to their VISTA CPL pipeline (to mimic much of what we do here). We will also take on reprocessing for HAWKI and VIMOS. Workflow system - (reflux??) - we will get that delivered here and we will fix it and make it better (update workflow/elements). 5Tbytes Hawki and 1Tbyte VIMOS. We get some money. Need to put in a new grant bid, (1st April 2012) - currently negociating when it will go in and what it will include. ** MJI attended NAM and talked about M31. He found that Llandudno was sunny and pretty, and that science is fun. 5. Data archives update ----------------------- Smashing 6. Optical/NIR processing ------------------------- VLT first light was March 26th, but we've heard nothing since. In principle the surveys should start in August. 7. WFCAM update --------------- All is going fairly well. The issue with incomplete headers is now under control. An extra check in the pipeline tests and fixes them (though only the WCS). Another logfile is being produced which says which headers have been fiddled with - plus some history keywords are added. Our suspicion is that this get's FU'd during the SDF to FITS conversion - and would be better trapped at source. All routine processing is up to date as well as a Bank Holiday special request. Some of the usual suspects are also asking for early access to the data (SJW, NRT). A couple of issues have arisen, particularly concerning photometry in crowded fields: 1. Aperture corrections are a bit ... ** not sure exactly what the problem is - well i can imagine it - but not sure if you are implementing something here to make it more robust.. 2. The saturation magnitude for 2MASS is rather fainter than in uncrowded regions, therefore our calibration star selection criteria needs a minor tweak. MJI has implemented this in his code. 3. Occasionally, in crowded fields, the astrometry is not convergeing to the final solution. i.e. sufficient mismatches can exist to give a false solution. MJI suggests that we add a new header keyword which is the expected number of 2MASS matches to compare with NUMCRMS. In fact we will almost certainly need to propogate this to VISTA too. <<<< 3. Cont.. Additionally, MJI has already implemented some small changes to more precisely define the VISTA + WFCAM footprints in the calibration routines, and thus give better matching of standard stars to the fields. Phil Lucas will send list of WFCAM files which need the astrometry updating. We will also look for frames with significant deviations in the number of crumbs,and see how often it's occurring, before deciding how to tackle it. <<<< UKIRT board meeting is a month away - and the report is due in tomorrow (Friday). Mike will make it so. <<<< 8.VISTA update -------------- Everything has now been processed to Version 1.1, and everything is up to date. The current plan from ESO is that VISTA goes on sky on May 4th. A couple of nights of testing will then follow, with WJS on hand to help. EGS updated us on the saga of the Phase 3 deliverables. After various to-ings and fro-ings between different servers, we now have a link delivering data via ftp to ESO at around 1-2 Mbytes/sec per thread (with 16 threads). We have now transferred Video, VVV and VHS (up until the end of September). For VHS this included catalogues and images for tiles and pawprints. The other surveys deliver only tile products. VMC and Viking are bing done by Edinburgh, while Ultravista are doing their own thing. Our understanding is that the PI's need to write a report and release notes prior to the OPC meeting which will be in May-ish. EGS commented that the Phase-III deliverables we transferred comprise around 1.5 Tbytes per survey (compressed). MJI asked if anyone has tried the VISTA ipad app from ESO ? Does anyone have an ipad ? Really ? EGS spotted a funny thing which he then forgot about - the action is to remember. <<<< We discussed the VIDEO photometry issue, which is really a pawsky problem. Recall that this strategy takes a single pawprint - and builds an object mask (from the dithers). Because there are relatively few dithers, then faint images below the detection threshold leave residual flux in the sky .. which starts to show in the deep stacked images. Amounting to as much as a 0.14 mag offset. One solution is not to include the current frame in the sky (but this is rather processing intensive). Another is to have a deep object mask, built from a deeper catalogue. Option 3 is to use more frames that are close enough in time to build the sky and thus minimize the effect. Our plan is to combine both options 2+3. The expectation is that we should see a similar effect in the UDS (if it ever gets hit by a deeper survey). This is already being done for ultravista. We discussed the wishlist for Version 1.2 upgrades, which include: + pawsky masks. + correct footprints for standard matching + Magzpt vs NightZPt .. should only be computed from the pawprints and inherited. + We want the description of the colour equations in the header. 9. CASUHELPSs ------------- A couple of thought provoking issues came to light: 1. Should we put some numbers in for the A_K vs E(B-v) values etc. and so on in the headers ? Our feeling is not to start telling people which extinction law they should use. 2. We need to revisit the question of what to put in the headers for the coordinate system (FK5,ICRS etc). <<<< Problem with GAIA overlaying catalogues. Apparently it depends which version of GAIA you use. Problem with compiling the software on DARWIN (OSX) has not gone away. Continuing Actions: ------------------- AKY - compare Henry's deep stack with our equivalent STH - put 2MASS JHKs -> i-band predictor information on web pages New Actions: ------------ AKY - write report on the flatfield investigations MJI - think about QC provenance in tiles All JRL - setup VISTA Cycle87 directories MJI - new header keyword to record the number of 2MASS standards in the detector footprint EGS, MR - work out how often the WCS mismatch problem seems to have occured for VISTA + WFCAM EGS - remember what he wanted to say JRL, MJI - check our coordinate conventions - are we happy ? [ Part 2: "Attached Text" ] ---------------------------------------------------------------------------------------------- Simon Hodgkin Cambridge Astronomical Survey Unit, Tel: +44 (0)1223 766657 Institute of Astronomy, Madingley Road Fax: +44 (0)1223 337523 Cambridge, CB3 0HA. UK ----------------------------------------------------------------------------------------------