WFCAM SCIENCE ARCHIVE: UPDATE ON PROGRESS FOR UKIRT BOARD, NOV 2009 Surveys: The phase-1 UKIDSS DR6 release (consisting of the LAS, GCS and DXS, complete with observations up to and including Semester 08B) took place on October 19th. New features in UKIDSS DR6 include * The deblending error for extended source photometry described previously has now been fixed. * The LAS now has second-epoch observations in the J filter in some fields. The detections from both epochs are merged into the lasSource table using attributes J_1 and J_2. * The DXS has some new attributes in the dxsVariability table: jMinMag, jMaxMag, jNGt3sig, etc. which give the minimum and maximum magnitude of the individual observations of a source and the number of observations that are more than 3 standard deviations from the mean. These are calculated for each filter. In the dxsVarFrameSetInfo table, the attributes aStrat, bStrat and cStrat which gave the parameters for the best fit to the photometric noise as a function of magnitude have been replaced by aStratPht, bStratPht and cStratPht and the equivalent attributes for the astrometric noise (aStratAst, bStratAst, cStratAst) have been added (but astrometric fits are not yet calculated, so the astrometric noise attributes are set to defaults). * The DXS deep stacks have been processed with the correct smoothing function at the time of extraction, resulting in fewer false detections at the faint end. * Data from intermediate stacks with one or more deprecated extensions are now used in the stacking. These extensions are given a weighting of 0 during stacking. The totalExpTime attribute in MultiframeDetector is calculated on an extension by extension basis and takes into account missing exposure from these frames. Details of all UKIDSS releases are available in the usual place, at http://surveys.roe.ac.uk/wsa/theSurveys.html with features and bugs noted on the linked pages. PI programmes: The status of non-survey data in the archive is as follows (see, for example, http://surveys.roe.ac.uk/wsa/nonSurveyStatus.html): PI programmes Flat-file Survey-like registered access? DBs released Semester 05A: 9 All 9 " 05B: 17 All 16 " 06A: 15 All 14 " 06B: 10 All 3 " 07A: 7 All 0 " 07B: 12 All 9 " 08A: 13 All 4 " 08B: 9 All 0 " 09A: 6 All 0 " 09B: 8 4 0 Service 51 47 21 (includes programmes up to U/SERV/1836) Survey-like release database production for private programme PIs, an activity that was always done on a "best efforts" basis, is currently on hold while the WFAU team are shaking done the archive system with the recently arrived VISTA commissioning data. Archive usage: registration To date, there are 931 users registered for survey data access in the WSA. Those users are distributed amongst 95 (mainly institution-based) communities. Private PI programme users currently stand at 159. (Figures from the last update to the Board: 862 users amongst 94 communities; 139 PI programme users). Archive usage: survey data Survey data usage remains at a high level: Database Queries Rows exported ----------------------------------------------- UKIDSSEDRPLUS 1974 203721322 UKIDSSDR1PLUS 8844 1219572736 UKIDSSDR2PLUS 81078 1115735237 UKIDSSDR3PLUS 559281 1882868648 UKIDSSDR4PLUS 98143 1579604688 UKIDSSDR5PLUS 4730 742675640 UKIDSSDR6PLUS 5771 150468126 ----------------------------------------------- (note: all archive usage statistics exclude Edinburgh activity; the figure for DR3+ is dominated by one user who used a script robot to launch queries at the interface). The number of UKIDSS SQL queries submitted by users not logged in (i.e. "world" access unregistered users) currently stands at 568 for the EDR, 2059 for DR1, 2606 for DR2 and 3940 for DR3 (which went live to the whole world community on 5th June 2009). Average monthly total data export rates for flat-file access from the WSA have risen from 46 GB/month in 2006, through 65 GB/month in 2007, 133 GB/month in 2008 to 168 GB/month in 2009. The total amount of catalogue data exported from the SQL database releases is currently 0.97 TB. General WSA development A number of minor enhancements have been made to the archive user interface and curation codes in response to user requests over the last six months. Some of these are new features (e.g. handling new "filtered" images in regions of fluffy nebulosity) while some involved handling bug fixes for various points in the data flow system. As of the time of writing, all further WSA enhancements are on hold while the archive team shakes down the VISTA Science Archive system. For much more information, please consult the WFCAM Science Archive home page at http://surveys.roe.ac.uk/wsa/index.html Resources: As noted in the last three reports to the Board (May & Nov 2008, May 2009), in early June 2007, WFAU submitted a Rolling Grant proposal for resources including VDFS operations for the period Q2 2008 to Q1 2013 (where VDFS operations includes continued operation of the WFCAM Science Archive). We are very pleased to report that the situation has now been resolved, and given the present UK financial circumstances the outcome is good. An initial communication from STFC indicated likely severe cuts in the budgets for the survey units, but this was almost immediately followed by another communication following the identification of VISTA exploitation monies for the phase over from development to operations within VDFS. The WFAU Rolling Grant is now funded (adequately at least for the first half of the 5yr grant period) by three seperate grants for i) hardware, ii) VISTA exploitation and iii) a remaining core wide-field grant.