WFCAM SCIENCE ARCHIVE: UPDATE ON PROGRESS FOR UKIRT BOARD, MAY 2008 Surveys: At the turn of the year, UKIDSS DR3 was released. Because of the volume of the GPS, the release was again split into two parts: in early December, the phase-1 release excluding the GPS was made; after some delays caused by the Christmas break and various technical problems, the final phase-2 release including the GPS took place at the end of January. Details of all UKIDSS releases are available in the usual place, at http://surveys.roe.ac.uk/wsa/theSurveys.html while features and bugs are noted at http://surveys.roe.ac.uk/wsa/releasehistory.html At the time of writing, preparations for UKIDSS DR4 are underway, and release is anticipated some time in June. DR4 will incorporate all UKIDSS observations from 05A to 07B inclusive. Enhancements will include further low-level photometric calibration refinements (the so-called "illumination correction" provided by CASU), as well as increased area in the wide-shallow surveys (LAS/GPS/GCS) and increased area/depth in the DXS. Furthermore, enhanced "synoptic" database tables for the time-resolved DXS intermediate stack catalogues will be included to open up the time domain in the infrared survey datasets, and as a precursor to those Vista public surveys that have a variability component. The UDS will remain the same as in DR3. PI programmes: The status of non-survey data in the archive is as follows (see, for example, http://surveys.roe.ac.uk/wsa/nonSurveyStatus.html): PI programmes Flat-file DBs registered access? released Semester 05A: 9 All 9 " 05B: 17 All 16 " 06A: 15 All 12 " 06B: 9 All 0 " 07A: 6 All 0 " 07B: 12 All 0 " 08A: 6 no 0 Service: 30 All 11 As noted several times previously, most non-survey users seem to be content with flat-file access to image and catalogue FITS file products as opposed to requiring "survey-like" SQL database releases. WFAU will, however, continue to produce those database releases on a best-efforts basis. Archive usage: registration To date, there are 748 users registered for survey data access in the WSA. Those users are distributed amongst 82 (mainly institution-based) communities. Private PI programme users currently stand at 101. (Figures from the last update to the Board: 671 users amongst 75 communities; 69 PI programme users). Archive usage: survey data Survey data usage remains at a high level: Database Queries Rows exported ----------------------------------------------- UKIDSSEDRPLUS 1853 162001238 UKIDSSDR1PLUS 7340 1323937110 UKIDSSDR2PLUS 78327 1121352541 UKIDSSDR3PLUS 516437 1525244854 ----------------------------------------------- (note: all archive usage statistics exclude Edinburgh activity; the figure for DR3+ is dominated by one user who is presumably using a script robot to launch queries at the interface; in this respect, these statistics are something of a blunt instrument but nonetheless they do represent genuine usage of the resource). The UKIDSS DR1 went world-public in January, and it is interesting to note that the number of SQL queries submitted by users not logged in (i.e. "world" access unregistered users) currently stand at 382 for the EDR and 442 for DR1. Average monthly total data export rates for flat-file access from the WSA have risen from 46 GB/month in 2006, through 65 GB/month in 2007 to currently 125 GB/month in 2008. The total amount of catalogue data exported from the SQL database releases is currently >0.4TB. General WSA development Most of the VDFS development work done over the last quarter or so has consisted of scaling the archive system in preparation for Vista. Implementation of wholesale recalibration at the low level now possible given the number of WFCAM frames observed and analysed at CASU has challenged the original relational design, and has required some modification to enable a rapid implementation. Much of the scaling and refactoring of curation code has been done to avoid having to phase large survey database releases (i.e. later releases of the UKIDSS GPS). Enhancements in the hardware set-up and some mirroring of critical database objects has enabled us to further improve archive uptime (currently standing at >95%). The refereed journal paper describing the archive development and design has been published in MNRAS: http://adsabs.harvard.edu/abs/2008MNRAS.384..637H For much more information, please consult the WFCAM Science Archive home page at http://surveys.roe.ac.uk/wsa/index.html Resources: In early June 2007, WFAU submitted a Rolling Grant proposal for resources including VDFS operations for the period Q2 2008 to Q1 2013 (where VDFS operations includes continued operation of the WFCAM Science Archive). The application was refereed and formally reviewed in January this year, with generally positive feedback from the review panel. WFAU recently received a "low priority" rating in the STFC Programmatic Review, with a qualification "excluding ESO committments". Communication with STFC on the exact meaning of the latter phrase has indicated that it will be interpreted quite broadly, but nothing can be said with any certainty concerning the likely level of operational resources finally awarded for the next few years.