From nch@roe.ac.uk Fri Feb 23 16:32:24 2007
Date: Fri, 23 Feb 2007 13:37:06 +0000 (GMT)
From: Nigel Hambly <nch@roe.ac.uk>
To: WFCAM Science Archive Team -- Eckhard Sutorius <etws@roe.ac.uk>,
Johann Bryant <jb@roe.ac.uk>, Mike Read <mar@roe.ac.uk>,
Nigel Hambly <nch@roe.ac.uk>, Nicholas Cross <njc@roe.ac.uk>,
Bob Mann <rgm@roe.ac.uk>, Ross Collins <rsc@roe.ac.uk>
Cc: CCs for WSA weekly meeting minutes distribution -- Andrew Lawrence
<al@roe.ac.uk>, Andy Adamson <a.adamson@jach.hawaii.edu>,
Brian Walshe <bcw@roe.ac.uk>, John Taylor <jdt@roe.ac.uk>,
Jim Emerson <j.p.emerson@qmul.ac.uk>, Malcolm Stewart <jms@roe.ac.uk>,
Lorenzo Rimoldini <lgr@roe.ac.uk>, Mike Irwin <mike@ast.cam.ac.uk>,
Mark Holliman <msh@roe.ac.uk>, Peredur Williams <pmw@roe.ac.uk>,
Stephen Warren <s.j.warren@ic.ac.uk>
Subject: WFAU VDFS Science Archive weekly project meeting minutes, 23/02/07
Minutes of WFAU VDFS Science Archive meeting: 23rd February 2007
-------------------------------------------------------------------------
-------------------------------------------------------------------------
Present: NCH, PMW, ETWS, RSC, LGR, NJC, AL, MAR, MSH
Apologies: JPE, JMS, JDT, JB, BCW
DONM: 10am, Friday 2nd March 2007 in the Plate Library
Actions discharged this week:
-----------------------------
ACTION: PMW to bring along the Q1'07 progress charts next week
Discharged
ACTION: MAR to switch off auto updates to the current website,
Discharged
ACTION: NCH to review glossary entries for all new attributes and
procedures
Discharged with help from NJC
ACTION: MAR to create a error quality bit information page, and release
notes update, with links to/from the glossary
Discharged
ACTION: NJC to add a Cookbook section concerning sample selection
trade-off between completeness and reliability in the context
of error quality flag thresholds.
Discharged with help from RSC
ACTION: ETWS to finalise Browser modifications necessary for DR2
Discharged
Actions partly discharged but continuing:
-----------------------------------------
The following from last time partly done but continue:
ACTION: JB to rearrange external catalogues onto batch catalogue
server
Continues; all done save 2MASS, GLIMPSE and BestDR2.
ACTION: RSC to profile CU4 and investigate possible optimisation
Continues; on hold until parallelisation and other refactoring is
complete.
o) Add in a default row for every detector appearing in every detection
table (for schema consistency when querying merged sources and
individual detections)
- ACTION: RSC & NCH
Continues; NCH, RSC and MAR discussed this in the week and because
of some subtle complexities it will be put on hold until after DR2.
RSC has made some code changes to make the curation applications
robust against this very minor schema change when it does happen.
Actions carried forward from 16/02/07 meeting:
----------------------------------------------
The following from last time continue:
ACTION: NCH to discuss and review AstroWISE interfacing with MAR
and JDT.
Specific points and new actions:
--------------------------------
Project management:
PMW passed around the progress charts for filling in, and noted that
we will do the same next week for February progress (!); there
is a VDMT telecon provisionally scheduled for 9th March (apparently).
WFCAM & VISTA updates:
NCH noted that WFCAM comes off telescope mid-May, and is then back
on August to November inclusive.
Nothing new to report on Vista this week.
Comments and issues arising from CASU fortnightly minutes:
No new minutes this week.
Networking:
Nothing new to report this week.
WSA Operations:
Operations have consisted of running the release CU19 to create
UKIDSSDR2PLUS. One or two complications have meant that this
has yet to finish; completion this weekend is anticipated.
Hardware:
Nothing new to report this week.
Software:
MAR reported:
"Mainly updating webpages and survey plots for DR2 and creating script and
cgi to allow WFAU colleagues to initiate updates themselves as well as via
the cron job."
RSC reported:
"Investigated improving the performance of CU7 by removing redundant
declination sort on outgest as the Pairing code does all the sorting it
requires (see trac ticket 30) - obtained a 10% speed gain overall.
Reviewed quality bit flag website documentation for DR2 release (see trac
ticket 10). Discovered and fixed a bug in CU18 that caused it to not create
indices if too many transactions are required.
Removed all redundant SQL fragments, wrappers and database constants (see
trac ticket 9). Updated all references to the loadServerHost() function to
point to SystemConstants instead of DbConstants."
ETWS reported refactoring the SQL parser to create tables automatically
from given views for DR2.
NCH noted (with apologies to the operations side) that his design of
CU19 is obviously overly cumbersome and unscalable, and he and RSC
will refactor next week to make release DB preparation faster and
much less disk-hungry in future.
ACTION: NCH and RSC to refactor CU19 next week.
NJC and RSC have been working on the SQL Cookbook to educate users
in sample selection trading off completeness with reliability via
use of quality error bit information.
Survey Data Release:
NCH asked that we make a final review the website documentation to make
sure all is in place to inform users of the changes over previous releases.
Non-survey Data Release:
Nothing to report this week.
Astrogrid deployment:
Nothing new to report this week.
Miscellaneous:
PMW noted the schedule for registering contributions to NAM. NCH
noted that there is a "Degenerate Astronomy" (sic) session which
should really include Brown Dwarfs as there is nowhere else for
this in the scientific programme, and there are major new results
coming out of UKIDSS in this area. The session organisers for
this and for "Survey Astronomy" will be contacted.
Finally, NCH noted (thanks to JDT) that the First International
Workshop on Database Preservation takes place on 23 March 2007
organised by the Digital Curation Centre and the Database Group in
the School of Informatics. This workshop "aims to bring together
an interdisciplinary group of researchers and practitioners who
will address archival issues associated with databases."
http://www.dcc.ac.uk/events/