10th October 2005 Obs meeting room 11.30 - 13:30
Present: DWE, MJI, PSB, STH, MR, JRL, RGM, WJS
Apologies: NAW, EGS
Agenda
------
1. Actions from last meeting
2. Comments on WFAU minutes
3. Review of SV, Calibration and VDMT meetings
4. News from ADASS
5. VDFS VISTA update
6. Data archives update - UKIRT, WFCAM, AAO, ING
7. WFCAM news
8. Optical processing update
9. AstroGrid liaison update
10. AOB - CASU web pages
- APM building
Minutes
-------
1. Actions from last meeting
STH have installed the SADT on the Solaris system. PSB said that this Java
PSB programme was dated 23 June, so a number of suggested changes have not
been included in this version. WJS has also been helping with these
tests. This needs to be completed (WJS to collate notes) and the
comments sent off - ongoing <<<<
JRL has finished writing the access scripts to the mirrored OMP database
tables. He will email people their location. A username and password
will be needed. JRL said that he would check to see which tables are
available and which are mirrored. <<<<
MR has finished writing the draft of the flat field properties and sky
frames report. MJI said he had some comments to feedback <<<<
MR ensured that the features web link is publically accessible
EGS
MJI at the AAO board meeting found out what was happening re: the
AAOmega data processing
MJI sorted out the problems regarding the new processed data versions
that were being transfered to WFAU. PSB said that the peak transfer had
been Saturday with 1.1TB transfered. With the current costing model,
the transfer for the whole weekend cost £1300. It was noted that all
data files were copied, not just the catalogues. It is expected that
up to 20 TB of WFCAM transfers will occur each year. It was then
suggested that PSB contact the University Computing Service to discuss
the UKERNA pricing policy. <<<<
It was noted that the transfer rate will rise to about 50TB per year
for data transfered for both WFCAM and VISTA from 2007 onwards.
MR reprocessed the catalogues for the V2 release (but see later comments)
JRL
MJI has made no progress with the URD feedback for the next VDMT meeting
(was away) - ongoing <<<<
There was a request to add the VDUC minutes to the internal CASU pages.
RGM will place these in a suitable location. <<<<
MR have started work on a survey progress web page linked to the DQC
EGS database. This task will now be added to the quarterly task list. RGM
pointed out that the JAC had a survey progress page:
http://www.jach.hawaii.edu/UKIRT/surveys/UKIDSS_Survey_Status.html
JRL has finished the AAO archive access scripts. He will distribute the URL
so that we can test it and send back comments <<<<
STH did a direct comparison of UKIRT FS and 2MASS photometry. The end result
is that there is a 4% offset in H intrinsic to 2MASS (this is a known
feature). After much discussion, including with the Chair of the
calibration working group, the end result is that the frame-by-frame
WFCAM system will now be based on 2MASS but with a -0.04 offset in H to
bring it more accurately onto the Vega (and UKIRT FS) system. This will
all be more fully documented in Simon's calibration report.
MJI has read, commented and signed off the PD33s.
ALL the issue of Plone and Solaris web pages was deferred until item 10.
2. Comments on WFAU minutes
There were a number of WFAU minutes since the last meeting. Mike noted that
there had been some problems with the transferring of reprocessed data which
was mainly linked to the semaphore flags. We think that this has now all been
sorted out.
3. Review of SV, Calibration and VDMT meetings
SV:
STH said that more science had been carried out with the data, but not many
more problems had been found and in general people were satisfied.
There were a few small issues raised:
There were several comments about the merged classification statistic.
This arose from discussions between MJI and NCH to satisfy users who
wanted a single number from the results of the classification programme,
which would have been run independently on several different passbands.
The current version only makes use of the quantised individual
classification flags and some users felt that using the entire
classification information might be a better solution. MJI said he was
investigating as soon as his email backlog allows. <<<<
Blended objects were occassionally ending up as single objects. MJI noted
that deblending was a statistical process so this was no surprise but
said that he would investigate the specific cases mentioned. <<<<
There was much discussion regarding the release of SV (and other 05A) data
to facilitate WFCAM data exploitation.
The status of the crosstalk processing was raised and it was reported that
this was currently being test run on 05B data and if this proves satisfactory
then 05A will be reprocessed with a crosstalk correction.
Calibration:
STH said he gave the talk as presented/discussed in draft form at the last
CASU meeting. A number of actions are now on STH. He had been requested to
redo the systematics analysis only using data from nights of good seeing and
good transparency. Also a specific investigation into radial terms was
requested in addition to the more general 2D systematic investigation. In
general, though attendees were satisified with progress.
VDMT:
MJI reported on the last telecon. The inconsistencies between the pipeline
release schedules of CASU and WFAU had been discussed. The main change for
CASU was to align the end of the VDFS development phase to the of Q3 2007
to coincide with the end of the current VDFS development funding. Some of
the nomenclature has been changed so that "Phases" of software development
will replace "Versions" to avoid confusion with Versions of the pipeline
processed data.
MJI said that JPE had initiated some discussion over methods of transferring
data for VISTA from ESO. MR noted that the current peak transfer is about
2Mbytes/s. However, due to software checking and other overheads at the ESO
archive end, the effective transfer rate was 0.4Mbytes/s. This is difficult
to reconcile with the claimed bandwidth of 155 Mbits/s to Garching.
4. News from ADASS
JRL said that his talk was well received. Although he had no questions asked
during the presentation, he got a good number afterwards (which he preferred).
PSB said that both JRL and EGS gave good talks, especially since these were
among the few presenting results using real data.
JRL said that the idea of ADASS 2007 being held in London was well received.
He has already been down to the British Museum to investigate using their
Education Centre. A LOC meeting should be held soon.
MJI said that he would forward the details regarding the Edinburgh NIR
meeting (9/10 November) to JRL. <<<<
PSB said that he had helped organize the Leap Second BOF (there will be one
at the end of the year - Second that is).
It was also noted that the FITS standard will imminently have a 64 bit
format for data.
5. VDFS VISTA update
WJS reported that an announcement has been made by JPE that the money has
been found to fund both a Z and Y filter for VISTA. In fact, the Y, J, H and
K_s have already been manufactured and their responses measured. Copenhagen
has also placed an order for a 1.19 µm filter.
Pictures of the latest developments can be seen on the ROE VISTA web page
http://www.roe.ac.uk/ukatc/projects/vista/news.html (see "Civil work" and
"Telescope News").
PSB suggested that it would be useful if some of CASU could go to RAL to
test drive the camera (with the science arrays) and "observe" with the system
to generate the sort of engineering data that we need to shakedown the VISTA
pipeline development system. If feasible this would have to be before the
camera departs RAL, implying ~January. WJS was asked to liaise with Gavin
Dalton and Kim Ward to see if this was possible. <<<<
JRL reported on his progress with the Data Reduction Library. There was a
need to deliver 0.1 by the end of the year. Several recipes and modules have
been written. The question of how to release the DRL was discussed. It was
thought that to start off with a direct release to ESO via a CVS would be
useful in order to get early feedback.
6. Data archives update - UKIRT, WFCAM, AAO, ING
UKIRT: Up-to-date.
WFCAM: MR reported that a new front page has been implemented for the raw
data archive. New data is being received (05B) and ingested.
AAO: The AAO archive interface is now mostly completed and is under test.
ING: Two new disc servers (8 Tbytes each) have arrived and been installed
in the APM building. One of these will be used for ING data and the other
for WFCAM data. Internet transfer of data is progressing well ie. usually
arrives within 24 hours of the data being taken. There is currently a gap
between the last DVD data ingested and the first Internet data. This is in
the process of being filled using transfer of data on 400 Gbyte IDE disks.
There are now several tasks for the ING archive requiring urgent action
MJI will sort this out offline with JRL, RWA and EGS. <<<<
7. WFCAM news
MR reported that processing of the 05B data was about to start. The delay in
processing was due to delays in generating twilight flats and finalising
the pipeline implementation of the cross-talk correction. These issues are
supposedly resolved, so processing will now start. It was noted that the
UDS will use 3x3 interleaving but interestingly that the image quality in
the recent set of test data was only ~0.7 arcsec.
JRL said that he would double check the current version of the pipeline to
make sure that it did correctly include cross-talk processing. <<<<
Changes in the various survey MSB designs have resulted in a significant
reduction in the data volume.
JRL was asked to arrange the next JAC telecon since there has not been one
for some time due to assorted absences. <<<<
MJI pointed out that there were a number of bugs in the V2 reprocessed data
(including table NaNs and repeated keywords). These should have been spotted
eg. by running fverify as recommended. This (again) emphasizes the need for
a test harness after any significant pipeline changes. Possible suitable
data for this would be some of the much studied 8th April data. MR and MJI
will investigate. <<<<
MJI and JRL said that they would discuss the various bugs and related
actions. <<<<
STH said that the various changes to the software should be noted on the web
site. MR said that the CVS has a log of changes to the software that can be
used for internal use. MR will look into this issue. <<<<
8. Optical processing update
MJI noted that the VST survey proposal resubmission deadline was the 15th
October. He is helping provide the data management and products section.
9. AstroGrid liaison update
No news.
10. AOB
CASU web pages
It was decided that a small task force would be formed (EGS and DWE) to see
how we can improve the group's web pages. The general feeling at the meeting
was that it was difficult to find things and that having two separate systems
(Plone and Solaris-based HTML) was not helping. MJI said that he would
forward EGS's email on this topic to DWE. <<<<
DWE and EGS will report back at the next meeting with suggested ToRs
and a list of issues to tackle. <<<<
APM building
MJI said that it was now 6 months after the completion of the refurbishment of
the APM building and it was time to draw up a snag list. Various issues were
raised: noise, wobbly toilet ride, esoteric heating system..... MJI said
that he would forward the email from Peter Weeks so that we can see if all
the snags are covered. <<<<
Gaia news
MJI reported that the UK consortium grant bid for Gaia processing will be
submitted on the 21st October and then go before the new PPRP process.
Continuing Actions
------------------
PSB finish off SADT tests, collate notes and sent off comments
WJS
MJI feedback comments to MR on flatfield and sky properties draft report
JRL distribute location of OMP database access scripts and check which OMP
tables are available and that they are mirrored correctly
MJI append more details to the URD feedback for the next VDMT
New Actions
-----------
PSB contact the University Computing Service to discuss the UKERNA pricing
policy
RGM put VDUC minutes on internal web pages and email location
JRL distribute the URL of the AAO archive interface
ALL test drive AAO archive and feedback comments
MJI investigate merged classification statistics
MJI investigate blended object complaints
MJI forward the details of Edinburgh IR meeting to JRL
WJS sort out a possible CASU visit to RAL with Gavin Dalton and Kim Ward
MJI convene meeting with EGS, RWA and JRL to discuss ING archive issues
JRL check current version of the pipeline does do cross-talk processing
JRL arrange the next JAC telecon
MR sort out a possible test harness (8th April data)
MJI
JRL discuss recent bugs in V2 reprocessed data
MJI
MR investigate web-based presentation of various changes to software
EGS sort out Plone/HTML for group web pages
DWE
MJI forward EGS's email regarding web pages to DWE
MJI forward email from Peter Weeks regarding snag list to all
ALL who will add to snag list if needed