VISTA IOT #11 MEETING 2009-11-03
=================================
Minutes by: M. Rejkuba
Additions by: J. Emerson, M. Irwin, M. Arnaboldi
================================================
November 03, 2009 10:00 Chile, 14:00 Garching
-----------------------------------------------
CONTENTS:
0: Presence at the meeting
1: VISTA status, schedule and planned activities
2: Science Verification Run Summary/Report
3: Preparation for the Dry Runs
4: Outstanding issues (punch list, data flow, first light, etc.)
5: Action Items review
6: Others
7: Dates of the next meetings
===========
=== 0. PRESENCE ========================================================
===========
Paranal - T. Szeifert ................................... [TSZ]
Santiago - V. Ivanov ................................... [VIV]
S. Mieske ................................... [SMI]
Garching - M. Arnaboldi ................................... [MAR]
M. Cullum ................................... [MCU]
L. De Bilbao ................................... [LBI]
W. Hummel ................................... [WHU]
G. Hussain ................................... [GHU]
M. Neeser ................................... [MNE]
M. Petr-Gotzens ................................... [MPE]
M. Rejkuba ................................... [MRE]
M. Romaniello ................................... [MRO]
via Phone - J. Emerson ................................... [JEM]
J. Lewis ................................... [JLE]
M. Irwin ................................... [MIR]
S. Hodgkin ................................... [SHO]
========================================================
=== 1. VISTA schedule, current status and planned activities ===============
========================================================
TSZ gave a quick overview of the current VISTA status. The science verification
run was very successful. At the very end of it there was a problem with M2.
The next step is unclear - we are waiting to get the green light for dry runs.
He then asked M. Cullum and J. Emerson for update of the handover status.
MCU: the situation is now in higher spheres - discussions are at the level of STFC
and the ESO DG. According to Tim de Zeeuw there should be no problems to go on,
but there is no written confirmation yet. For the SV there was an e-mail and
agreement between Tim de Zeeuw and Pat Roche, giving OK. For the dry runs we need
to wait for confirmation from Tim and Richard Wade. Essentially we are now waiting
for STFC upper management.
JEM pointed out that it is crazy not to go ahead and to leave the telescope idle.
TSZ agrees, but points out that we do not know what can be done with data. We are
waiting to get a go ahead because the data we are (about to start) taking should
be public immediately. For now ESO does not have permission to put them public or
deliver to the PIs.
JEM suggests that data are put under embargo. TSZ confirms that this is what we
already did effectively, because some data were taken during the SV run for Public
Survey PIs and for now we cannot deliver them.
According to MCU A. Kaufer is worried that since the telescope is not yet ESO
property and there is nothing in writing to say that it is ok to use the
telescope, it is not clear whether we can do that. All that would be needed is a
short e-mail from R. Wade to Tim.
JEM says he will try to push for that, but there are no guarantees that he can get
to talk to R. Wade quickly.
TSZ then summarized the current activities. There will be a telecon with T.
Bierwirth on Observing Tool (OT) delivery for Paranal for the end of this week. M.
Hilker will test the tool on Paranal. Currently M. Hilker is there (and could
potentially observe if allowed), and after him there will be S. Mieske. After that
there is still need to change the Paranal schedule.
TSZ will need to meet with engineers to decide what will be done in terms of
Software and Hardware work on punch list items.
JEM reported that CASU made several very nice color pictures for PR
purposes and 3 had been chosen for release. R. Hook has them all.
MAR asked if the PR date has been set.
JEM: this is provisionally set to 18 Nov, but may be shifted to another date,
depending to handover date. Previously handover was supposed to happen
before the PR event, but now the PR waits for the official handover. JEM then
asks what is the actual issue with the acceptance.
MCU replies that there are discussions about the punch-list. ESO is willing to
close a number of items on punch-list and VPO should pay for this. Paranal now
needs to give the estimate of man-hours and cost. In addition there is the issue
of spare parts for which Paranal needs to give the list. And then there are some
discussions because the list presented by Paranal includes some improvement items
that need to be treated separately. There was a meeting in Santiago between S.
Guniat, I. Robson and M. Cullum and it is expected that by mid November all should
be closed.
TSZ says that Paranal people are in a difficult situation as the software
engineers do not know which tests can now be done. There is till missing go ahead
for tests before handover.
==========================================
=== 2. Science Verification Run Summary/Report ==========================
==========================================
MRE gave an overview of SV run. The observations were carried out between
15-30 October. Then due to M2 problem few nights there were no observations.
The last night of SV data was taken on the night 02-03 November. She then
described some technical problems encountered during the science
verification run. Overall the run went very well and none of the problems
is a show-stopper for the start of operations. The observations went quite
smoothly and a large amount of excellent quality data were collected.
The observations allowed to test a large number of observing templates with
different parameters. One problem found was that Tile 3px and Tile 3nx do
not appear to cover uniformly the full tile area. This appears to be due
to the way that SADT starts to tile the area.
This started some discussion between JEM and MRE on what is actually
expected behaviour. It was decided to go on with the discussion offline.
The SV "technical summary" was circulated to the IOT members after the
meeting via e-mail.
MPE gave description of data taken for the Orion, Galactic mini-survey
during the SV. The full area was completed last night. According to a quick
check the sensitivity is as expected. Data were reduced on Paranal with the
Pipeline ESO has there, but it would be very desirable to compare the results
of this reduction with the reductions from pipeline by CASU. During the SV
run there were some exchanges of e-mails with J. Lewis and M. Irwin and they
provided help with some issues that were encountered.
MPE then proposed to initiate an informal exchange of e-mails and send to
CASU information about the particular things to keep an eye on. For example
some OBs had to be re-started several times due to technical CCD crashes.
This information is not always available from the night reports.
CASU got the first disk with SV data on Monday 2nd and will probably get
most of the rest with the next shipment. Additional information that
could help in reductions is welcome. They plan to come to ESO on
25/26 November and it would be ideal to meet then, compare the reductions,
and discuss the SV data publication. Not many alternative dates before the
end of the year are possible. By 25-26 Nov. it should be feasible to have the
majority of SV data reduced.
MAR gave summary of NGC253 observations, the extragalactic mini-survey. The
observing strategy had deep J, Z, and NB118 observations and shallow imaging in
all broad band filters. In total there were collected about 8h in Z, 6h in NB118,
and 24h in J-band. The seeing was slightly worse than for Orion observations, but
it ranged between 0.7"-1.2" and only few observations had up to ~1.4".
The highlights are that the stars in the disk and halo of NGC253 are nicely
resolved, and the shallow K-band image of the disk shows very interesting structure.
In addition to the two mini-surveys the observations requested by the
public survey PIs to be executed during the SV were done within constraints: these
are VMC test observation, some VVV and 4h total for joint VIDEO and UltraVISTA
teams.
MRE added that due to passage of NGC253 through zenith some short ~5min exposures
were taken on NGC300 in all broad band filters.
JEM stated that NGC300 was not so spectacular and both this and NGC253 were taken
with the idea to have them for PR purposes. In the end these were not chosen. In
addition there are shallow observations of Orion NGC2024 field. These could be
sent to Magda and Monika to add them to the rest of the SV data.
MAR presented plans for the SV data reduction and publication.
MRE and MAR went together through the SV matrix and a report can be expected within
~10 days. This will be posted on the web together with the logs of all observations.
After that we plan to publish the data products from the pipeline reductions. Later
it is expected to publish also stacked images and catalogues.
MAR asked whether we should aim to publish together with CASU the reduced data?
JLE inquired about the time-scale for that.
MRE said that 70-80% of the reductions were one in Paranal in parallel with the
observations. The rest will be reduced in Garching on a machine that runs the
identical pipeline as on Paranal. The time-scale for the release/publishing of these
data is not so much dependent now on the time needed to process them, as much it
depends on the handover and PR release dates. We could be ready to publish OB level
reduced data in ~2 weeks.
JEM/MIR mentioned that CASU reductions should be better due to some improved recipes,
but also due to fact that the latest calibrations can be used in reduction.
MRE explained that the reductions on Paranal were made "manually" adding offset sky
fields and also re-running the calibrations through the pipeline and associating the
up-to-date calibrations to the science images.
MRO asked if UK (CASU/VPO) could publish the data before the acceptance?
JEM answered that the formal point is that ESO has not accepted the telescope and
therefore cannot publish the data without UK agreement. Formally speaking UK could
do that without ESO agreement, but that would be foolish.
MAR noted that the publication of SV data could be discussed and agreed on 25-26
November.
MIR points out that it is necessary to make the tiles to get deeper images. MAR
confirms that ESO plans to use Swarp to do that. It will be interesting to compare
the results with what can be done with CASU pipeline.
MIR asks for confirmation that the last night of SV is 02/03 November. That was
confirmed.
==========================================
=== 3. Preparation for the Dry Runs ================
==========================================
TSZ states that everything is ready for the dry runs.
The only issues are the schedule of people on Paranal, the tools (OT) are
expected to arrive very soon. The big problem is the disk space -in particular
on the pipeline work station, but there is action requested to remedy this.
================================================================
=== 4. Outstanding issues (punch list, data flow, first light, etc.) ======
================================================================
As most of the items were already discussed (punch list, first light/PR, etc.)
VIV points out that the only outstanding issue is the data flow related
disk space.
TSZ adds that on the pipeline workstation this can be solved by giving up
redundancy.
MRO asked if there is enough disk space for data transfer and on staging area.
TSZ replies that there is 2TB disk, but it is not possible to keep all the data on
the disk while the USB disk is traveling.
MRO: This is worrying. It would be better to have staging area with larger disks.
TSZ mentions that there is also problem with the operating system.
MRO says that this needs to be given the highest priority to bring it to the
original specs of 7TB.
JEM mentions that ESO already ordered additional disks, but TSZ explains that these
are different machines. The problem with the staging area disk space is important
pending issue.
========================
=== 5. Action Items review =============================================
========================
AI 2009-01: Monitor execution times & survey completion status
Is being followed up by MAR, MRE.
2009-07-29: There is a first prototype
2009-10-06: no news (MRe)
STATUS: ONGOING
*
AI 2009-04: SADT delivery
2009-10-06: MAr reported that F. Comeron asked to have formal
acceptance meeting. The plan is to have report with all
feedback collected from the Science Verification and Dry Runs
observation preparations ready by November 17. This report will
be sent to JEm and after ~2 weeks the reply is expected. After
that the acceptance meeting should be scheduled in the first
week of December.
2009-11-03: JEM asked to have an informal copy with the list of
issues found to be able to go ahead with the work on SADT.
MAR should send this list to JEM.
STATUS: OPEN
*
AI 2009-06: VIV: by the end of June send the new simulation
report and code to be taken over for future simulation.
2009-10-06: report writing in progress (VDI); to be transferred
to the EST during Oct SciVer (TSz)
2009-11-03: VIV gave a version of the code to MAR. This will be taken
over by GHU. VIV still needs to add some improvements of ranking and
priorities and is writing a report.
STATUS: ONGOING
*
AI 2009-09: Offset sky template would be requested for the next
(post SV, dry run) instrument package. Pipeline data reduction
recipes are ready to work with the offset sky data sets.
2009-10-06: VIV should organize it, with help from software (i.e.
Pedro Baksai), and help from JEm; by Dec 2009; starts to be
changed from on hold to open (MAr)
2009-11-03: VIV circulated some ideas with some IOT members. Some of these
would need changes in SADT. Also a classical on-off-on-off template is
discussed that would not need guide stars.
JEM mentions that S. Beard made template that could be tested in
the next few days.
TSZ points out that this goes exactly back to the discussion of what we can
and what we cannot do now due to handover problem. It would be necessary to
get a specialized SW engineer and astronomer at the same place and at the
same time. This is currently not foreseen.
JEM suggests to get a remote help from S. Beard.
VIV clarifies that TSZ is going down from Paranal today and it is not
known when either TSZ or VIV would be scheduled on VISTA.
TSZ says that the green light is still missing and this is no-mans land
where it is not clear whether we are allowed to spend resources on this
before acceptance.
VIV explains that from operations point of view only two surveys VVV and VMC
would need offset sky exposures. For VVV due to short exposures skies could
be taken from concatenated tiles, while for VMC there is some workaround with
the editing of XML files. However, the classical sky offset templates will be
needed in the future.
TSZ and VIV will look into what was implemented by S. Beard.
STATUS: OPEN
*
AI 2009-11: TSz should find out if the constraints sets could be
carried with the OBs, to the OS and the fits header of the VIRCAM
files. Otherwise it is not easy to propagate these data by
interfaces.
2009-07-29: TSZ iterating with R. Schmutzer, but DFI (T.
Bierwirth) will also need to be involved. For surveys, given
the small range of programmes it is expected that there are not
too many different OBs, and this might need to be followed in a
different/manual way in the beginning - this could imply a major
change for the tools that cannot be done in the last moment.
2009-11-03: MRE drafted a short summary document (dated: September 18)
of what is needed. There is a scheduled telecon with T. Bierwirth
on FITS keywords and tools.
STATUS: ONGOING
*
AI 2009-12: TSz should check how/if the fits keywords of the raw
file have the information about the concatenation and other
scheduling containers.
2009-07-29: Answer is NO. But is this necessary? JEM thinks yes
if processing of data takes this into account. TSZ: this will
need to be done together with P2PP/OT programmers and VLT
software support after discussion. TSZ will raise it, but cannot
promise a solution.
2009-10-06: the information about the containers must be included
in the keywords, CASU must be included in this work because it
will have an effect on the pipeline (JEm); this is also relevant
for the sky offset template (MAr); not really, because the sky
offset is in the same OB as the primary target tile (JEm); in the
past we used for this purpose the DPR keywords (TSz); this info
might be needed also for the ESO archive users (WHu); a good
example for the archive is to identify the telluric standards,
corresponding to a given data set, so the issue is relevant
generally for all ESO instruments (MRe).
2009-11-03: This goes together with AI 2009-11;
MRE drafted a short summary document (dated: September 18)
of what is needed. There is a scheduled telecon with T. Bierwirth
on FITS keywords and tools.
STATUS: ONGOING
*
AI 2009-14: GHU to check that the seeing distribution function
in the scheduler for VIRCAM takes into account the finite (0.51")
nominal resolution of the instrument.
2009-11-03: GHU reports that this is possible. She needs to get in
touch with T. Bierwirth to have this implemented in tatoo.
This will be primarily necessary once VISTA becomes offered (period
P87).
STATUS: ONGOING
*
AI 2009-19: JEM should sent the latest updates for the exposure
time calculator to Marina (silver reflectivity, filter curves)
2009-10-06: the transmission curves for the NB filters are still
missing (MRe); OK, will send them when back to UK (JEm); next
the discussion goes towards the differences between the ESO and
VPO VISTA ETCs: it is important to understand the differences
(MRe); the VPO ETC was not even meant to be updated anymore to
avoid competition with the ESO ETC, which was meant to be the
main VISTA ETC (JEm); how is the sky brightness calculated in
the VPO VISTA ETC? (TSz); it is a parameter that can be modified
by the user (JEm); there are known problems with the sky
brightness in the IR ESO ETCs, especially in the H-band, they
seem to be calculated from a wrong sky spectrum, although there
is a good one available, but it is used only for the EELT ETC
(TSz).
2009-11-03: Filter curves received from JEM (date: 2009-10-28).
The SV provided a good dataset to check the ETC.
MRE reports that some changes have been implemented by K. Shabun.
This will now be checked and if needed changes can be made based
on the results from SV.
STATUS: CLOSED
* *
AI 2009-20: installation of a new server for faster/cheaper/better
data reduction, to be available for semi-real time data reduction
during the Science Verification run on 16-30.10.2009. TSz will
talk to Christophe Dumas.
2009-11-03: was done
STATUS: CLOSED
*
AI 2009-21: Create poor weather conditions (particularly poor
seeing) back up SciVer programs. The approved SciVer programs
request seeing 0.8 arcsec or better, if the SciVer was last week,
it would result in almost zero data.
2009-11-03: was not necessary
STATUS: CLOSED
New Action Items from this meeting:
*
AI 2009-22: Increase disk space for staging area on Paranal.
To be followed by TSZ.
========================
=== 6. Others =============================================
========================
WHU asks what is the status of calibration unit.
TSZ says it is disconnected. I is necessary to check if the additional power supply
that was added is needed and if so, then how to implement it permanently. Currently
the cabling is not safe. It is expected that after the peak activities for the SV
are passed CASU should say if the additional stability is ok.
JEM says CASU have checked that the new lamps stability is ok and
says that the schedule in getting the new transformers to use them
routinely is unclear.
TSZ answers that he was not fully in the loop for this and it is not clear.
WHU says that the SV data reduction needs to rely on gain, non-linearity
corrections and bad pixel mask from before intervention (re-coating). Was the
camera warmed up?
JEM says it was warmed up but not opened up.
TSZ mentions that the linearity sequence could be done in "Hawk-I" style in K-band.
JLE will send a report based on PR data and some suggestions on observing practices
"dos" and "don'ts".
JLE suggests to not have DIT<5 sec for twilight flats.
MIR: it should be significantly longer than 1sec at least.
TSZ says he will update the twilight flat templates.
========================
=== 7. Next meeting date =============================================
========================
December 1, Tuesday, 14h Garching, 10h Chile.