VISTA IOT #8 MEETING 2009-03-26 ================================ Minutes by: M. Arnaboldi. Additions by T. Szeifert and W. Hummel ====================== June 30, 2009 10:00 Chile, 16:00 Garching, 17:00 UK -------------------------------------------- ******************************************************************** Santiago, Athena room 10:00 Garching, SkyLight 16:00 Paranal, Control room 10:00 London&Cambridge by phone 15:00 ******************************************************************** CONTENTS: 0. Presence at the meeting 1. Overview about the estimated time lines for the telescope/instrument and phase2 2. Readiness Review for the call for science verification and dry run OBs 3. Round table (be prepared to say a few words about the state of preparation for VISTA in your group) 4. Status update of old action items 5. New action items 6. Date of next meeting (last Tuesday of the month if there is no special reason like a deadlines) =========== === 0. PRESENCE ======================================================== =========== Santiago - T. Szeifert ................................... [TSZ] S. Mieske ................................... [SMI] Paranal - V. Ivanov ................................... [VIV] Garching - M. Arnaboldi ................................... [MAR] W. Hummel ................................... [WHU] M. Hilker ................................... [MHI] via Phone - J. Emerson ................................... [JEM] CASU: M. Irwin, ................................... [MIR] VIV did not attend the VC. ======================================================== === 1. Overview about the estimated time lines for the telescope/instrument and phase2 ======================================================== TSZ gave the summary of the schedule for VISTA which is included below: Schedule: ----------- 2008-06-21 - Start of the VISTA Commissioning with the Camera (VIRCAM); tests by TSZ of the data flow (p2pp, OT) with full engineering support 2009-06-29 - Start of the ESO Verification Period 2009-07-24 - Start Camera and components verification by ESO (camera down) 2009-08-29 - Telescope ready after verification and coating 2009-09-03* - end of verifcation, end of night time performance check 2009-09-16 - change of M1 actuator 2009-09-18 - check on sky (after 2009-07-24) - 1st letter out to inform PIs for dry runs (after 2009-09-08) - Call for phase 2 dry runs 2009-09-19 - start of the Science Verification run (10 nights out of 14) 2009-10-03 - Early Science Operations: Paranalization and Dry Runs (*) -- 4 weeks between this date, until when we should eventually wait to call for OBs, and start the dry runs. TSZ described the plan related to dry runs. After October 03, 2009, the VISTA paranalization will start and some science time may become available after technical interventions. TSZ, MAR and MRE proposed to have dry runs to take advantage of the available observing time. The PS teams will be contacted and asked to prepare OBs for a given amount of observing time, and these OBs will be prepared with the tools (SADT, p2pp, IPs) which are released for VISTA science verification (SV). The call for the dry runs observation will be issued at the beginning of August. The exact date will be fixed after the ESO commissioning period, i.e. after July 24, 2009, subject to the positive outcome of the Data Flow Operations tests by TSZ. ESO will organize an internal review and inform the PIs after that. MAR requested that TSZ is the official person who sets the date for the call for dry runs. Upon green light from TSZ, then EST and USD will contact the PIs, inform them that VISTA is close to ready, and ask them to identify the point of contacts in the teams for the preparations of the phase 2 of the dry runs. ======================================================== == 2. Readiness Review for the call for science verification and dry run OBs ======================================================== ** 2.1 Concerns TSZ pointed out that there are two concerns regarding the above plans: - i) the support of M2 working - ii) status of SADT i) JEM reported that M2 got stacked in the last few days - it is still not known why. It may be related with micro-oscillations of M2 that also degraded the image quality. VPO is trying to fix this problem asap. Perhaps M2 needs to be disassembled. If it cannot be fixed, it may in fact impact on the start of the July 10 run by ESO and cause a delay. ii) About SADT: JEM reported that he implemented the rotation of the longer axis of the camera, but a sign is wrong and the guide stars/ao stars fall on the other guiders, causing a problem with the execution of the OBs. JEM (or somebody else) will fix it. It may require some time after the Review in Edinburgh planned for the July 6, 2009 week. TSZ added that the last available night to test SADT at the telescope during the ESO run is July 23rd. This is the last date when a new SADT can be released and tested on the telescope, before it is accepted as the official version for VISTA SV and dry runs. JEM replied that he has all the information from TSZ concerning the problem with SADT and he will release a new version to EST, USD and Paranal for SV and dry runs. The SADT manual will need to be updated and the problem clearly documented, if still unsolved.. TSZ and JEM discuss about the problem of the camera rotation and how to exchange information so that JEM can fix it in the SADT application. MAR pointed out that EST/USD need to have the SADT manual and the tips regarding the area definitions by 2nd half of July for the preparation of the phase 2. ** 2.2 Phase 2 tools TSZ recalled that we need also the information about the VISTA templates, without those Stephan Marteau cannot finalize the EVM for the OBs check in. Front end - p2pp & OT : TSZ reported that p2pp and OT will be updated once the new database infrastructure is released and this will happen during the 2nd half of July. Only small issues should be fixed for p2pp and OT. ** 2.3 Backend and Sciops Back end - dataflow ok, pipeline running. TSZ will inquiry and apply with Cecilia for additional disk space to be installed at the VISTA workstation. Operations : TSZ reported that Cecilia is looking into the error reporting/night log for VISTA and TSZ is waiting for feedback. WHU reported on the current work on the QC procedure. At the moment CalobBuild is missing so the darks/biases currently do not match the acquired flats and the QC pipeline cannot work as expected. CalobBuild should be in place for the July 14-21 period. WHU reported that the HAWK-I astrometric distortion was computed during commissioning and now it is not valid anymore. WHU asks whether the VDFS pipeline applies a static distortion correction or whether it is updated regularly. JEM reported that an astrometric solution is computed using 2MASS stars for every science frame, therefore the quality of the WCS fit is monitored. MIW stated that the distortion model coefficients for the astrometric solution will be updated by CASU from time to time. The updated coefficients will be available on the CASU webpages but they won't be feed back to the pipeline recipe unless a new release is installed. TSZ added that in the case of HAWK-I the camera field lens was changed and this was the main source of the problem. WHU stated that this was new to him and would check this. Discussion between JEM and TSZ on what is causing the different distortion in HAWKI vs VIRCAM. ** 2.3 Photemtric calibration and illumination correction TSZ inquired whether CASU will be ready to reduce the data from September 2009. JEM replied that they are. The discussion then turned to ULTRAVISTA: CASU will get the ULTRAVISTA raw data and another copy is sent to Leiden. MAR: need to coordinate the data delivery with AOG. TSZ asked about the illumination correction for ULTRAVISTA and about the narrow filter. Does ULTRAVISTA need additional calibrations? Should a new template be defined for the illumination correction? MIR stated that they plan to use 2MASS stars and color equations for the narrow band filter too. The problem is that the bandpass will change with the field position. JEM stated that this can be addressed by using the same standard star and offset it on a regular grid and compute the illumination correction for the narrow band filter this way. TSZ agreed to prepare the OBS for this in August and get the data, and ask MIW (CASU) to help with the data reduction. JEM stated that they have data up to 16 hrs integration of a region of sky . CASU is coadding the data and the co-added image will be given to both VIDEO and ULTRAVISTA for their tests. MIR added that in addition to the release of this co-added image, they are planning observations of regions with high and low extinctions, and CASU is planning to help with the calibration of the photometry. TSZ suggests to observe also UKIRT stdstars and compare whether the results on the extinction correction are similar between 2MASS and UKIRT stds.TSZ suggests that the Sagittarius dwarf can be a good region for the extinctions tests also. ** 2.4 Fact Sheet for dry runs: Fact Sheet for dry runs: TSZ requested assistance from JEM for its revision before circulating it to the PIs with the announcement for dry runs. JEM agreed and CASU would provide input also: these are needed for the 2nd half of July.JEM stated that the acceptance data package will not help for the Fact sheets. This information will be finalized with the instrument package release. ** 2.5 VISTA ETC ETC: the ESO ETC for VISTA and VIRCAM is being finalized. MHI: main testing of ETC with the pipeline group, and MRE is providing feedback from her tests. VIV sent feedback on the ETC reporting some problems with the zero points. JEM is planning to send the filters transmission + ccd response curves but did not do it yet. Filter transmission curves: they need to be documented in the manual and web pages should refer to this information. =========================== === 3. Round table activities =========================== TSZ asked whether he can modify the template signature of files. JEM agreed and suggested to keep Steven Beard in the the loop. TSZ updates on the progress by VIV on the survey simulations and the upgrade to include realistic overheads. TSZ + VIV will work on the manuals and the preparations for the tests of the dataflow, and the active optic guiding for the ESO run in July. MAR reported on the proposal for dry runs, the SV and the pipeline tutorials to be scheduled in September. Observing runs will be created in the ESO database. Each public survey will be as a large program, with runs scheduled each semester. MHI reported on the current tests for p2pp, OT, the survey visualization monitoring tool, and the tool for OBs verification. USD is working at the web pages for the public survey Phase 2. MHI reported that Stephan Marteau is working on the ETVM but this many not work for dry runs, as it will be including the computation of execution time for scheduling containers. WHU stated that there is a review of the current pipeline tickets and negotiations with CASU are currently done. The trending plots will be updated at the end of July. In August there will be a migration of the VISTA QC compute cluster from GFS to LUSTRE. JEM had nothing to add. MIR reported that CASU received raw data from May 20 to June 18. The zero points, astrometry solution are computed and they will be used as input to JEM report at the VPO review in Edinburgh. JEM requested that the data taken on June 12, when the seeing was very good (0".4) all through the night, should be processed and ready for the VPO review. MIR stated that CASU can process the data before the end of the week. ====================== === 4. Action Items ======================== ====================== AI 2009-01: Monitor execution times & survey completion status (WIP: MAR,MRE) Is being followed up by MAR, MRE. AI 2009-01: ongoing AI 2009-02: Iterate draft of implementation plan, on hold waiting for C. Dumas (TSZ) AI 2009-02: ongoing (timeline draft circulated 02/07/09) AI 2009-03: Formalization of the procedure of feedback from CASU about the image quality of science data (WIP draft fragment sent by TSZ, MRE opened mailing lists to CASU, QMUL people involved in science operations). Sciops Paranal does check the data for image quality, checks the sky for clouds, will later query the image quality and zero points from the opslogs files. QC does check the calibration data, and does spots check on some selected files to track down principle problems. QC does not check if the data is taken within constraints. CASU does for WFCAM a WEB page with a night report, but so far doesn't check if observations were taken within the constraints or not. TSz pointed out that the night report will be sent to CASU, QMUL,... as well and will look if the constraints can be propagated in a convenient way downwards. Any other steps are to be discussed soon. (TSz: This needs to be simple enough not to cause additional workloads. It wasn't meant to send new requirements, but to look if we can get some simple feedback at low cost, if there is something found in the data). AI 2009-03: ongoing, TSZ is currently working on the document (draft circulated on 03/07/09) AI 2009-04: JEM: get in contact with F. Comeron to formalize the SADT delivery. Not yet ready to do this, though JEM chatted to Fernando about it. AI 2009-04: ongoing, JEM is in charge of the SADT revisions AI 2009-05: JLE: send CASU report to TSZ; particularly critical sky subtraction issue, but also needed information about the astrometry, calibration accuracy, and other items that should be covered in the User Manual A draft version is close to completition. AI 2009-05: A dedicated template is not foreseen at this stage. There is also no pupil rotation detected in the latest dataset. Better wait for the final VPO report. This AI is now considered CLOSED. AI 2009-06: VIV: by the end of June send the new simulation report and code to be taken over for future simulation AI 2009-06: ongoing. VIV is finalizing it. AI 2009-07: MAR/MRE: contact PIs at the end of May (once the handover is done/certain) and inform them about the time line and activities. Should wait until the Closed Cycle Cooler is repaired. AI 2009-07: AI shifted to the end of July 2009 AI 2009-08: QC to check if the opslogs contain the values for the image quality AI 2009-08; yes, CLOSED AI 2009-09: After there is a task in "track" to work on the template for proper sky field observations: TSz should forward the information to IOT to follow up the steps to make the pipeline work with this change. AI 2009-09: ongoing AI 2009-10: TSz to send to CASU an example feature introduced by the effect of the rotating pupil AI 2009-10: CLOSED. It was an effect caused by a dirty pupil in January AI 2009-11: TSz should find out if the constraints sets could be carried with the OBs, to the OS and the fits header of the VIRCAM files. Otherwise it is not easy to propagate these data by interfaces. AI 2009-11: ongoing AI 2009-12: TSz should check how/if the fits keywords of the raw file have the information about the concatenation and other scheduling containers. (TSz: I have had already something like this in my mail and must have forgotten later to look into this). AI 2009-12: ongoing =========================== === 5. Next meeting =============== =========================== Dates proposed by TSZ: ----------------------- 29 July 2009 NN 15:00 - 17:00 (09:00 - 11:00 Chile) 25 Aug. 2009 NN 15:00 - 17:00 (09:00 - 11:00 Chile) 22 Sep. 2009 NN 15:00 - 17:00 (09:00 - 11:00 Chile) Please feel free to propose special topics for the next agenda.