VISTA IOT MEETING 2010-06-29 ============================ Minutes by: M. Rejkuba ====================== June 29, 2010 09:00 Chile, 15:00 Garching, 14:00 UK ----------------------------------------------------- CONTENTS: 0: Presence at the meeting 1: VISTA status, last intervention and planned activities, PPRS-s summary (TSZ, VIV) 2: Status of various software 2.1. SADT 2.2. OT 2.3. IP 3: QC issues, status of the reprocessing (WHU, CASU) 4. Progress of the surveys (MRE) 5. Survey progress vs. simulations and feedback from the PIs (MRE, VDI) 6: Summary of Action Items 7: Other issues 8: Date of the next meeting =========== === 0. PRESENCE ======================================================== =========== Paranal - T. Szeifert ................................... [TSZ] V. Ivanov ................................... [VIV] Garching - M. Hilker ................................... [MHI] M. Neeser ................................... [MNE] M. Rejkuba ................................... [MRE] via Phone - J. Emerson ................................... [JEM] CASU: M. Irwin ................................... [MIR] W. Hummel ................................... [WHU] G. Hussain ................................... [GHU] S. Mieske ................................... [SMI] ================================================================= === 1. VISTA status, last intervention and planned activities, PPRS-s ======== ================================================================= TSZ gives the summary of the last technical intervention on VISTA. Between June 16-21 the broken gyro was changed, some spare parts were tested. Images look now much rounder. There are from time to time some exceptions when there are fast jitters and fast integrations. The parameters are still not optimal and the investigations are not finished. Flat field should not have changed because nothing was changed in the optics during this intervention. Discussion on the position of the camera after the shim was put is initiated by JEM. There is a possibility that the right procedure was not followed when the shim was put in and the camera did not go back in the b-blocks? TSZ replied that currently engineering is proposing to put camera horizontally and check, but there was no time to do these checks yet. TSZ reports that on M2 problems there is no news yet. He is waiting for the information from the other turno of people. The calibration flux will be adjusted just after the IOT meeting and the screen flats will be taken. VIV reports that there are no new relevant PPRS-s. MRE asks about problem with standard star templates reported in the NR. TSZ answers that this is related to a applied to tsfs to fix the problem with guiding that starts too early. This worked well for tile templates, but did not work for standard stars - therefore the change was rolled-back. Pedro Baksai will make the fix in the P86 templates. WHU asks whether the ellipticity will now be better because the gyro was fixed and the template was adjusted. TSZ answers that indeed gyro improved the situation, but templates were not making the difference. Sometimes the azimuth axis is a bit slow and the only way to solve this is with a control of the motor. There is once in a while the an elliptic image, but over the last few nights there were only 2 cases when the images were found elongated. Now the worst outliers are elongated in azimuth direction. Except for detectors 1-4 that are sometimes out of focus, the ellipticity is below 0.2. ======================================= ============== 2. Software tools status ===================== ======================================= 2.1. SADT JEM reports on the SADT Preliminary Acceptance meeting in Garching on June 8. Two bugs in the previous version were discussed. The first bug is related to the incorrect radius for search of stars and the other to the fact that SADT did not take the correct jitter length at the edge of the field. The two more or less canceled each other. The first bug is corrected in v4.06. The second is not yet fixed. Another point discussed was that now with more than one catalogue, the star selection magnitude limits are hardwired in the IP and this cannot be changed easily. JEM is putting the now the magnitude ranges for each catalogue in the SADT config file so that for the next semester this can be adjusted. For UCAC3 catalogue magnitude limits need to be established for the next period. Currently the same magnitude limits as for GSC2 apply, but this is not optimal. MHI reports on testing done in Garching. Additional on-sky tests were done on Paranal. Two small bugs were found and MHI will send the informal report to JEM. ESO expects to receive the final version of SADT on August 1, 2010. JEM confirms that this is agreement. MHI further asks for clarifications from JEM on how the toggle on/of flags for different parameter ranges for catalogue star selections are set. JEM will send this via e-mail to MHI. 2.2. OT & P2PP: MRE reports that there are no news on OT. For this Phase 2 there is a new P2PP version 3.1.4 which has implemented the new twilight constraint. USD is waiting today for the final testing of P2PP 3.1.4, but it is expected that by Thursday, July 1 the web-letters and call for Phase 2 can go out. The deadline for the submission of Phase 2 for P86 for VISTA is August 2, 2010, few days before that of VLT. This de-coupling was requested from technical support to prevent submissions of all VISTA and VLT OBs on the same date. 2.3 IP IP was delivered to USD and is almost ready for release to the users (pending P2PP release). Most of the change requests for EVM/ETRM were implemented. Still pending is the change request for ETRM regarding AO priorities. TSZ states that this is not critical. The new waivers for multiple filters in the same template and for AO priority HIGH in case seeing<1.0" were implemented. ===================================================== ======= 3. QC issues, status of VISTA data reprocessing ============ ====================================================== 3.1 PERSISTENCE WHU analyzed persistence frames and sent report to USD and Paranal for feedback. The decay parameter was computed, but it was pointed out that for the users it is more important to get amplitude. Next steps: - WHU will update the report in 2 steps a) change the results from ADU/300sec to ADU/sec b) correct in the second saturation decay the values from the persistence coming from the first saturation event. This is not clear how to do. Alternative is to ignore the second saturation event since this was taken only to confirm that resets cannot remove the persistence. Essentially after saturating the detector there were 100 resets made, but this made no difference for the persistence amplitude. New AI: WHU will distribute the updated report to the whole IOT. New AI: VIV put in the User manual persistence amplitude and slope. Add to the instrument page the persistence report. Discussion on the effects of consecutive saturations: can they be handled independently? and how much persistence there is at different levels of saturation? After the discussion between TSZ, WHU, MIR and VIV the conclusion is that persistence depends on the level of saturation and it is impossible to predict it. The report and the measurements give one example for the level of 10000 ADU saturation. The users should be warned and given advice how to best avoid it. 3.2. VALIDITY OF CALIBRATIONS: TWILIGHT FLATS PROBLEM IN JUNE? In the mid of June it seems that the H-band twilight flat was outdated. There are large scale gradients that could not be taken out. MHI asks whether we should wait for CASU results to see if it is necessary to re-classify the observations. Flats were taken with 2 weeks difference from the observations. Before that there was intervention that affected the optics. One problem with the flat fielding is the fast variation in striping pattern for some detectors, but this typically can be corrected by sky subtraction. In this one particular case the residuals are bad. SMI checks the opslogs scoring for these two detectors and finds them ok. Based on the rms ratio of the twilight flats there is no single detector that is outside of 2% value. WHU points out that we need to get a better monitoring of twilight flats. In CASU flats are updated every month. MIR reports that they take advantage of the fact that the observations are taken after the observations. Currently J. Lewis has paused with processing in order to get the full set of calibrations for May/June (after the first intervention when shim was placed) and to catch up with the re-processing of the data. TSZ reports that on few nights there were some darks missing. Problem was identified due to the fact that the calob script was not running by default in the data directory, but was started from a wrong directory. This should be made more robust. He informed the operations accordingly. 3.3 REPROCESSING OF DATA: MIR reports that the re-processing is going well. The new version will be v1.0. It includes creating tiles for each night and all data are processed from October to March period. Eduardo is putting them in place for users. SV and Survey users should get data until processed until end of March. This includes tuning of the sky subtraction for VVV - a different method was implemented on request specifically for VVV. At the moment CASU is waiting for good flats for June. They should be ok for April and May - therefore should be more or less up to date with processing soon. The improvement with respect to the last version is hard to spot. Mainly it concerns the rejection of small artifacts and that includes persistence. Main changes are generating tile products for every night. VIV asks about the status of image subtraction and differential image analysis, but MIR replies that this is done by Edinburgh and JEM says to contact the VVV PI for that (Minniti). ===================================================== ======= 4. Survey progress ============ ====================================================== MRE reported that at the last IOT. Since then very little was done, due to bad weather and the intervention in between. Therefore no new plots/statistics are produced. UltraVISTA is almost done - only 4 OBs remain in the queue, but the longer OBs cannot any more be completed. ============================================================== ======= 5. Survey progress vs. simulations and feedback from the PIs ==== =============================================================== VIV re-run simulations with expanded downtime and amount of idle time in July and Aug shrunk. He estimates that we are mostly safe for this gap that the simulations are predicting. TSZ points out that we are in a bit of difficulty because we used lots of THN, bad seeing observations. Not much that can be done about it - we used most of the bad weather tail from distribution. MRE reports on the pre-emptive complaint from UltraVISTA team - essentially, even before getting information on the time allocation, they asked quite strongly to get more than what was in the SMP in order to compensate for the less than expected allocation & observation in the first season. For the time allocation it was essentially decided to give to the PIs what they asked in the SMPs for P86. GHU reports that from the comparison of MRE plots (completion) and VIV plots (expectation from simulations) the simulations are quite realistic. She waits from Gautier for the confirmation of the allocation & then the information can be sent to the PIs. ===================================================== ======= 6. Action Items review ============ ====================================================== TSZ points out that the action item needs to be revised and cleaned. The old, closed action items (closed before this meeting) are therefore NOT reported here. See the agenda circulated by e-mail for these. OLD AIs: -------- AI 2009-11: TSz should find out if the constraints sets could be carried with the OBs, to the OS and the fits header of the VIRCAM files. Otherwise it is not easy to propagate these data by interfaces. 2009-07-29: TSz iterating with R. Schmutzer, but DFI (T. Bierwirth) will also need to be involved. For surveys, given the small range of programs it is expected that there are not too many different OBs, and this might need to be followed in a different/manual way in the beginning - this could imply a major change for the tools that cannot be done in the last moment. 2010-03-23: the keywords have been implemented except for the moon; there is a ticket with details; some testing done by VDI, no errors found; keep ongoing, try to complete during my next shift end of Mar - early Apr. 2010-04-23: VDI reports one formula is missing 2010-05-27: Corresponding PPRS is still pending. VIV asks to keep this AI open until the PPRS is closed. 2010-06-29: No changes with respect to last meeting. VIV asks to keep this AI kept open/pending until the PPRS is closed. STATUS: PENDING ** AI 2009-14: GHu to check that the seeing distribution function in the scheduler for VIRCAM takes into account the finite (0.51") nominal resolution of the instrument. 2010-03-23: GHu reported that it is not clear if TaToo will be used for scheduling time on VISTA; decided to keep this AI open until it becomes clear how VISTA time will be scheduled. 2010-04-23 no news 2010-05-27: TSZ states that for the scheduling the seeing distribution is _not_ considered in the requested band. MRE notes that there may be some chance that this is updated by ESO. 2010-06-29: GHU reports that Gautier Mathys is reluctant to make these changes if the open time is scheduled in the specific time slots. We need to formalize and few weeks the most efficient way to do scheduling. For the surveys this is with the VIV simulation rather than with TaToo. VIV states that he will be on Paranal 2 weeks. He points out that it is either necessary to re-create the database for the next semester based on SMPs or to wait for the PIs to submit the OBs. GHU says that we cannot wait for the PIs and this should be discussed. Gautier needs to be involved in this decision as well. Best is to discussed by phone next week. Decision on how to do scheduling for surveys + open time has to be taken before the next IOT. VIV points out that he would like to have it done before going down on July 10. STATUS: PENDING ** AI 2010-03: completion rate of the surveys to be compared with the new simulation of scheduling which should now include real submitted OBs. This comparison should be used to review the ranking and decide if it is necessary to revise it for the future. MRe to provide completion rate until May 1. GHu+VDI to run a new simulation with realistic OB database. Deadline for this AI is May IOT. 2010-04-23: no news 2010-05-27: simulation to be redone including technical time loss. 2010-06-29: DONE STATUS: CLOSED ** AI 2010-06: TSz+VDI to improve the TIO training, to prepare an operational manual 2010-05-27: Operations wiki has been updated. More training needed. 2010-06-29: Not much was done due to problems with IQ + intervention + work on IP. Keep the AI open. STATUS: OPEN ** AI 2010-07: JEm to fix the SADT 2010-04-23: JEm will fix the most critical items on the bug list, but does not guarantee to fix items that go beyond the original requirements. Some critical items are to be discussed, like the control of parameter values that might be manually edited by the user. The PA bug was identified and will be fixed. The new catalog issue on UCAC-3 isn't solved yet. The tool will be delivered by the end of next week. Ancillary material (documentation) will be distributed. 2010-05-27: Ongoing. New corrected version of SADT released, testing by ESO staff until June 8. Summary of corresponding discussion: Jim Emerson notes that tile2 is still not treated properly. It is commonly decided to remove this from the IP for P86. SADT also now reads UCAC3. He states that the remaining issue with proper motion are an issue of the catalogue, not SADT. He furthermore suggests to spend a night of engineering time to check well what are the correct suitable magnitude ranges for Technical / Guiding chip. Keeping the same allowed magnitude range at a different catalogue (UCAC3) may not necessarily help. Thomas Szeifert adds that there can be very high magnitude uncertainties in the catalogues. For Ultravista the problem is that possible guide stars are generally on the brightness limit. Marina Rejkuba inquires whether the UCAC3 will help Ultravista. Jim Emerson replies that there are less stars in UCAC3 than in GSC. Jim Emerson also asks to send comments/objections to the modifications done to SADT, which are summarised in his recent e-mail. Michael Hilker ask Valentin Ivanov to do SADT tests on the sky during the next week or so. He will try to do this, together with the IP test. Michael Hilker asks for comments regarding SADT until June 8. 2010-06-29: TSZ points out that this is not an action point for the IOT. In any case the new version of SADT for P86 Phase 2 was delivered and was tested. MHI adds that the final version of the SADT is agreed to be delivered on August 1, 2010. STATUS: CLOSED ** AI 2010-08: VDI to take care of the new IP by May 2010-04-23: will be done 2010-05-27: IP prepared, test is still pending in combination with new SADT. 2010-06-29: IP delivered STATUS: CLOSED ** AI 2010-09: photometric zeropoints as calculated by the pipeline. 2010-05-27: Ongoing, see corresponding discussion this meeting 2010-06-29: TSZ points out that this is not well defined AI. MIR adds that the currently existing plots already would have shown the problems with zero points from pawprints to tiles. It is not clear whether this is about the zero points from Garching pipeline? WHU states that Garching pipeline has de-blending on. MIR: There is no significant increase in the scatter with respect to what one would expect from the random noise. It is agreed in the end to send one example data set and then the comparison should be done directly between Garching and CASU. STATUS: PENDING ** AI 2010-10 WHu: It should be investigated, if the quartile of the ellipticity can be used as a threshold for OB rejection. 2010-05-27 Ongoing, to be investigated after new de-blending option 2010-06-29: ellipticity was discussed in the beginning of the meeting. The situation has improved and VIV suggests to close this AI. STATUS: CLOSED ** NEW AIs: AI 2010-11: WHU to distribute the persistence report to the whole IOT. AI 2010-12: VIV put in the User manual persistence amplitude and slope. Add to the instrument page the persistence report. ======================================= ============== 7. Other issues ===================== ======================================= 7.1. Standard star observations validity MRE asks how is the standard star classification done. Essentially the question comes after the discussion with J. Lewis. This is triggered by the question what to put in the standard star headers (valid/non-valid observation) that are distributed to the users. She asks whether it would be possible to mark in some way when the standard star is taken, but then the observer decides that this observation should not be used. MIR says that for the processing this does not matter, because for photometry science data are used, not only standards. TSZ says that the standards are to be taken only when the weather is clear or photometric. In other words whenever the calibration is taken then they should be considered as valid. Sometimes the standards are taken when there are clouds, but than that should be considered as human mistake... It should be re-iterated in the training to observers that standards should not be taken when there are clouds. 7.2. P87 info on VISTA TSZ: The text for the Call for Proposals is ready since the last period. GHU says that we'll need some info for the management about how much time is available for normal/short programmes on VISTA. She will discuss this with various groups offline - this does not need to be within the IOT meeting. There should be about 20 nights available. ======================================= ============== 8. Date of the next meeting ===================== ======================================= TUESDAY, 24. August, 2010 9:00 Chile, 15:00 Garching