Queued Service Observations with WIRCam Phase 1 Proposal Submission for Semester 2010A Updated 08/24/2009

A - Introduction

• The QSO Project
• Document Outline
• Two important notes! One on non-sidereal tracking!

• Programs: Q or not Q?
• Types of QSO Programs
• Proposal Submission Phases

• Night Reports
• Data Evaluation
• Data Distribution
• Proprietary Period
• The QSO Team

A - Introduction [Back to Table of Content]

(1) The QSO Project

The main concept behind the queue observation scheme with WIRCam is to perform observation programs only during sky conditions required to meet their science goals, as defined by the investigators. This can only be achieved if the programs are all grouped together in a database and are selected appropriately according to a set of constraints, rules and sky conditions. Programs are then carried out by a well trained, local team of observers in a service mode (i.e. investigators are not present at the observatory).

During 1999, CFHT has started a project to implement the necessary software and to review all the issues for achieving a queue/service observing mode with its CFH12K mosaic camera. This Queued Service Observations (QSO) Project has been developed in parallel to other projects necessary for the data acquisition (NEO), processing and analysis (Elixir), and archiving and distribution (DADS). The necessary software tools for proposal submission (Phase 2), selection of programs, management of the observations and execution of the observations have all been developed within the QSO Project. Most of these software components are for internal use only except for two obvious exceptions: Poopsy, proposals submission tool developed and maintained at CADC, and PH2, a Web based tool developed and maintained by CFHT for the second tier of proposal submission (see below).

The QSO mode has been used for CFH12K for over 200 nights between January 2001 and January 2003. Since the semester 2003A, MegaCam has been operated in the queue mode. Observations for WIRCam and ESPaDOnS also have been conducted under QSO since 2005 and 2008 respectively. WIRCam is exclusively offered in the queued service observing mode and observations are conducted following the same process as with MegaPrime, with the general architecture of QSO (and NOP) being modified to accommodate those observations.

This tutorial was written as a guide for helping in the preparation of the proposal submission for the QSO mode with WIRCam. At the time of writing, WIRCam has been fully operated for several semesters under QSO which resulted in large amounts of data collected. At the end of 2007A, improvements in the electronics of the camera resulted in much cleaner cosmetics. Except for some minor technical changes, we offer WIRCam with the same modes and options as in the previous semesters.

For technical information, please refer to the WIRCam page.

(2) Document Outline

This document presents the information for submitting a QSO proposal with WIRCam. A complete description of the submission process with Poopsy and an outline of what will have to be done following the TAC evaluation for the second phase, planned for Nov or Dec 2009 , are included. A few QSO Rules used for the selection of the programs are presented and some other issues related to the QSO programs are also discussed.

(3) Two Important Notes!

IMPORTANT NOTE 1 : Non-sidereal guiding option with QSO WIRCam is not available (and it looks impossible to implement for now). We have however implemented non sidereal tracking (i.e., following a target with non-sidereal rates, but without guiding on stars). If you have questions or concerns related to this option for your 2010A proposal, please do not hesitate to contact the QSO Team (qsoteam -=at=- cfht.hawaii.edu).

IMPORTANT NOTE 2 : All the 10 filters are offered for a given semester for WIRCam. However, a maximum number of 8 filters can be mounted in the WIRCam wheel for an entire semester. The dewar cannot be opened more than once during a semester (this is a big operation and arrays are guaranteed for a limited number of cycles). In case of conflict (too many filters requested for a given semester so conflict between programs), the SAC TAC will make the decision and might be led to reject a program.

For more information about the submission of your WIRCam QSO proposals, contact the QSO Team at qsoteam -=at=- cfht.hawaii.edu

B - Applying for WIRCam Time [Back to Table of Content]

(1) Programs: Q or not Q?

The Scientific Advisory Committee (SAC) has recommended that the QSO mode to be the ONLY mode of operation for the WIRCam mosaic camera. So, the classical mode is NOT offered for WIRCam.

Observation of Solar System targets through the QSO system is possible through ephemeris entries in the Phase 2 Tool, and we offer non-sidereal tracking (not guiding) for 2010A.

(2) Types of QSO Programs

Many types of programs can benefit from the queue observing mode. Programs requesting excellent or exceptional sky conditions, surveys, short- or long-term monitoring, target-of-opportunity programs, are all well suited for the QSO mode. Also, contrary to the classical mode, it is now possible to submit very short programs necessitating only a few hours of observations. Programs scientifically valuable during bad sky conditions are also possible. During the submission phase with Poopsy (Phase 1), you are asked to specify what type of programs you are submitting for the QSO mode. These types of programs are defined below:

• Regular: This category includes all of the normal QSO programs, with fixed or Solar System objects. For all these programs, the Phase 2 must be completed before a specific deadline prior to the semester with QSO periods. Only investigators with Solar System targets or other restrictive constraints might be allowed to modify their observations during the observing semester.


• Target-of-opportunity (TOO): For this type of programs, the targets are not known before the beginning of the semester. In fact, the TOO programs are divided into two distinct categories: (1) Programs approved by TAC during the evaluation of all the proposals but for which the targets will be identified during the semester (e.g. gamma-ray bursts). For these programs, the investigators will have the possibility to define their targets using the appropriate tools developed for submission of the queue observations during the semester. (2) Programs submitted during a semester to observe unpredictable objects (e.g. supernova). In that case, the program must first be submitted through a special Web form (available on the QSO page) and approved according to the CFHT policy. If accepted, a certain amount of time is allocated and the information, entered during the Web submission, is used by the QSO Team to prepare and carry out the observations.


• Snapshot Programs: Since 2002A, we introduced a specific concept for a QSO "snapshot program". In a queue mode, it is very important to have programs requesting bad sky conditions and low completion level. The definition of a snapshot program is the following:
  1. A snapshot program describes valuable science to be obtained on targets observed only in the worse sky conditions (seeing larger than 1.2" (in K band) and preferably during non-photometric conditions)
  2. A snapshot program will be scientifically useful even of the completion is low
  3. A snapshot program preferably requests observations with the "standard" JHK filter set (narrow-band are possible too but are not always available)
  4. A snapshot program should preferably include short blocks of observations
  5. The time allocated for such programs is not accounted for in the statistics of time spent for the different Agencies
  6. The proprietary time for the data is 3 months following the end of the semester.
  Note that the chance of such programs to be executed, even partially, during a semester can be quite high.

For the WIRCam programs in the queued service observing mode, two submission phases are necessary. The first phase (Phase 1) is done through CADC Poopsy and consists in a general description of the program used for the evaluation by the Time Allocation Committee (TAC). This is the submission procedure for all proposals requesting time at CFHT. The second phase (Phase 2) is requested for ALL the WIRCam queue programs for which telescope time has been allocated by the TAC. As described below, it is done prior to an observing semester (with a few exceptions) through an entirely new Web based tool developed at CFHT. During this phase, all the information necessary for the local staff to perform the observations is entered by the investigators and stored in a database at CFHT.

• Phase 1: Poopsy
  The first submission process for your queue proposal must be done through CADC's Poopsy. As described in the next section, it is very important to include as much information as possible now in your proposal. In particular, please include in your Technical Justification a table with the number of hours requested per RA bin. The Poopsy database, located at CADC, is copied at CFHT and the relevant information for queue programs is parsed to the CFHT queue database. For instance, if you enter all your targets in Poopsy, you will not have to do it again during the Phase 2!


• Phase 2: PH2
  The second phase aims at gathering all the information necessary for carrying out the observations for queue programs with allocated telescope time. This phase is done using a Web based interface, specially designed and developed for this purpose at CFHT. The tool is populating a relational database which constitutes the "brain" of all of the QSO operational processes. The Phase 2 for semester 2010A should take place in Nov or Dec 2009. Investigators with programs requesting updated information during a semester (e.g. moving targets, target-of-opportunity) will be allowed access to PH2 during that semester. The Phase 2 for all the other regular programs will have to be completed by the deadline specified by CFHT. Complete information will be made available with the opening of the Phase 2 submission tool.
  

  Only investigators with approved QSO programs with WIRCam, will be given access to PH2. The user ID and password used for Poopsy will still be valid for PH2. Try to remember them!

C - Phase 1: Instructions for Proposal Submission with Poopsy [Back to Table of Content]

The current version of Poopsy allows investigators to send proposals for the queue mode for WIRCam. A section has been created which includes some information necessary for TAC evaluation, and for the QSO Team in the preparation of the queue database. Below, we review the questions related to QSO proposals as introduced in this version of Poopsy.

1 - Are you applying for a queue program with WIRCam? (Run Info Section)

As explained above, the queue mode is the only mode of observation for WIRCam. Answer "Yes" and complete the Queue Section

2 - Indicate the type of queue observing program: Regular, Target-of-Opportunity, Snapshot (Run Info section)

These three types of programs have been reviewed earlier.

3 - Indicate a global image quality (IQ) constraint (K band) describing your program ? (Run Info section)

The main objective of the queue mode is to observe targets under sky conditions required to meet the science goals defined for each program. During Phase 2, the investigators will have to define precisely observational constraints for their project. However, it is important that a global image quality for each program is defined during Phase 1 as well. An efficient queue can only be achieved if the database contains programs requesting a wide range in constraints, especially on the image quality, and the TAC will strongly consider the choice of image quality indicated here for the overall selection of the queue programs.

It is clear that the probability of a specific program depends strongly on the image quality requested versus the statistics of the site. The chance that your program is performed might be less if you request an exceptional image quality. Of course, science should be the priority in your selection of IQ but beware that it cannot be the only criterion in your choice. So, do not request a better image quality than what is really needed for your program. For guidance, the table below describes approximately the seeing statistics on Mauna Kea, in K band. Statistics reveal that seeing is highly variable during a night and globally better during the summer months (Seigar et al., 2002, SPIE, 4844, 366).

Image Quality in K Band (IQ) (airmass=1)	Frequency (%)
IQ 0.55"	20
0.55" < IQ 0.65"	25
0.65" < IQ 0.80"	20
0.80" < IQ 1.0"	15
1.0" < IQ 1.2"	15
IQ > 1.2"	5

For your information, the table below gives the average weather statistics for Mauna Kea. In general, the A semester time lost is higher than the B semester, in particular during the first few months of the winter when time lost of 50% and even more has been seen.

Important note on micro-dithering: The pixel scale in WIRCam is 0.3"/pixel so the best image quality which is adequately sampled is ~0.7". To make good use of better seeing, WIRCam uses micro-stepping (or micro-dithering): 4 individual exposures, shifted by 0.15" on a square grid, provide Nyquist sampling of the recombined image for all conditions (several groups of 4 micro-dithered exposures can be done sequentially on one dithering pattern position). In some cases this may require additional detectors readouts, and therefore introduce overheads. Note that micro-dithering is not expected to yield much improvement when the natural seeing is ~ > 0.8". NOTE: Micro-dithering is NOT offered for the narrow-band filters.

4 -How many hours are required for this queue program for this semester ? (Run Info section)

In queue mode, the time requested is in HOURS, and might or might not include overheads. You can follow these directives:

1. You must request HOURS of observations. If the total time of your program is fractional (e.g. 32.4 hr.), please indicate so (.4 hr in a queue mode is possible).
2. In your calculation of integration time, two different overheads must be calculated (that is, they will be charged at the phase 2 level):
   1. The readout overhead (reset + readout time) of the array should be included for every exposure planned. For 2010A, this is 10 seconds for the entire array (n.b.: binning is not possible with WIRCAm).
   2. If you plan to do nodding mode (i.e. target-sky-target-), an overhead of 60 seconds must be added in your calculations for each slew between the target and the sky position.
3. Slewing and acquisition of a guide star SHOULD NOT be accounted for in your calculations.
4. Detailed instructions regarding the calibrations are given in the next section.

General slewing and acquisition of guide stars SHOULD NOT be accounted for in your calculations. Detailed instructions regarding the calibrations are given in the next section.

Important Note on Maximum Exposure times: To limit the sky background and avoid severe non-linearity effects and saturation, exposure times for the filters have an upper limit. This limit is applied automatically in the Phase 2 tool. However, for purpose of correctly calculated the time needed for your observations, the maximum exposure times are given in the following table:

Important Note on Nodding: Infra-red astronomy differs from visible observations mostly because the sky background is a strong contributor and is much more variable. The near-IR sky on Mauna Kea can change by up to 10-20% in a few minutes. This is why exposure times and observing strategy must be able to frequently sample the sky. If the object is not very extended, the sky can be derived in regions of the mosaic without scientific signal for each dithering pattern position. However, when the object is extended (e.g. > 40% of the field of view of the mosaic), another technique must be used to sample the sky: nodding. The idea is simple: during the observations on the target, the telescope is slewed to a position away from the object in order to frequently established the sky background. For WIRCam, this is possible by applying regular offsets to the telescope. When calculating the time needed for your program, please take into consideration how much time will be needed to sample the sky background, if needed.

5 - How many additional HOURS would be requested to complete this project ? (General Info section)

If you are planning to submit an additional proposal for the same program for the subsequent semester (and the following, if necessary), please indicate the number of hours that might be required.

Note on Sky Brightness: The sky brightness options presented in Poopsy (i.e. position of the Moon) are different than the ones presented in PH2. For the Phase 1, these are used to help us evaluate the best periods for a scheduling a queue observing period covering as many programs as possible. More precise options, adapted for near-IR observations, will be described during the phase 2.

D - Calibrations [Back to Table of Content]

One of the main advantages of the queue mode scheme is the possibility to share calibrations between a set of programs. More so, since the queue runs are spread over several nights, the quality of the calibrations can also be greatly improved compared to the ones obtained during a short run in a classical mode. To achieve this, a calibration plan has been defined and will be carried out regularly by the queue/service observers. This plan includes the necessary "detrend" frames for removal of the instrument signatures (bias, darks, flat-fields, fringing) and the astronomical calibrations (standard stars, astrometric fields). More details can be found on the `I`iwi page.

For this semester, you can consider the following situations:

1. No programs under any circumstances are allowed to request "detrend" calibrations during Phase 1 or Phase 2. These calibrations are exclusively handled by the QSO and `I`iwi Teams.
2. Photometric calibration will be obtained by `I`iwi using the 2MASS catalog for every field observed. For observations done during non-photometric conditions, NO short exposures will be taken like we do with MegaCam. The 2MASS calibration should suffice.
3. If your program includes any filters with a broad bandpass and you prefer to obtain your own astronomical calibrations, this time must be added during the Phase 1. However, TAC will evaluate this request. Of course, the integration time will be automatically charged to the program for this kind of observations during the Phase 2.

E - Program Evaluation [Back to Table of Content]

Following the submission of your queue proposal, the TAC will review it and evaluate it. Your proposal, if accepted, will receive the following grades, ranks and allocated time. This information will be entered by us in the queue database and will be available through PH2 for the second submission phase.

• TAC Grade: Your proposal will be first classified in a "grade" category. Details are given in the table below.

• Important Note: Telescope time is scheduled for A + B programs. Programs below the normal evaluation cut-off are generally C and S programs and are considered for overfilling the queue database. There is, of course, a good chance for these programs to get some data especially if the conditions requested are realistic and take advantage of poor sky conditions. Final choice of C programs is left to the QSO Team, after revision of the actual constraints imposed by the A+B programs.
• TAC Rank: The proposals will be quantitatively ranked with respect to all the proposals included within this group and within this Agency (e.g. 2/5,3/5, etc.).
• Integration Time (I-Time): The TAC will allow a certain number of hours, the "integration time", for each approved program. The I-time is the total of the "shutter open time" + readout. During the Phase 2, this number constitutes an upper limit and is automatically taken into account by the software in the preparation of the observations.

F - DIET: The WIRCam Exposure Time Simulator [Back to Table of Content]

An important component in the preparation of the proposals during Phase 1 and 2 of all the queue mode programs instituted around the world is the availability of a exposure time simulator. Since the investigators are not present during the observations, it is crucial for the QSO observers to know that if the observations are undertaken during sky conditions requested, the science goals should be reached because the PIs have verified that using a robust exposure time simulator. It is not always easy to judge the science merit of an exposure frame and this is better accessed by quantitative evaluation.

In preparation for the QSO observation, the investigators have access to an elaborate exposure time simulator for the WIRCam mosaic. Developed first for CFH12K and MegaPrime under the supervision of Jean-Charles Cuillandre, the simulator, named DIET for Direct Imaging Exposure Time, allows the user to evaluate the integration time in different sky conditions for WIRCam. Moreover, the simulator offers different combinations of exposure time and number of exposures to minimize overheads. A Web based interface for the simulator is available. It is strongly recommended to use the simulator during Phase 1 and Phase 2 of the queue programs!

G - A Few QSO Rules [Back to Table of Content]

Maybe the most difficult task facing the queue observing model is found in the selection process leading to the execution of a science program. This selection algorithm can be based on simple criteria (e.g. mounted filters) but it becomes immensely complicated when other parameters like actual sky conditions, completeness level, science merit, monitoring constraints, or targets visibility are taken into account.

The process, resulting in the choice of a specific program to be undertaken for the queue observations with WIRCam, is done in three steps:

1. Selection: This is the first selection of the viable observations stored in the database according to instrumental constraints, sky constraints, actual sky conditions, completeness level, and the position of the targets.
2. Ordering: This second step creates a prioritized list of observations to be sequentially executed regarding their TAC grade, rank, target positions, and user's priorities.
3. Human filtering: The final step consists in the possibility for the QSO observer to modify the queue list according to special constraint like the focus sequences, calibration plan, etc.
4. Without going into too many details, each of these steps include an algorithm based on a set of observing rules. More information will be available during the release of the Phase 2 tool but some rules might be of interest already for the Phase 1 submission process. The rules given here are not presented in any order of priority. Among them:
   • As much as possible, images should not be obtained in worse IQ (or sky brightness) conditions than required. The IQ (sky brightness) measured should not exceed the upper limit of the IQ band required by more than 10-15%.
   • Images can be obtained in conditions better than required, if no other observations actually requesting these conditions are available.
   • For very bad IQ periods (>1.2"), the snapshot programs requesting bad conditions will be executed, unless other possibilities exist among the regular programs.
   • The priority of the programs started is automatically increased compared to programs not started.
   • When possible, the observations will be tentatively done with airmass smaller than 1.5.
   • During selection and ordering of the queues, the priority goes from grade A to B to C, followed by the snapshots.
   • Inside these grades, priority is given according to the TAC rank as much as possible.
   • Balancing Agency time has a very high priority and might exceed the other selection and ranking criteria.
   • As much as possible, the observer will execute the observations belonging to a given program according to the priority index (high, medium, low) given by the investigators during the Phase 2 period.
   • A QSO run should never be completed without getting all the necessary calibrations for all the programs fully or partially executed during the run.
   • No programs will be recycled for completion during the next semester.
   • When started, a monitoring program receives a higher priority so that the observations to be repeated can be carried out within the specified time frame period.
   • The QSO Team will always try to obtain the required number of observations for a given monitoring program. In case of other constraints, the minimum number of observations specified during Phase 2 is the minimum acceptable.
   • For target-of-opportunity programs submitted during the semester, these program will be subject to the same selection process and prioritization based on a grade and a rank as the regular programs.

(1) Communication and Night reports

The QSO Team is always available by email at qsoteam -=at=- cfht.hawaii.edu . To ensure prompt replies, emails have to be sent to the whole QSO Team and not to individuals! Emailing the whole team also allows all members to be aware of current issues, and to be up to date with all available information.

After each observing night, a Night Report detailing what observations were performed is available on the CFHT Web site. These reports include the observations groups executed and the sky conditions at the time of the observations. This does not mean that your data will be immediately available (see below). The goal of these reports is to inform the community of the progress of the queue and, in particular, the current status of your program.

Current global statistics for a semester are also available on the QSO Web page.

(2) Data Evaluation

As part of the data quality control assessment, all data taken will be automatically processed and calibrated by the `I`iwi Team. Data evaluation will be done in two steps: during the observation by the Service Observer ("on-line" evaluation) and, during and after the data processing. This last step is very involving and represents one of the reasons why data cannot be distributed immediately after a QSO run, unless specifically justified during the Phase 2 period. If the observations are judged satisfactory, the queue database is then updated by the Queue coordinator.

(3) Data Distribution

Data distribution will be ensured by the DADS Team. Our goal is to be able to distribute the data to the PI of each project (or another member if specified during the Phase 2) and the relevant calibrations at the period specified during the phase 2. Due to the heavy workload, it will not be possible to send the data to the investigators during a run. However, for certain types of programs (e.g. TOO) where looking at the data as soon as possible is important, this will be possible under the supervision of the Queue coordinator.

(4) Proprietary Period

By default, the proprietary period of QSO data extends to 1 year + 1 month starting at the end of the QSO semester. For instance, data taken for the 2010A semester (Feb 1 - Jul 31 2010) will have a default release date set to 08/31/2011. If an extension is requested in Poopsy and approved by TAC, a new date will be set for this program through the QSO system. The release date for the data is indicated in the fits headers by the keyword REL_DATE. For snapshot programs, the proprietary time is 3 months following the end of the semester.

(5) The QSO Team

The QSO Team members are: Nadine Manset (QSO Manager), Billy Mahoney (Database/System Programmer), Tom Vermeulen (System Programmer), and Mary Beth Laychak, Peter Forshay, Adam draginda, and Todd Burdullis as Service Observers. During a QSO run, supervision is ensured by the QSO Coordinator who, among other things, is responsible for managing the queue database, planning of the observations, and maintaining the contact with the investigators, if necessary. Observations will be conducted by the Service Observer and the Resident Astronomers, with a strong involvement by the Observing Assistants. Software support will also be provided during the observing nights. For TOO programs and decisions related to the viability of some programs, the CFHT Executive Director acts as the final authority.

Need More Information? Contact the QSO Team at qsoteam -=at=- cfht.hawaii.edu