1. INTRODUCTION
SAC has recommended that CFH12k be operated in a queue scheduled/ service observing (QS/SO) mode beginning in Semester 99II. This document presents an overview of the proposed operational model for the queue scheduling/service observing (QS/SO) mode to be implemented for CFH12k. Another relevant document has been published by T. Abbott and D. Crabtree on the proposed queue-scheduling rules and will only be summarized here. At this point, details for the implementation of specific operational issues have been developed and actions and resources needed to carry them out have been identified. For other issues (e.g., the calibration plan and the reduction pipeline), details will be better defined during the next semester after commissioning of the camera. Experience gathered with CFH12k during that period will be extremely valuable to evaluate the model proposed and test the diverse algorithms.
Queue scheduling/service observing (QS/SO) mode was primarily designed to increase the scientific productivity of telescope time and not to reduce the cost of the telescope operations. This document includes some rough estimates on the resources and manpower needed to fully implement an affordable and highly efficient model for QS/SO with CFH12k. Readers should keep in mind, however, that even for an imager QS/SO mode is a complicated process. Changes to the model described here are to be expected during its implementation.
2. OUTLINE OF THE QS/SO MODEL
The operational model proposed can be divided in a few specific steps which are more extensively discussed in the following sections:
ii) Queue Execution: Prior and during each observing night, a list of observing blocks is assembled to optimize science productivity according to the weather conditions and requested specifications. This list is derived from the database using sophisticated software, the scheduler, evaluating the priority of each observing block using a weighting system. Dedicated and trained observers perform the observations, as well as any required calibrations. The management of the queue and all decisions related to the QS/SO run are made by one of the resident astronomers assigned as queue coordinator.
iii) Data Reduction and Evaluation: The astronomical observations are automatically pre-reduced to remove instrumental signatures using a database of relevant calibration frames. A thorough data control quality evaluation is carried out on the data and the pre-reduced data by the queue coordinator.
iv) Data Distribution and Archiving: The astronomical data (raw and/or pre-reduced) and relevant calibration frames are distributed to the investigators. The actual medium use and the manpower needed for data distribution has not been identified yet. All data will be archived at CADC.
3. THE DATABASE
An essential tool for the QS/SO mode is the database. This tool will be used for different purposes at CFHT and its development is already in progress. For QS/SO, the database will contain essentially the programs (i.e. the observing blocks) prepared by the investigators. Database management software, an electronic logbook and the scheduler will give access to the database for the control of the queue (i.e. blocks remaining for a given program, programs completed, comments on data evaluation, etc).
Basically, the observing blocks should contain the following information:
4. PROPOSAL SUBMISSION
The proposal submission will take place in two tiers. Phase I applies to all the instruments at CFHT and is a conventional proposal oriented on the scientific justification. Phase II is specific to CFH12k (at least for the moment) and provides all the information required to execute the observing queue.
4.1 Phase I
The first tier proposal is based on the current observing proposals used at CFHT with some refinements required for QS/SO with CFH12k. Its purpose is to allow the TAC to establish a scientific priority for each observing program and to provide the amount of time to be allocated for each accepted proposal. As described in Abbott & Crabtree document, the easiest way to manage the time allocated for QS/SO is to consider only the integration time (I-time) for a given program, that is, the time at the telescope under a clear dark sky with the camera shutter open. This does not include overheads associated with telescope pointing and camera readout time. We estimate that 25% of time will be lost to weather and that CFH12k will be operated at 70% efficiency, i.e. from twilight to twilight the shutter will be open 70% of the time. If the time from twilight to twilight is 10 hours, this translates to an average of 5.35 hours of open shutter time for each night. A program requiring 16 hours of I-time will need 3 nights to be completed on average.
After its evaluation, the TAC must provide some information that will be used for Phase II preparation and during the queue execution:
4.2 Phase II
The second tier proposal addresses directly the information required for each observing program. From an observing proposal form especially designed for Phase II and provided by CFHT, the investigators can specify individual or group of exposures (observing blocks). The set of pre-defined keywords/value pairs assembled from this form (see section 3 for the list) is then sent to the QS/SO database and constitutes the specifications examined by the scheduling software for the prioritized list of programs.
The Phase II proposal form will have the following characteristics:
5. QUEUE EXECUTION
5.1) Queue execution rules
The QS/SO observations will be executed according to specific rules. The algorithms sorting out the observing blocks from the queue database (see below) will automatically apply these rules. The decision trees are explicitly described in Abbott & Crabtree document and are only summarized here:
Optimizing science productivity being the ultimate goal of the QS/SO mode, it is necessary to introduce a sophisticated algorithm to prioritize the diverse programs available in the database. The development of this software, the scheduler, is one of the most challenging tasks of all QS/SO programs. Basically the scheduler assembles a list of observing blocks to be executed according to a set of constraints evolving with time. The scheduler will also be able to produce a suggested list of observation blocks given the LST, seeing, sky brightness and transparency. The priority of each program is constantly evaluated during a QS/SO run (and for during a given night) from a weight system according to the specifications requested by the investigators, TAC evaluation and the actual sky conditions. The following constraints will be included in the scheduler algorithm:
5.3) Calibration Plan
An important element of QS/SO mode is the calibration plan, that is, a set of procedures applied regularly through a queued observing run to gather the necessary calibration data. For CFH12k, the plan is divided in two parts:
1) Instrument and CCD Maintenance Plan:
In QS/SO, it is crucial to monitor the status of the instrument and the detectors on a regular basis. For instance, calibration ``master frames'' for the data reduction can only become usable when the stability of the CCDs is known. At the beginning of each QS/SO run, a set of CCD calibration frames (biases, darks, flat-fields) will be obtained and included in the database for the upcoming observations. All these frames will also be processed automatically to analyze the characteristics of the CCDs (readout noise, cosmetics, etc).
2) Astronomical Calibrations:
In order that the data obtained with CFH12k in QS/SO mode useful for archival research, we plan on observing every field, in each filter requested, during at least one photometric night. If a night is photometric we will take short exposures of as many uncalibrated fields as possible, as well as photometric standards. These exposures can then be used to calibrate the data obtained for the scientific program. This astronomical calibration plan will be executed as an independent observing program in the queue. It will have the highest rank among the other scheduled observations but will not consumed any I-time allocated to these programs by the TAC. A set of photometric standards will be observed each night the sky is photometric. The general plan will be published before the deadline for the Phase II proposal submission. If specific calibrations are required for certain programs, the investigators will have to include them in their program and I-time will then be accounted. The main strategy for the all the astronomical programs requiring photometry will be to obtain at least one exposure in the desired filter for each field of the program during a photometric night so that subsequent exposures not done under good conditions can be calibrated.
J.C. Cuillandre and P. Martin will develop the full calibration plan after the commissioning of CH12k and with the experience acquired with the camera during the 1999I. A detector specialist supervised by the resident astronomer responsible for CFH12k will carry out the maintenance plan. The astronomical calibration will be performed by the designed queue observers and reviewed by the queue coordinator (see below).
5.4) Observations
Operations in a QS/SO run extend beyond the normal hours associated with observing. The database and the queue need to be prepared prior to each observing night, the data quality from the previous night must be evaluated and data from the observations have to be gathered. A team of individuals is needed to carry out these tasks:
1a) The QS/SO coordinator is stationed in Waimea and is responsible for maintaining the database, evaluating the data, overseeing the archiving and any contact with the investigators. He has the final authority involving the feasibility of programs already started and returning observing blocks to the queue if they do not meet the specifications. This role is assigned to one of the resident astronomers.
2a) The Observer, based in either Waimea or at the summit, is the one primarily responsible for operating the instrument and data acquisition system. He is also responsible for running the queue and the quick-look reduction pipeline. He is the authority in the decision process affecting which observing block should be executed or not. If Waimea-based, he is in constant contact with the summit observer via audio/video. This role is filled by one of the resident astronomers and/or an internal observer specifically trained for QS/SO (post-doc, graduate student, hired QS/SO observer).
4) The CFHT director represents the final authority in the case of conflicts between the queue coordinator and the investigators.
Although this model seems to be the most efficient alternative to carry out observations in the QS/SO mode with CFH12k, it introduces a few logistic difficulties. First, there is a clear manpower problem for the coordinator and Waimea observer positions. It's unrealistic to assume that on the long-term, both positions should be assigned to two CFHT resident astronomers. For instance, in 1999I, 80 nights have been assigned for CFH12k observations. To cover their other duties and assuming the role of coordinator and/or observer at the same time is not possible. CFHT is actually considering the possibility to hire post-docs for which an important fraction of their time could be spent as a QS/SO. Another solution is to hire a specifically trained observer for the task. Second, there is a policy at Mauna Kea requiring that two persons should be present in the dome during an observing night at all time for security reasons. With the model above, this condition is not filled. We are discussing this issue actively with other observatories at the summit to find a solution. A security program with well-defined actions in the context that the summit observer is in constant communication by video/audio link with the Waimea summit could certainly be developed. Finally, an observing room has to be installed at the Waimea headquarters, the actual computer room being very inadequate. We have already identified the location and the resources necessary to make this room available during the 1999I semester.
6. DATA EVALUATION AND REDUCTION
A thorough control of the data quality must be performed during a QS/SO run to optimize the science productivity of the telescope time. The data quality will be evaluated through many channels with the queue coordinator having the final responsibility for deciding if an observation meets specification or should be repeated. For a given observation, the following process is proposed:
7. DATA HANDLING AND ARCHIVING
The amount of data generated with CFH12k ( 200 Mbytes/image) will be very large. Copies of the data will only be kept at CADC, as CFH does not have the appropriate resources. However, CFHT will be responsible for distributing the data to the investigators. This is a process for which the complexity cannot be overestimated. At the moment, data distribution introduces many unanswered questions for the QS/SO mode with CFH12k at CFHT. It is still unclear how the data will be distributed due to the lack of manpower required to take care of this task if physical devices (e.g. DLTs) are used. Compressed data could also be transferred by fast Internet access but this is not yet available.
Some other issues have been addressed but comments and suggestions from the SAC will be particularly useful at this point:
8. OBSERVING TIME ACCOUNTING
It is important that the accounting of observing time between the three Agencies, and Discretionary time, be properly handled in QS/SO mode. There are two points of concern. First, the number of hours of observation carried out for each Agency must be close to that expected. The expected number of hours is the number of nights allocated to QS/SO by the Agency multiplied by average length of a night and the expected overall efficiency, including time lost to weather. For example, if NRC allocates 30 nights to QS/SO, and with an average night length of 10 hours and overall efficiency of 53.5%, then they expect approximately 157.5 hours of exposure time on Canadian programs. No one Agency should benefit to a large degree at the expense of another Agency.
The second important point is the amount time actually charged to each Agency. The number of nights allocated to each Agency during any one Semester no longer needs to conform to the nominal 42.5/42.5/15 distribution. This flexibility will also extend to the QS/SO time. The total exposure time for each Agency will be calculated at the end of each Semester. Each Agency will be charged the number of nights that their fraction of the total exposure time represents. For example, if Canada's total exposure time was 44.3% of the total for the semester, and there were 66 nights allocated for QS/SO, then Canada would be charged for 29.24 nights. This number would be added to the number of nights scheduled traditionally to determine Canada's allocation for the Semester. Canada's initial allocation for the next Semester would be adjusted to approach the long-term average of 42.5%.
9. SCHEDULE
It is expected that most of the observations performed with CFH12k in
the 1999II semester will be performed in this mode. The following Table
gives a rough estimate of the schedule expected for the implementation
of the QS/SO mode with CFH12k.
Item |
|
|
Database Development |
|
|
Database software |
|
|
Phase I Proposal Web-form |
|
|
Phase II Proposal Tool |
|
|
Queue scheduler |
|
|
Calibration Plan |
|
|
Data evaluation tools |
|
|
Data Reduction Pipeline |
|
|
Waimea observing room |
|
|
Simulation/Testing |
|
|