In semester 1999II, CFHT will begin a queued observing programme using
the CFH12k 8192
12288 pixel mosaic camera with the goal of
optimising the science returns based on TAC rankings. At any given
time, only the most highly TAC-ranked programmes that are compatible
with the current observing conditions will be executed. Thus the most
highly ranked programmes are also those most likely to be completed.
We describe an overview of the programme and its required
infrastructure.
409615
pixels arranged in
a 6x2 mosaic producing a total imaging area of 8,192x12,288 pixels.
CFHT has found the UH8k mosaic camera to be exceedingly popular with
proposing observers. We anticipate a higher demand for the more
sensitive CFH12k and the latest semester's proposals reflect this.
Investigators tend to propose programmes which make strong demands on
the capabilities of the instrument. In particular, the better seeing
available at CFHT is often required to achieve the aims of the
proposal. Clearly, this is not always obtained and highly ranked
proposals may suffer as a result of poor weather conditions whereas
lower ranked, but scheduled, programmes may be completed in a run of
good weather. Overall, this does not optimally apply the scientific
priorities set to CFHT by the Telescope Allocation Committees (TAC),
and does not guarantee that CFHT achieves the best science output.
Consequently, the Science Advisory Committee requested a study and
have now approved a queued observing programme for the CFH12k for
semester 1999II.
In queued observing, the most highly ranked proposal appropriate to current observing conditions is selected ``on the fly'' and is pursued while these conditions are maintained or until the programme is completed. When conditions degrade, the observer moves on to a programme with less stringent requirements. When a programme is completed, another of lower rank, but requiring the same conditions, takes its place. Thus, in the event that seeing improves, a highly ranked programme requiring exceptionally good seeing may interrupt a lower-ranked programme which requires less exceptional seeing. Work will proceed on the higher ranked programme until seeing degrades again, the target objects set, or it is completed. Depending on the programme rules adopted, it is also possible that a lower-ranked programme, requiring excellent seeing, may interrupt a higher ranked programme when the seeing improves, particularly if the former is nearer completion than the latter. However, programme ranking will necessarily determine which of the feasible programmes is started next when an opportunity arises.
The queued observing mode is intended to replace the classical mode in which programmes are granted fixed allotments of time and are vulnerable to poor weather conditions, instrument failure and other effects which cause partial or complete loss of the run. Queued observing will result in a greater fraction of successfully completed programmes by matching programme requirements with current conditions. Secondarily, the more highly ranked programmes among those which require the same observing conditions will be more likely to be among those completed.
Since CFHT's imaging quality is so high, frequently achieving images at prime focus of 0.5" FWHM, queued observing is also a method of maximising the scientific return from such valuable conditions because any given programme will be executed in seeing which is only as good as required and no better (or worse).
Observing at CFHT will be rendered more efficient by queued observing. Under classical observing it is possible, even likely, that two programmes, scheduled back-to-back, will obtain identical calibration observations, thus wasting telescope time which could be used to scientific advantage. Queued observing will eliminate this wastage. Furthermore, calibrations obtained in queued observing will be better, on average, than in classical observing: the data will always be obtained by an experienced observer, more images from a wider variety of fields can go into a superflat, and greater numbers of different calibration images will be available to each investigator. Finally, we will obtain better internal consistency of all data.
The following sections describe the proposed form of the CFH12k queued observing programme, including: a new proposal format, how calibration data will be collected, the factors affecting queue execution, the queue execution rules, and some expected consequences and policies. This document is abstracted from a complete preliminary specification which may be found on the CFHT web pages here
2.1 First Tier Proposals
The first tier proposal will be based on the current observing proposals used at CFHT, but with refinements required for queued observing. The purpose of the first tier proposal is to establish a scientific priority for each observing programme and to set guidelines for the amount of I-time which should be allocated to the programme. (I-time is defined as time at the telescope under a dark sky with the camera shutter open. This does not include overhead associated with telescope pointing, camera readout time, etc.) Investigators will be asked to provide all the usual information required by CFHT proposals, including a general scientific justification for the programme.
Based on this information, the TAC will then provide:
1) The maximum total I-time to be expended for each proposal.
2) The relative rank of each accepted proposal (to be
normalised to unity for each TAC)
3) The number of nights to be allocated to the queued observing
programme from the given TAC's allotment of available time.
Supporting information in the form of an estimate of observing efficiency and the typical distribution of observing conditions will be provided to investigators and the TAC.
2.2 Second Tier Proposals
The second tier proposal is an explicit statement of each accepted programme's needs. This takes the form of a set of keyword/value pairs assembled by each investigator which completely describes the needs of his or her programme and which defines the use of the I-time as allocated by the TAC. CFHT will provide the mechanism by which the investigator will compile the required information - a set of web forms and/or ASCII templates. CFHT will also develop a database engine which will parse the provided information into a database on the front end and will be capable of identifying, from this database, the small set of optimum observations to be taken at any given time once the observing queue is started. The submission mechanism will allow for the grouping of observations when appropriate.
The investigator may specify that some exposures, of the same field, be obtained together. This is to avoid the circumstance by which one programme interrupts another which is already under way when the current programme requires more exposures of the same field. This will alleviate some of the inefficiency due to repointing the telescope (although exposures within a group may be dithered for purposes of generating data for a superflat). For the purposes of establishing the viability of a group, exposures listed within that group will be treated as a single exposure of total integration time equal to the sum of the individual exposure times plus their readout and instrument reconfiguration overhead. Thus a group is not started unless there is sufficient I-time remaining within the night to complete that group. This also encourages investigators to specify groups of minimum length.
The final format will be formalised to be handled automatically and will include considerable detail not described here. Nevertheless, we expect only the most general parameters will be explicitly described by the templates, any other information pertinent to the observations should be written into "Comment" fields.
The original inspiration for this scheme was the HET phase II proposal language (Gaffney, 1997, private communication, and here).
Calibration data will be obtained as part of an independent observing programme within the observing queue which will be executed according to the same rules as any other programme. This programme will be established independently of the proposals actually accepted for observation in order to ensure the integrity and isotropy of the archived data set.
To ensure its execution at the earliest possible opportunity, concommitant with available conditions, the calibration programme will have higher rank than any other scheduled programme. It will not consume I-time allocated to the individual investigators' programmes. If two programmes have overlapping calibration requirements, the investigators will share a single set of calibration data and the telescope is used with greater efficiency.
Not all calibrations requested by an investigator may be included in the calibration programme. In this case, any calibration observations extra to the combined programme will constitute part of the investigator's science programme and will consume some of its allocated I-time.
The sequence in which the queue is executed is a complex function of object location on the sky, sky conditions, degree of programme completion, TAC ranking, usable observing time remaining in the run, and current weather forecast.
4.1 Sky conditions
The required S/N for each field constrains the usable sky conditions. The signal and Poisson noise at the required limiting magnitude or target brightness is assumed to be calculable from the filters used, the CCD QE curves and readout noise, and the optical efficiency of CFHT prime focus. Dynamically changeable factors which are relevant during a run are as follows:
Sky Brightness is commonly defined by any of a number of
different methods, including lunar phase, time of night, or angular
distance to the moon, but the most straightforward and applicable is
to specify the actual, instantaneous brightness of the sky in the
target field. This is the measure that the queued observing programme
will adopt throughout. At the telescope, the instantaneous sky
brightness will be known, either because:
a) the moon is down and there is currently no twilight - sky
brightness will be assumed constant everywhere, or
b) observations have begun and the sky is the same brightness or
darker everywhere the current distance or more from the moon, or
c) an estimate of the analytical function of sky brightness with
distance from a partial moon will be defined observationally, or
d) we can go and look (if the new programme is otherwise acceptable to
begin).
The queued observing programme will therefore adopt a direct sky brightness definition (detected photons per square arcsec per second in the waveband of interest) and will not request lunar phase limits from the investigators. Investigators will be provided with an estimate of the range and distribution of such values as determined from prior experience.
Seeing. The investigator will specify the acceptable seeing for his programme according to a set of seeing bands (see Appendix 1). It is always assumed that a programme can be executed with better seeing than specified by the investigator, unless specifically stated otherwise. (Programmes will only be executed under better seeing than requested if there are no other programmes which require such conditions.)
Transparency. conditions will be set by the investigator, stated as a fraction of normal observing efficiency under photometric conditions.
4.2 Other Factors
Degree of programme completion. Once a programme has been started, it will be given preference over other programmes which have not yet been started.
TAC Ranking. Under most circumstances, the TAC ranking will be among the last parameters considered. This is intuitively obvious since object observability, sky conditions and degree of programme completion directly constrain the group of programmes which it is possible to execute. Such programmes are termed "viable" in this document (see Appendix 1). Only after this group is established will TAC ranking come into play.
Potential I-time remaining. The queue must always take into account the amount of usable observing time remaining in a run. A new programme will not be started unless there is sufficient usable I-time available in the run to complete it and all other programmes which are already under way and not yet complete. If there are no executing programmes and there is not sufficient I-time remaining in the night or run to complete a new programme, the viable, unstarted programme which may be most nearly completed in the time remaining will be started (this may be overriden, at the observer and support staff's discretion on inspection of other pending programmes).
Current weather forecast. The observer may decide not to begin a new programme, even if it is viable, if the projected observing conditions are likely to significantly reduce the maximum remaining I-time in a run. If the timescale of weather changes is within the current night, then the observer is best situated to make the necessary decisions. If the changes are expected to occur over more than one night, then decisions will be made with daytime staff in Waimea.
The following criteria must be met by a programme group for it to be considered viable for observation:
After a programme group is completed, or at the beginning of observations for the night, the queue database engine performs the following tasks:
A. Add up total I-time & overhead still required by started
programmes.
B. Compile complete list of currently viable programmes,
ordered according to TAC ranking.
These operations may be performed entirely automatically. Therefore, there is no reason not to repeat them after every exposure, thus providing the observer with a constant update on the state of the queue according to current conditions. The observer may also force a recalculation if, for example, the seeing degrades, tranparency improves, etc. or if he wishes to anticipate the next exposure, he may refresh the queue during the current exposure on the assumption that this exposure completes successfully.
One of the following three condition/response combinations will then apply:
Thus, if a started programme becomes viable while another programme of lower rank is running, the current observation group for the executing programme is completed and then the higher ranked programme takes over.
The observer will be provided with list of observation groups to execute as each observation group is completed ordered according to TAC ranking and following the above rules. Normally, the observation group at the top of this list will be executed next, but the observer is presumed to be best able to assess the current observing conditions and observations made so far and may therefore override this choice.
Observers and Manpower: Investigators will not be able to make their own observations, since there is no guarantee that it will be possible to obtain their observations within any given short time window. Moreover, since the observer will participate in the choice of programme to execute at any given time, he or she must be impartial to the aims of the programmes themselves.
Depending on how much time is allocated to the queued observing programme CFHT may not have sufficient resources in-house to carry it out alone. Suitably qualified observers will need to be recruited from the participating communities.
Either way, CFHT itself will incur a greater cost per observing run, in terms of an increased need for time and energy input by the astronomy department. Since the current resident astronomers are already working at full capacity, the implication is that the community will need to fund further positions. This expense will be compensated by, for example, fewer intercontinental airfares.
Proposal Preparation: Investigators must be prepared to put significantly greater effort into proposal preparation than is currently the case, defining the sequence of required observations and calibrations and necessary conditions for each, so that the observer has all the information needed to execute the programme. Note that it is not in the investigator's interest to request better conditions than he or she needs, since this will make it less likely that the observations will be performed.
Queued versus Classical Observations: We strongly advise, as far as possible, the TACs allow only queued observing with the CFH12k in the semesters in which it is offered. This is because the queued observing paradigm works best if there is a large pool of observations from which to draw, with a wide range of allowable observing conditions.
In any case, if classical observations are to be made, they must be entirely independent, in all aspects, from the queued observing programme. If calibrations can be shared, this is normally a matter for negotiation ex post facto since the degree of completion of the queued observations cannot be established a priori.
Time critical observations and target-of-opportunity observations do not easily fit within the queued observing paradigm and should be obtained in a classical mode which overrides any queued programme.
Failed Observations: If a mistake is made by the observer, or if there is an instrument failure, the failed observation does not consume I-time allocated to the programme concerned. (That is, a failed observation will be immediately repeated unless it is no longer viable).
Tier 2 Proposal Resubmission: After a programme has begun, an investigator may inspect her data and may find that her Tier 2 proposal did not anticipate circumstances as experienced. In this event, the investigator may change and resubmit her Tier 2 proposal. This must be done before sunset and with sufficient time for the queued observing team to examine any changes. Under these circumstances, any I-time already expended on this programme is considered lost and the investigator may rearrange only what time remains allocated to her programme. Similarly, a programme may be suspended to prevent the waste of further I-time while the investigator considers her options.
Programme Termination: The programme will terminate when the maximum allocated I-time has been consumed unless there are clear reasons for continuing, according to the discretion of the observer within a night and that of the queued observing team and the Director of CFHT if further nights are involved. Under no other circumstances may a programme consume more I-time than is allocated by the TAC.
Incomplete Programmes and Non-Starters: Programmes that are started, but not completed at the end of a run or semester may be carried over into the next run or semester. This process is not automatic, but will be subject to review and dependent on the degree of completion of the relevent programmes. For example, the proposals may be resubmitted to the TAC to be separately reviewed for continuation.
Programmes scheduled, but not started in a given semester will be returned to the TAC for reassessment. (The alternative is to carry over all incomplete or unstarted programmes, but such proposals will have to be re-ranked against newly submitted proposals anyway.)
