|
Queued Service Observing with
MegaPrime, WIRCam, and ESPaDOnS:
Semester 2008B
Report
04/02/2009
|
|
TABLE OF CONTENT
A - Introduction
B - General Comments
C - Global Statistics and Overheads
D - Agency Time Accounting
E - Conclusions
A - Introduction
The Queued Service Observing (QSO) Project is
part of a larger ensemble of software components defining the New
Observing Process (NOP) which includes NEO (acquisition software),
Elixir (data analysis for MegaCam), `I`iwi (data analysis for WIRCam),
Upena (data analysis for ESPaDOnS) and DADS (data archiving and
distribution). Semester 2008B was the second semester during which three
instruments were offered in QSO mode: MegaPrime, WIRCam, and ESPaDOnS.
The main development for semester 2008B was related
to the integration of non-sidereal tracking in the QSO mode, for all 3
instruments. After the required software modifications and on-sky tests
on all 3 instruments, non-sidereal tracking is now offered for all 3
instruments. The tests confirmed that the telescope's tracking abilities
starts to degrade after 120-180 seconds. There is no plan to offer
non-sidereal guiding.
2008B was also the last semester for CFHTLS, and the
first semester with Large Programs, one on MegaCam and two on ESPaDOnS.
B - General Comments
MegaPrime
The 2008B semester for MegaPrime started with
excellent weather and science productivity, but the bad winter season
started to affect observations in Oct/Nov, and overall, about half the
time was lost to weather! The only major technical failure was due to an
electronics board failure, which cost a little over one night of time
lost.
- Technically, the entire chain of operation, QSO --> NEO
--> TCS, is efficient and robust. The time lost to the NOP
chain is negligible. The system is quite reliable and efficient.
- The QSO concept is sound. The possibility of preparing
several queues covering a wide range of possible sky conditions
(absorption and Image Quality) in advance of an observing night is
essential for imaging in the visible, where seeing can change quite a
lot, and very quickly. The ensemble of QSO tools allows also the quick
preparation of queues during an observing night for adaptation to
variable conditions, or in case of unexpected overheads. The QSO mode
also makes possible time constrained programs such as the CFHTLS. In
2008B, due to the inclement weather, the validation rate of A+B+C
programs (hours validated/hours observed) for MegaPrime is lower than
usual, at 84% (usual rates are over 90%).
- QSO is well adapted for time constrained programs.The
Phase 2 Tool allows the PIs to specify time constraints, even very
restrictive ones. Time constrained programs complicate the planning and
scheduling but CFHT provides tools for both PIs and Queue Coordinators.
- Very variable seeing and non-photometric nights represent the
worse sky conditions for the QSO mode with MegaPrime. Snapshot
programs and regular programs requesting mediocre conditions (1" to
1.2") are used to cover those conditions. Fields requesting photometric
sky conditions but originally done during non-photometric conditions are
calibrated on perfect nights. SkyProbe and real-time measurements of
the transparency are used to decide what observations should be undertaken.
WIRCam
The 2008B semester for WIRCam went very well. There
were no major loss of time due to technical problems; a couple of hours
total were lost to small software glitches with WIRCam, and another
couple of hours were lost after a fluid leak on the telescope. WIRCam
also got lucky and was the instrument the less affected by the winter weather
(36% of the time lost).
- Technically, the entire chain of operation, QSO -->
NEO --> TCS, is efficient and robust. The time lost to the
NOP chain is very small. Certain operational modes specific to WIRCam, like
nodding (target-sky-target...) and chip-to-chip dithering, have higher,
unavoidable, overheads but some of them are charged directly to PIs
during Phase 2.
- The QSO concept is sound. As with MegaPrime, the
possibility of preparing several queues covering a wide range of
possible sky conditions (absorption and Image Quality) in advance of an
observing night results in a very large fraction of the observations done
within the specifications. For WIRCam, the sky background is more of a
factor although its global variation through the night on Mauna Kea is
fairly well known. Seeing is of course another important parameter but
variations during the night in the near-IR are generally not as brutal
as in the visible.
- QSO is well adapted for time constrained programs.The
Phase 2 Tool allows the PIs to specify time constraints. We can handle
those easily if the weather is cooperative although the introduction of
time constrained observations on a large-scale adds up definitive
complexity in the scheduling process.
- Non-photometric nights represent the worse sky conditions for
the QSO mode with WIRCam. An important difficulty on near-IR
astronomy is the removal of the sky background. Non-photometric
conditions make that operation a more difficult one. Nodding for
instance cannot be done. The availability of SkyProbe and real-time
measurements of the transparency is extremely valuable and regularly
used do decide what observations should be undertaken. Also, the
real-time analysis through `I`iwi provides a direct estimate of the
extinction through the 2MASS catalog, helping even more the observing
process.
ESPaDOnS
The 2008B semester with ESPaDOnS was almost
completely free of any technical issues; only the telescope caused a
couple of hours of time lost. The last 2 runs were severely affected by
excessively snowy weather. Severe winter conditions even delayed the
installation of the instrument until the 9th day of the last
run. Overall, about 43% of the time was lost to weather, which severely
affected the success of one Large Program. Efficiency on the sky was also
affected by telescope pointing issues.
- Technically, the entire chain of operation, QSO -->
NEO --> TCS, is efficient and robust. The time lost to the
NOP chain is small. Besides telescope pointing difficulties, most of the
overheads come from acquiring (finding) the targets, performing full
telescope focus sequences for the first 1-3 nights of a run, measuring
the Image Quality, and initiating the guiding.
- The QSO concept is sound. For ESPaDOnS, and in
contrast to our imagers, seeing and extinction are much less of an issue
and do not factor much in the preparation of the queues. Queue
Coordinators usually prepare one or at most 2 queues per night. The
advantage of the QSO mode comes from the ability to schedule
observations exactly when they are needed.
- QSO is well adapted for time constrained programs.
ESPaDOnS observations are characterized by a high demand for time
constrained observations and monitoring requirements: in 2008B, almost
all programs had special requests for timing. The queues are usually
prepared by taking first into account programs with strict time
windows. It was not unusual to juggle a program with a target to observe
every night, a second program with a target to observe 2-3 times a night
with 1-hr gaps between Observing Groups, a third program with a target
to observe every 2 or 3 or 4 nights, and 2 programs to execute within a
certain time window from one another. There are also programs that
necessitates continuous blocks of 4 to 8 hours during the same
night. The PH2 tool allows PIs to specify all sorts of time constraints,
and add any comment to help the QSO Team select appropriate programs.
- ESPaDOnS can deal with non-photometric nights and bad Image
Quality Except for very faint targets (fainter than about 13)
which can be difficult to find and center with cloudy conditions or
highly degraded seeing, most observations can be carried under a very
wide range of sky conditions. Extinction and bad seeing reduce the
amount of flux getting into the instrument, but the Service Observers
compensate by repeating Observing Groups (at no cost to the PIs) to
recover some of the lost flux.
C - Global Statistics and Overheads
(1) Global Statistics
The following table presents some general numbers regarding the queue
observations for 2008B (C, F, H, L agency for MegaPrime, P agency for the
Large Programs , and T, excluding Discretionary time, engineering
time, and snapshot programs unless noted otherwise).
Parameter |
MegaPrime |
WIRCam |
ESPaDOnS |
Number of Nights (CFHTLP; calculated from the
hours allocated by TACs) | 79.3 |
44.1 |
38.7 |
Hours per night |
5.5 |
6.0 |
7.5 |
Hours lost to weather and % lost (% from Number of nights X
Hours per night) | 239hrs (49%) |
108hrs (36%) |
148hrs (43%) |
Hours used for engineering or lost to technical
problems, and % | 32hrs (6.6%) |
29hrs (9.7%) |
23hrs (6.7%) |
QSO Programs Requested |
30 (A/B/C) + 2 Snapshots |
21 (A/B/C) + 3 Snapshots |
14 (A/B/C) + 2 Snapshots |
QSO Programs Started |
29 (A/B/C) + 3 Snapshots |
21 (A/B/C) + 3 Snapshots |
14 (A/B/C) + 1 Snapshot |
QSO Programs Completed |
17 (A/B/C) + 0 Snapshot |
14 (A/B/C) + 1 Snapshot |
5 (A/B/C) + 1 Snapshot |
Total I-time allocated by the TACs (i.e., A and B
programs) |
431hrs |
259hrs |
306hrs |
Total I-time validated (for A+B+C programs) |
393hrs |
248hrs |
256hrs |
Completion rate
(A+B+C validated time / A+B allocated time) |
91% |
96% |
84% |
Queue Validation Efficiency (A+B+C time validated
/ A+B+C time observed) |
84% |
94% |
84% |
Notes concerning MegaPrime
- MegaPrime was the instrument most affected by bad weather, exactly
like during 2008A.
- The time lost technical problems was about 15 hours (board
failure). Some sky time was used to implement and test non-sidereal
tracking through QSO.
- The only program that was not started was a C program (used to
overfill the queue).
- The completion rate based on the total time validated (A+B+C
programs) and the total time allocated by the agencies and requested by
PIs (A+B) is very good at 91%. A+B programs were completed at 95% (total
time validated divided by total time requested), which is excellent,
given the bad weather! This means that the C and Snapshot programs have
low completion rates.
- The validation efficiency measures how well the QSO Team matches the
current sky conditions to the requests from PIs. The poor weather for
2008B meant that it was more difficult to observe, and to validate the
observations, which resulted in a lower than usual efficient of 84%
(numbers above 90% are more in the norm).
- The total number of hours validated during the semester is 393hrs,
which gives 393/79.3 = 5.0hrs per night of observing. The bad weather is
mostly responsible for this lower than average (5.5hrs/night) number.
Notes concerning WIRCam
- The weather was obviously better during WIRCam runs, as had also
been the case in 2008A. This is reflected in the higher number of completed
programs (15 out of 24), the overall higher completion rate, and the
higher validation efficiency. The particular set of circumstances for
2008B has conspired to make the completion of B programs higher (92%)
than for A programs (84%).
- Some sky time was used for engineering tests of non-sidereal
tracking through QSO, and for telescope pointing tests.
- The total number of hours validated during the semester is 248hrs,
which gives 248/44.1 = 5.6hrs per night of observing, lower than the
average of 6.0hrs due to the weather.
Notes concerning ESPaDOnS
This second semester with ESPaDOnS in QSO mode has confirmed the
significant differences found in 2008A in terms of queue preparation,
handling of priorities (grades/ranks), and particular scheduling
difficulties.
- The time lost to weather was high. Heavy and repeated snowfalls
often made observing impossible due to snow and ice accumulation on the
dome, or even prevented access to the summit. Unfortunately, the bad
weather hit in December and January, when one of the Large Programs had
all of its targets.
- Some sky time was used to re-measure the cross-talk, test
non-sidereal tracking through QSO, and perform telescope pointing
tests. A new triplet lens, with a new design and new optical materials,
was installed in the Fall. ESPaDOnS is the most trouble-free instrument.
- All ESPaDOnS programs except one Snapshot program were started.
- The low number of completed programs (only 6 of the 15 started
programs) and the low completion rate in general (84% overall, but only
80% for A+B programs, and 77% for A programs!) is entirely due to the
horrible weather and also to the presence of many time-constrained
programs (almost all programs): when an opportunity to observe a
target is gone (due to weather for 2008B) and the scheduled time window
has passed, there is no way to complete a program as initially
prepared. As it had already been noted for 2008A, it will probably
have to be accepted that the completion of ESPaDOnS programs will be
lower when compared to MegaPrime and WIRCam.
- The total number of hours validated during the semester is 256hrs,
which gives 256/38.7 = 6.6 hrs per night of observing, which
is still quite short of our goal of 7.5hrs per night.
(2) Overheads
MegaPrime
The following table presents the main operational overheads (that is,
other than readout time of the mosaic) with MegaPrime. These numbers
have not changed, and are given as a reference. Overheads are highly
variable during a given night depending on the conditions, complexity of
science programs, etc. The table below shows that overheads take a
maximum of ~35min per night. A short summer night lasts about 9 hours
with MegaPrime, so overheads take less than 10%. The number originally
expected was 10-15%.
Event |
Events/night |
Overhead |
Total overhead per night |
Filter Change |
~12 / night |
90s /change |
1115 seconds |
Focus Sequence |
~ 0 / night |
|
0 seconds |
Dome Rotation > 45deg |
5 ? |
120s |
< 600 seconds |
Guide Star Acquisition |
20 - 30 ? |
20s / acq |
< 600 seconds |
Notes:
- Overheads to change filters are important, but this is done in
parallel during readout or while the telescope is moving. Therefore,
there isn't always an overhead due to a change of filter. The global
overheads also depends strongly on the number of standard stars observed
for a given night and also if switching from a queue to another is
necessary (since overheads due to filter change are minimized within a
specific queue).
- Focus sequences have been almost completely removed from
operations. The auto-focus model has been available for a while and
significantly increases the time spent observing instead of focussing. A
few focus sequences are taken during the first nights of a run to
confirm the zero points of the model, but after the model has been
updated, no more time is spent on focussing
- Overheads due to dome rotation are minimized as much as possible
within a specific queue. Dome rotation is now optimized and cannot be
made faster.
- Guide star acquisition is fully automated and except from some rare
problematic acquisitions, it works really well. Acquisition tends to
take longer when the seeing is bad or cirrus are present. Programs with
frequent guide star acquisition with short exposure strategy increase
the global overheads. The main overhead related to the guide star
acquisition has been reduced dramatically in 2005A by accelerating the probe
motion. Dithering patterns offsets for instance are now completely
hidden in the readout time, which was not the case in the
past.
- Overheads for calibrations (standard stars and Q98 short exposures
for photometric purposes) are not included in this table. We usually
observe standard star fields twice per photometric night, using
only 1-2 filter per night.
WIRCam
The following table presents the main operational overheads (that is,
other than readout time of the mosaic) with WIRCam. Overheads are highly
variable during a given night depending on the conditions, complexity of
science programs, etc. The table below shows that overheads take a
maximum of ~25min per night. A short summer night lasts about 9.5 hours
with WIRCam, so overheads take around 5%, which is quite low.
Event |
Events/night |
Overhead |
Total overhead per night |
Filter Change |
15 / night |
15s /change |
225 seconds |
Focus Sequence |
2 / night |
65s / seq |
130 seconds |
Dome Rotation > 45deg |
5 ? |
120s |
< 600 seconds |
Acquisition |
36 / night |
12.5s / acq |
455 seconds |
ESPaDOnS
The following table presents the main operational overheads (that is,
other than readout time of the mosaic) with ESPaDOnS. Overheads during a
given night depend mostly on the number of targets and the need to
change Observing Mode or not. The table below shows that overheads can
take up to 1hr per night, most of that coming from the acquisition stage
(pointing the telescope, finding the target, centering the target,
starting the guiding). A telescope pointing issue has been confirmed in
2008/2009, and means that more time is needed to acquire targets; the
guide camera field of view has to be expanded (which takes more time to
readout), and observers take extra care in identifying targets. A short
summer night lasts about 9.5 hours with ESPaDOnS, so overheads take
around 10%.
Event |
Events/night |
Overhead |
Total overhead per night |
Readout Mode Change |
3 |
0 |
0 |
Obs Mode Change |
1 to 3 (< 1 on average) |
7-8min / change |
4-5min / night on average |
Telescope Focus Sequence |
~ 1-1.5 / night on average |
120-180s / focus seq. |
160 - 240 seconds |
Image Quality Measurement |
~ 6 - 12 / night on average |
30 - 60s / measurement |
180 - 600 seconds |
Dome Rotation > 45deg |
5 ? |
120s |
< 600 seconds |
Star Acquisition (pointing, finding) |
12 / night on average |
1-5 min / acq |
10 - 30 min |
Notes:
- The change from one CCD Readout Mode to another takes no time; the
change from one Observing Mode to another, if done during the night, is
accompanied by a spectrograph focus and takes 7-8 minutes (mostly
because the spectrograph focus requires 4 exposures).
- When only 1 Observing Mode change is required, no calibrations need
to be taken during the night; the reduction uses calibrations taken
during the late afternoon or at dawn. If 2 or more Observing Mode
changes are required, calibrations must be taken during the
night, and can take up to 20-25 minutes of sky time. This is very costly
and the QSO Team tries to avoid these cases as much as possible.
- Fewer time is now spent on taking telescope focus sequences. Thanks
to the data accumulated so far (starting in 2008A), the focus model is
reliable and used quickly after the beginning of a run.
- After the telescope is in focus, a few seconds (30-60s) are also
taken to calculate the Image Quality on each target, using again an
automated sequence. A couple a minutes are spent making sure the right
target is picked, using Finding Charts provided by PIs.
D - Agency Time
Accounting
(1) Global Accounting
Balancing of the telescope time between the different Agencies is
another constraint in the selection of the programs used to build the
queues. After 2 semesters with ESPaDOnS in QSO mode, it was
found that this is much more difficult to do with ESPaDOnS than with the
other 2 instruments, because a lot of ESPaDOnS programs request strict
time constraints or have narrow windows of execution, which takes
precedence over Agency Balancing. The queues are basically done
according to the requested time constraints and try to follow the
programs' ranks as much as possible. The agency balance is whatever it
ends up being at the end of the semester. The bad weather for 2008B
compounded this difficulty.
MegaPrime
The table below presents the hours allocated by the TACs, requested
by the PIs, and validated by the QSO Team for each agency.
MegaPrime |
C |
F+O |
H |
T |
L |
P |
Allocated (A+B) | 73.9 | 53 | 69 | 5.5 | 154 | 75.3 |
Requested (A+B) | 73.8 | 52.8 | 68 | 1.4 | 131 | 75.4 |
Validated (A+B+C) | 65.5 | 50.9 | 70.3 | 1.4 | 129.3 | 75.3 |
Validated (C) | 0 | 5 | 4.6 | 0 | 0 | 0 |
The CFHTLS was finished during 2008B, and despite the bad weather,
almost all the time requested was observed and validated. The Large
Program was also completed. Taiwan did not request all the time that had
been allocated by its TAC because one program was a
Target-of-Opportunity program with less targets to observe than
anticipated. Canada is the agency which did not quite get its share of
time with MegaPrime.
WIRCam
The table below presents the hours allocated by the TACs, requested
by the PIs, and validated by the QSO Team for each agency.
WIRCam |
C |
F+O |
H |
T |
Allocated (A+B) | 76.5 | 88.1 | 63.4 | 31.3 |
Requested (A+B) | 66.3 | 81.9 | 63.2 | 31.2 |
Validated (A+B+C) | 67.6 | 94.3 | 54.6 | 31.2 |
Validated (C) | 5.5 | 29.2 | 1.8 | 0 |
The 10 hours missing for the Canadian agency belong to a program
which is actually executed outside of QSO. The French + Opticon agency
is ahead, although a fair portion of the validates hours belong to C
(overfill) programs.
ESPaDOnS
The table below presents the hours allocated by the TACs, requested
by the PIs, and validated by the QSO Team for each agency.
ESPaDOnS |
C |
F+O |
H |
T |
P |
Allocated (A+B) | 52 | 62.5 | 0 | 7.5 | 184.1 |
Requested (A+B) | 51.7 | 62.5 | 0 | 7.5 | 181.8 |
Validated (A+B+C) | 50.8 | 47.6 | 0 | 7.5 | 150.2 |
Validated (C) | 14 | 0 | 0 | 0 | 0 |
In 2008B, the Canadian and Taiwanese agencies fared much better than the
French/Opticon and the Large Programs ones. The poor weather affected
timed-constrained programs tremendously: observations which were
planned for a specific night and time, or within a certain number of
nights from a previous observations, simply became impossible, and it
was often impossible to re-schedule an additional observation to compensate.
(2) CFHTLS Accounting
Up to its last semester, the CFHTLS took a large fraction of the
I-time allocated for QSO for MegaPrime.
The following table shows that despite the inferior weather, the QSO
Team managed to complete the survey.
Survey |
Programs |
Hours requested |
Hours validated |
Deep Synoptic |
L01 |
40.0 |
38.3 |
Wide Synoptic |
L02 + L05 |
85.7 |
85.7 |
Photometric Grid for the Wide |
L99 |
5.3 |
5.3 |
E - Conclusions
MegaPrime
The 12th semester with MegaPrime in QSO mode went well despite the
incredible stormy winter weather. There were no major technical
problems. Non-sidereal tracking has been integrated into
QSO. Observations are running very smoothly. The CFHTLS has been
completed, and one of the 2 MegaPrime Large Programs has been started
without any issue.
WIRCam
The 7th semester with WIRCam in QSO mode also went well despite the
bad weather. There were no major technical issues with the
instrument. Non-sidereal tracking has been integrated into QSO. WIRCam
has the highest completion rate and validation efficiency rate.
ESPaDOnS
The second semester with ESPaDOnS in QSO mode went very well, aside
from the weather, which severely affected many time-constrained programs,
in particular one of the Large Programs. There were no major technical
issues with the instrument, but telescope pointing issues are lowering
the efficiency on the sky. The completion and validation rates are the
lowest of the 3 instruments, due to hard-to-meet time constraints given
the impossible weather.