Perhaps the most imporatant type of Elixir data product is the
collection of master detrend images. By working with the collection
of images from an entire CFH12K run (the period the camera is mounted
on the telescope), we have a unique opportunity to create the best
possible set of master detrend images. A crucial element of this
process is a database of the resultant detrend images which can be
used to make the associations necessary between master detrend images
and raw science images.
Detrend Images and the Detrend Database
Elixir generates master detrend images for each type of detrend data.
So far, the types which we are supporting consist of: darks, biases,
flats and fringe frames. To give an idea of the number of master
detrend images, here are some typical numbers. We produce flat field
images for each of the broad-band filters (5) and narrow-band filters
as needed. We also produce bias images, as well as dark images for
several pre-selected exposure times: 60, 300, 600, and 1200 seconds.
The dark images may be scaled from one of these special exposure times
to whatever is appropriate for the specific image. We also produce
fringe images for each of R, I, and Z. Assume for now that we don't
produce more than one set for the run, and that we don't produce
detrend images in the narrow-band filters. In that case, a single 12k
run will produce 11 distinct detrend images (full-mosaic). This
represents 2.5 GB. Over the course of 2 years, we can expect to have
roughly 20 12k runs, which means about 50 GB worth of detrend images,
or about 2600 different CCD frames, for two years of operations.
It is not a significant drain on resources to keep the 50GB of detrend
images on spinning disk. However, it is crucial to have a method to
determine the appropriate detrend image for a given situation, or
a given image. This is the role of the Detrend database.
The detrend database is used to organize the detrend images and make
the association between the specific detrend image and those images
for which it is valid, and vice versa. The database follows the
format of the other databases used in Elixir: a file written with
binary data defined by a C header structure, along with a FITS header
giving summary information. File locking is used to maintain the
database integrity.
/* Detrend Database structure */
typedef struct {
unsigned long int tstart;
unsigned long int tstop;
unsigned long int treg;
float sigma, clipsigma;
int type;
int filter;
int ccd;
char filename[254];
char flag1, flag2;
} DetReg; /* 272 bytes */
Above we show the C structure used to define the detrend database
data. The data included in this structure includes: the image name,
image type (flat, etc), filters as appropriate, complete path to the
images, and so forth. In addition, to basic information about the
image, three important time references are kept as well: The time the
detrend image was registered in the database, and the start and end
times for which this detrend image is valid. These dates are assigned
during the detrend creation process (see ....).
Interaction with the detrend database takes place through two
command-line tools, one for input, one for output. The first tool,
detregister, is used to add, and if needed delete, detrend
images from the detrend database. Ideally, the header of the detrend
image supplies all of the information needed by this program to fill
in the database fields. However, command line options allow database
fields to be provided by the user if those in the header are missing
or inaccurate.
The second tool, detsearch, lets the user peruse the detrend
database and grab specific detrend images or sets of images as needed.
One use of this tool is to select the appropriate detrend image of a
given type for a specific image:
detsearch -image example.fits 00 split -type flat
Lines of this type are used in the 'flatten' routine to provide the
appropriate flats, biases, etc. In this example, we request the most
appropriate flat field image for the image example.fits. The program
looks in the header of example.fits to decide the appropriate filter,
time, etc, and selects the image from the database. Other uses of
'detsearch' could include listing all detrend images of a given type,
ccd, filter combination, etc. The program 'detsearch' therefore
provides all necessary queries of the detrend database and provides
the association between science images and detrend images. By
incorporating the function in the detrending program, the association
is automatic and transparent to the user.
A crucial element in associating a given image with a given detrend
image is the time of the image. A given detrend image has a period of
time over which it is valid. This may be short, as in the case of
fringe frames which are only valid for a small number of days, or it
may be very long, as in the case of mask images, which are valid as
long as the CCD arrangement is not changed. During the detrend
creation process, the valid time range for the images is defined. If
an image is added to the detrend database which overlaps an existing
image, the most recent image take precedence by default. A search for
a detrend image may specify if the resulting images exactly overlap
the specified time, or the search may optionally return the most
recent image, even if the end of the valid time range is before the
time in question. This option is useful to perform a rough analysis
on image from an ongoing run before the latest detrend images have
been produced. In such a case, the detrend images of the previous run
would most likely be chosen as acceptable, if not ideal.
|