Elixir: Progress Report Oct 2000

Elixir is a project to achieve the combined goals of 1) providing high-quality calibration date and detrend\footnote{We have chosen the word 'detrend' to refer to the 'instrumental signature images' (Bias, Darks, Flats, etc). We distinguish 'detrending' from 'calibration' (astrometric or photometric) because of the difference in treatment.} images from CFH12K (and later MegaCam) images and 2) evaluating the quality of the science images produced at the telescope.

Since May 2000, we have had more successful tests of the real-time analysis portion of Elixir. We have made significant progress in tuning our detrend data analysis pipeline. We have worked out the details of the mechanisms by which we associate science images with the appropriate detrend and photometric zeropoint data. We have purchased virtually all of the hardware necessary for the Elixir system. Real-time Test Runs

We have run the real-time analysis system for each of the March, June, and July 2000 CFH12K observing runs. These runs showed several areas where details needed to be improved, mostly in the handling of the image replication process and in the process communication mechanisms. These runs also guided some of our choices related to data organization: where the various data products are placed, how they are named, how the configuration scripts are called, etc.

These real-time runs also provided us with test data for several important developments. In particular, the data collected in these runs suggested some changes in the sample of data recorded by the various analysis steps, and the types of data stored in the various databases. For example, it became clear that we should include in the Elixir databases the temperatures and bonnette settings relevant for each image, instead of relying on searches of the data logger to make the association dynamically.

We also have used the resulting test data to guide the development of some of our data display products. For example, the large collection of seeing and sky flux data helped us to design appropriate tools to display these data in real-time. The data collected have also been helpful in guiding analysis of the sky flux during the twilight, necessary for optimizing flat-field collection. Improvements in the Detrend System

We have used the data from the June and July 2000 runs, as well as data from the September 1999 run to guide our design of the Detrend analysis pipeline. The June and July runs were particularly important because we paid special attention to the photometric quality of the sky during every flatfield period, morning and evening twilight. We have learned that cirrus can be problematic for the creation of flats, because it keeps the sky from having uniform illumination on the scale of 12k. By knowing when the sky had noticeable cirrus, we could test the strategies we use to detect the gradients caused by cirrus.

We have now worked out the details of the Detrend analysis system, including the iterative process by which the system selects only the appropriate source images when creating the flat-field master. We have found several interesting 'gotchas' that our software needs to avoid. For example, we discovered that dome flats, which are not sufficiently flat to create a good flat field image, are difficult to distinguish from the twilight flats. Nonetheless, it is necessary to reject them when creating the master twilight flat, so we have implemented a test by which the detrend system can reject an image on the basis of the time it was obtained. This trick is particularly useful in the analysis of the old, archived data for which we have no control over the header information. Detrend \& Photometric Zeropoint databases

One of the important data products from Elixir is the calibration data, both detrend images and photometric zero points. In addition to creating the master detrend images and determining the zero points for each night, it is necessary to provide a mechanism by which the end users can make the association between a specific image and these quantities. We have put together a set of databases for both quantities along with tools by which the users interact with these databases. For the detrend data, the user can request the detrend image of a given type (ie flat) for a particular time period, CCD, and filter, or ask for the detrend image appropriate to a given image. A similar mechanism is provided for the zero points: the user can request the photometric calibration coefficients for a given set of conditions or for a particular image. Elixir Hardware

We have now purchased and installed virtually all of the hardware that will be needed for the Elixir system. We have a cluster of Dell 410 / 420 machines running Linux connected with a 100 MB/sec switch. At this time, we have 5 dedicated computers, each with two processors, running at speeds ranging from 500 MHz to 733 MHz. In addition to this compute power, we also have a large amount of disk space available for the image and data product storage. We are using RAID arrays of 40 - 60 GB IDE disks, with up to 7 disks per machine. Currently we have 3 setups with 7 x 40 GB = 280 GB available dataspace and 1 setup with 7 x 60 GB = 420 GB. We are using the largest partition as storage for the images, one for the detrend images and reference data, one for the intermediate steps, and one for maneuvering, as needed.