Applying flat field and dark frame corrections

Flat field frames

Flat fielding corresponds to correcting the combined optical-system and CCD throughput at each pixel so that each pixel on the CCD would respond equally to a source with the same photon flux.  Flat fielding removes the effect of the pixel-to-pixel sensitivity variations across the array as well as the effect of dust or scratches on the CCD window, and vignetting by the telescope or camera lens optics.  A flat field frame must be produced for each filter/lens/telescope combination to obtain good results.  The flat-field frames are obtained by photographing an evenly illuminated target at very close range (and therefore out of focus) or by shooting the sky near dusk or dawn.  The prior technique using a halogen illuminated flat white target is preferrable because the light level is more easily controlled and can be performed carefully at leisure rather than hastily, in the field.  If the CCD is cooled and the exposures are reasonably short, then the dark (thermal) count may be neglected.  It is best to shoot several frames (at least ten) and average these to reduce the noise.  The more, brighter, frames the better, however the CCD should be operated well away from saturation where the response may be nonlinear.  In the case of the Pyxis camera, anything below 50,000 ADU is acceptable.  The flat field generation tool (see multiple file processing) simplifies the process of creating the flat field frames by first subtracting the offset level from each selected flat field image, and averaging the images together.


Dark or thermal count frames

During long integrations at moderate cooling a significant amount of charge can accumulate in the CCD pixels due to thermal excitation.  The purpose of producing dark or thermal count images is to subtract the contribution from thermally generated charge in the image.  Professional astronomers overcome the problem of dark charge by cooling their CCD's to temperatures near the boiling point of liquid nitrogen (~77K).  This of course is not practical for most amateurs, so moderate cooling between -20 °C and -40 °C using a thermo-electric cooler is usually employed.  Inevitably, for integrations longer than 5 minutes, the dark count will begin appearing in certain pixels on the CCD which have particularly high dark currents; these are referred to as "hot-pixels".  It is the hot-pixels that cause the greatest problem in the image degradation because they are often as bright as some of the dim stars in the image, so that simple thresholding to remove the hot-pixels often results in a significant loss of image information.  Median filtering is helpful in removing the hot-pixels because these are often isolated, however the median filter can significantly "soften" the image, especially in cases when the PSF is less than 2-3 pixels wide.

The dark count may be subtracted if a dark count image is available that exhibits less noise than the dark count contribution in the image being processed.  This last point is very important; if the dark count image to be subtracted has more noise than the image to be processed, then subtraction will actually lead to an increase in noise!  This means that dark count images must be obtained for much longer integration times (at the same cooling temperature) as the original image to be processed.  Alternatively, many dark count images may be obtained and averaged, however the noise will be higher when multiple images are averaged due to the higher read noise compared to reading a single image with an equivalent total integration time.  The integration should not be so long that the hot pixels approach saturation.  Management of the dark count frames is somewhat easier than for the flat field frames because the same dark frame can be applied to all images, regardless of what filters or optics were used to produce the image.  A tool is provided to create an average dark count-frame from the dark-count images (see multiple image processing).


How are the photometric frames created?

Internal to the Pyxis firmware is a routine that estimates the dark count and offset level in each image acquired.  The dark count is obtained from a portion of 4 shielded CCD rows in the KAF-401/1602 sensors; this is saved as the image parameter "BottomThermalCount".  The offset (or bias) level is determined by averaging 1000 "overscan" horizontal CCD pixels so that no thermal or photo-electrons are measured.  For the Pyxis KAF-401 camera, the offset is typically 4000-6000 ADU; it is saved as the image parameter "BiasLevel".

The procedure for creating the dark-count and flat-field frames is identical.  The "BiasLevel" is subtracted when the images used to create the flat-field or dark frames are summed so that only the contribution from thermally or photo-generated electrons remains.  The summed images are then scaled so that the maximum intensity in the sum-image is 65535; the maximum value for a 16-bit unsigned integer.  The median value in the center of the image is computed and assigned to the "BiasLevel" parameter of the image.


How are the photometric corrections applied?

The dark count frame should always be subtracted first because the dark count is not affected by the effective CCD pixel sensitivity nor the number of photo-electrons collected at the pixel.  If the automatic routine is used in the multiple file processor, then the dark frame intensity is weighted by the ratio of the "BottomThermalCount" in the image to be processed to the "BiasLevel" in the dark count image.   The  "BiasLevel" in the dark count image should represent the median intensity in the image, if the "Create thermal frame" function on the multiple image processing form was used.  The purpose of the prior subtraction of the bias and later scaling of the dark count frame is to allow dark count frames obtained for different integration times to be used.  Otherwise, dark count frames must be produced for each integration time used, which is rarely practical.

The "flat-fielding" operation consists of dividing the intensities in the image to be processed (minus the offset level of the image) by the flat-field image.  The resulting image is automatically scaled in intensity so that the most is made of the dynamic range of a 16-bit integer.  An important point to remember is that the flat-fielding produces an image whose pixel intensities more accurately reflect the light intensity of the imaged object, however the connection of the pixel intensities to the collected pixel photo-electrons is lost due to the somewhat arbitrary scaling used in the flat-fielding procedure. 

Note that if the multiple image processing tool is employed, then three options are available to determine the bias to subtract from the images.  The bias can be estimated from a histogram of a rectangular region of the image, determined from the quoted value in the image header or a fixed value can be selected.  The first method is not recommended because the bias estimate will also include contribution from sky glow.  If you are worried about a fixed pattern in the image offset frame (that is a frame obtained in the dark under conditions where the thermal count is negligible) then a series of offset frames can be generated and subtracted from the images prior to dark-frame and flat-field correction.  During flat-field and dark-frame correction the bias to subtract should then be set to 0.  Note that no fixed pattern has been noticed in the Pyxis KAF401e prototype camera offset frame - the spatial (meaning from pixel to pixel in the same image) and temporal (meaning from frame to frame looking at the same pixel) noise properties of the offset frames are identical.