Photometry: comparisons between different methods

I discuss comparisons between photometric measuremets from the three main packages: Sextractor, DAOphot, and Dophot.

There are several packages available for performing photometry on the images. They all operate on similar principles, but have subtle differences in the details. I have compared the performance of three of the most common packages, Sextractor, DAOphot, and Dophot using real astronomical images. First, a brief discussion of some of the differences and relative advantages of the three packages.

Dophot: This program uses a purely analytical function (a polynomial approximation to a 2-D Gaussian) to represent the stellar profile. The profile is represented by four parameters unique to each object (X, Y, Sky, Peak Flux) and three 'shape' parameters ($sigma_x$, $\sigma_y$, $\theta$). Dophot classifies objects based on how well they fit to the stellar profile. It attempts to determine a 'best-guess' set of shape parameters which represent the stellar objects, and then fits a profile keeping those parameters fixed for stellar objects. For non-stellar objects, it may fit a profile with floating shape parameters or it may fit a double stellar profile. Dophot reports both the magnitude derived from the fit and an aperture magnitude for each object. Dophot requires little information specific to an image and can therefore be run quite automatically. The version of dophot which we are using is one which has been somewhat improved by the author (EM). First, it has been translated to C for improved portability and stability. Second, it uses dynamical allocation for image memory, so it can be given arbitrarily large images. Third, it can accept all FITS image formats, and understands the BZERO & BSCALE keywords, in addition to the BITPIX keyword.

Sextractor: NEED TO HAVE JCC write something here. We have used the PSFex implementation of Sextractor for improved photometric accuracy. Sextractor also required little in the way of tuned input, and can be run blindly on many images.

DAOphot: This program uses a combination of analytical and numerical models for the stellar profile. The model is determined with some substantial user interaction, selecting appropriate, isolated stars. As a result, it is not trivial to automate the processing of a large number of images.

Data used in comparison

We are using images obtained by Greg Fahlman of the open cluster NGC 6819. This field is an interesting challenge for the analysis routines because it has many bright stars with the accompanying bleeding and saturation. The cluster resides on the inner 4 chips, so we are only analysing chips 2, 3, 8, and 9, at the most. We have both $B$ and $V$ images, so we can generate CMDs for comparison as well. The $B$ and $V$ cluster images are the result of combining 9 separate 5 minute exposures, and have been cleaned, flat-fielded and co-added to improve the signal to noise. In addition to the summed image, we also have the option of analysing the 9 input images independently, which can be quite informative, as we shall see.

Comparisons on the summed $B$ chip 02 image.

For the first comparison, we have analysed the chip 02 $B$ summed image with each of the three techniques. We then match the stars between the three resulting data files and compare the measured magnitudes. Figures 1 & 2 show the comparison between Dophot & Sextractor and Dophot & DAOphot magnitudes. First, there is the issue of the aperture correction. DAOphot and Sextractor both determine and apply an aperture correction to the reported magnitudes. Dophot does not, but the aperture magnitudes measured can be used to determine a best-guess aperture correction. The bottom panel of these two figures shows dophot fit - dophot aperture magnitudes plotted versus the fit magnitude, and shows the aperture correction to be 0.19 mag. The middle panel shows the difference between the dophot fit magnitude and the Sextractor or DAOphot measured magnitude versus magnitude. For this plot, only objects which dophot found to be realistic stars are plotted. The middle panel shows the effect of applying the dophot aperture correction to the observed dophot magnitudes. These plots show the same as the middle panel, corrected now for the aperture correction found in the bottom panel.

Several conclusions can be drawn from these plots. First, the effect of slightly saturated stars can be seen in the relatively high scatter of points at magntides $< -15$. In the magnitude range of roughly -15 to -11, there are some large outliers, especially in the comparison of Sextractor to Dophot, but the bulk of the scatter is less than roughly 0.02 or 0.01 magnitudes. As the plots go to fainter magnitudes, there is a systematic trend in the Sex-Dophot comparison, but not so much in the DAO-Dophot comparison. There are also some aperture correction errors in the Sex-Dophot comparison. The source of this error is for the moment unclear.

For the Sextractor - Dophot comparison, we looked in some detail at the detection efficiency of the two methods. Both methods have a tendency to identify many 'stars' among the bleeding pixels and halos of bright stars. Sextractor does a better job of finding stars down to the faintest limits, but it also finds objects in the low-surface-brightness features on the images which it identifies as good stars as well. Dophot has a tendancy to miss all stars below a particular cutoff independent of the minimum flux threshold assigned. Dophot does a better job than Sextractor of identifying stars close to other stars and therefore tends to be more complete at mid-level magnitudes.

Comparisons via the CMD.

Figure 3 shows color-magnitude diagams derived using each of the three programs. These comparions show the relative success fo the various methods. The main criterion to test the accuracy of the analysis is the tightness of the main-sequence for this cluster. There are four CMDs shown here. There are the two CMDs derived from the Dophot and DAOphot analysis. These two CMDs show that the main sequence for the middle magnitude range is quite comparable. They also show the failure of Dophot to pull out the faintest stars (roughly 1.5 - 2 mag different completeness limits). It also appears that DAOphot handles the slightly saturated stars somewhat better as well. The next two CMDs are both based on Sextractor. The first one uses the data provided by Jason with the parameters he used. The second uses parameters Catherine and Eugene chose for the Taurus run. While both CMDs are comparable to the DAOphot CMD in terms of the bright and faint ends, the first of these CMDs appears to have a more less well-defined main sequence than either of the DAOphot or Dophot CMDs. The second one seems to do a much better job, and has qualitatively as tight a main sequence as the Dophot and DAOphot CMDs.

Comparisons using 9 overlapping $B$ images.

A more quantitative comparison between the various analysis methods can be made by analysing the 9 overlapping individual images which were combined to create the images discussed above. The idea is that, within these 9 images, the measurements of the stellar magnitudes should be constant, after an offset is made to compensate for the relative throughput (zero point) of each image. What we can do is, for the ensemble of all measurements of all stars on all images, determine a set of 9 zero-point offsets, one for each image, that collectively minimizes the scatter for the stellar measurements about their mean.

We analysed each of the 9 images with both Dophot and Sextractor (C & E's parameters), and then determined these zero-point corrections. As a result, we can calculate the mean magnitude for each star, as well as the scatter (standard deviation) of the measurements about that mean. Figure N shows this scatter as a function of stellar magnitude for all objects in the 9 images, for both dophot (boxes) and sextractor (crosses). Figure N shows a histogram of the scatter for all stars in the magnitude range 18 - 19. This plot shows that there is a great deal of consistency for these measurements. For example, at 18 magnitude, the typical scatter is in the vicinity of 3 millimags. There is a small tendancy for the scatter in the sextractor measurements to be less than that for the dophot measurements. This is seen in an offset of about 1 millimag in the centroid of the histogram in Figure N.

Comparing the Dophot and Sextractor measurements individually for each of the 9 input images shows that different images have qualitative differences: Some show more scatter between the two measurements, some less; for some the systematic trends are stronger; for some there are fewer outliers, etc. A comparison between the mean magnitudes from the 9 Dophot and 9 Sextractor measurements is consistent with the level of scatter seen in the ensemble comparison above. The averaging has reduced the number of outliers, and the typical scatter is within 0.01 mag.

Conclusions

To a modest degree, all three of these methods produce similar results. It appears that Sextractor may be somewhat dependent on the choice of parameters, but this is probably true for each of the three analysis methods. Dophot has a clear problem pulling out the faintest stars, for reasons which we understand and which we may implement some improvements. Sextractor has trouble with close stars, and as a result is somewhat less complete at the middle magnitudes. DAOphot does quite well, but the overhead in running the program makes its utility for mass reduction rather limited. On a final note, the speeds of Dophot and Sextractor are quite comparable. For the group of 8 images used in the CMDs, Dophot required 46 minutes while Sextractor took 43 minutes. For the 9 overlapping B images, Dophot took 22 minutes while Sextractor required 24 minutes.