A TUTORIAL :
YOUR FIRST DATA REDUCTION

Basics...

1. Getting the data ready
2. Check overscan zone on the raw CCD frame
3. Build the bias frame
4. Build the high frequency flat field frame
5. Preprocess the raw CCD frame
6. Find the positions of the spectra centers on the CCD
7. Find the positions of the spectra ridges on the CCD
8. Create the spectra extraction mask
9. Extract the spectra from the CCD frame
10. Prepare the wavelength calibration coefficients
11. Apply the wavelength calibration
12. Prepare the low frequency flat field reference spectra
13. Apply the low frequency flat field spectra correction
14. Remove cosmic ray impacts from the spectra
15. Obtain the sky spectrum
16. Apply the sky spectrum subtraction
17. Prepare the absolute flux reference spectrum
18. Apply the absolute flux spectral calibration

More...

1. Sum energy over a spectral interval
2. Average spectra over an aperture of any shape
3. Separate the continuum and line components in all the spectra
4. Compute any spectral quantity
5. Reconstruct an image from any spectral quantity

BASICS...

Getting the data ready

You have a set of files, obtained while observing the famous Messier 105 object, with the MR3 spectral configuration; they all pertain to the same scenario, which you wisely christened M105MR3. Note that it is important to keep each scenario file set into a separate directory. Naming the work directory from the scenario to be reduced would be good practice; so, m105mr3 would be a nice choice.

To start with, let us suppose you just downloaded the following CFHT files to the .../m105mr3/ work directory :


999001B.fits	Bias frame (median of N)
999100c.fits	Micropupil exposure frame
999200F.fits	GUMBALL continuum lamp exposure frame ("Flat", median of N)
999201c.fits	GUMBALL neon spectral lamp exposure frame associated with the above GUMBALL Flat exposure
999300o.fits	Messier 105 object exposure frame
999301c.fits	GUMBALL neon spectral lamp exposure frame associated with the above Messier 105 exposure
999400o.fits	HD88888 standard star exposure frame
999401c.fits	GUMBALL neon spectral lamp exposure frame associated with the above HD88888 standard star exposure
999500c.fits	Dawn or dusk sky exposure frame
999501c.fits	GUMBALL neon spectral lamp exposure frame associated with the above sky exposure

If you did not get an already medianed bias file, but a collection of individual frames, see the note at the end of the Check overscan section below.
You are now willing to go from this set of raw data files to a set of one thousand and one hundred Messier 105 spectra, wavelength- and flux-calibrated, don't you ?.. Easy ! Well, almost...
Read first the Getting Started section of this manual, to understand the various tools provided, specially the various logbooks and file management tools.

Then, go to directory .../m105mr3/, and type : XOasis. When the XOasis interface is displayed :



• In the main menu, click on Setings, then on Preferences and set any option which you may think useful for you. See the Settings section of chapter Getting started to know everything about that.


• In the main menu, click on Tools, then on Observation logbook. In the new window, click on Files, then on Build logbook. Once the display is updated, if your data files have been obtained before the end of 1998, click on Files, then on Patch fits files. This is mandatory IF you are to reduce data obtained before December 1998, as the native CFHT files were then missing some essential header information; this has since been fixed. It is useless (but harmless) to patch later files. You may go to the Observation logbook section of the Getting started chapter to know more about the Observation logbook; but at this point, this is not essential.


• You may now close the Observation logbook, which will no longer be used.


• From the main menu, choose Tools, then Reduction folder. All the files in directory m105mr3 are listed, with their identifications, file names, and file classes. See the Reduction folder to know more about that tool.


• From the main menu, choose Tools, then Reduction logbook. You may save there various messages during the reduction. See the Reduction logbook section of the Getting started chapter to learn more about that.





Check overscan zone on the raw CCD frame

As described in the Check overscan help section, some LORAL3 frames suffer from a non-uniform offset. This may be cured here, or you may as well skip the present step if you have good reasons to trust the LORAL3 controller (maybe one of your relatives)... If you do not, click on CCD in the main menu, then on Offset, then on Check overscan. In the new window, enter the name of a raw frame, for instance 999001B.fits. Click on View strip average, and a graphics window opens, showing you the [x] average of the overscan strip. If you think this mean will be fitted correctly by a polynomial, check the Polynomial fit option as Correction method; it should be already checked, as this is the "normal" default. If the mean looks like a staircase, check Median filter instead. Click on Accept. Your choice is now recorded in the file header, for later use. Repeat the same operation for all the raw files, from 999100c.fits to 999401c.fits, in the directory.

Biases : CFH medians or not ?
• As long as LORAL3 exhibits the offset problem, you may prefer to get back home with N individual biases, check them with the present tool, then use the Build bias function described just below on each of them, then median them outside XOasis (as this function is not yet provided in the current release) to obtain the final median bias file. But if you perform on Mauna Kea a median over, say, 40 or 50 individual biases, you would have to be very unlucky to get a polluted median.
• But : we encountered some data encoding problem on CFHT (that is IRAF in fact) medians, with image values being strangely quantified in the integer-to-floating point conversion at the end of the median process. One way of going around this is to convert, from within XOasis, the median bias file to some non-FITS format (MIDAS for instance), then back to FITS format, using the Convert, Convert tools function. In the process, the image data is correctly converted to floating point format, and some descriptors are set to the right value. You may delete the intermediate MIDAS file after completion. This procedure is strongly recommended if you wish to use CFHT medians.

In the following text, we shall suppose the raw bias is named 999001B.fits, regardless of its past history, double-converted or not.
A complete description of this process can be found in the Preprocessing section of the manual.



Build the bias frame

First, read carefully the last paragraph of the preceding section...
Click on CCD in the main menu, then on Build bias. Enter the name of the raw bias frame (which should have been checked for offset characteristics, as described in the Check overscan zone section), here 999001B.fits, then click on Accept. A new file named B999001B.fits is created, which is 999001B.fits subtracted from its offset value, and clipped off from the overscan strip.
A complete description of this process can be found in the Preprocessing section of the manual.



Build the high frequency flat field frame

This step is to be performed only if you have at hand a continuum frame which is a high signal-to-noise ratio median. As far as we know, nobody has yet obtained such a marvel. So, this step is skipped in this simple cook book.
A complete description of this process can be found in the Preprocessing section of the manual.



Preprocess the raw CCD frame

In this step, you clip the CCD frames from the unused "offset band", and correct from the electronic constants added to the signal by the readout electronics (the "offset", the "bias"). For that, click on CCD in the main menu, then on Preprocess. In the new window, enter B999001B.fits as the bias file name, and, to start with, 999100c.fits as the input frame. Click Accept. A new file is created : p999100c.fits, displayed as a Preprocessed micropupil frame in the Reduction folder. Preprocess in the same way the other non-bias files, creating :
p999200F.fits (Preprocessed continuum lamp frame),
p999201c.fits (Preprocessed wavelength calibration frame),
p999300o.fits (Preprocessed object frame),
p999301c.fits (Preprocessed wavelength calibration frame).
p999400o.fits (Preprocessed object frame),
p999401c.fits (Preprocessed wavelength calibration frame).
p999500c.fits (Preprocessed sky frame),
p999501c.fits (Preprocessed wavelength calibration frame).
Later on, you will learn to use the [Catalog] function, which eases mass work. More on this point in the Reduction folder section of this manual.
A warning : The preprocessed wavelength calibration frame associated with the continuum lamp frame (p999201c.fits in our example) is to be kept until ALL the object frames are lambda-calibrated. It is used to calculate, for each of these frames, any residual shift to be taken into account before calibrating. More, it should be accessible from the work directory. If you work with an elaborate directory structure, one for each object/scenario, one for service exposures, and so on (a good idea, to keep your reduction folder readable), this can be easily made outside XOasis with a simple link like :
ln {access path}/p999201c.fits ./p999201c.fits, after going (cd) to the place where the object frames are; the same can be done with the (built) bias frame.
A complete description of this process can be found in the Preprocessing section of the manual.



Find the positions of the spectra "centers" on the CCD

You are now about to create the information which will allow the retrieval of the spectral data on the CCD frame. You have first to find the positions on the CCD of the undeviated rays originating from all the microlenses, grism removed. This is done on the micropupil frame, using the Mask function of the main menu, then the Search lenses subfunction. In the new window, input the name of the Preprocessed micropupil frame : p999100c.fits, and the name you like for the future extraction mask (it is a "table", that is a file where data are arranged in rows and columns, not an image, that is the reason for the Output mask table request). As the scenario is M105MR3, you may choose M105MR3mask as a name, and the table full name will be M105MR3mask.fits.
A complete description of this process can be found in the Mask section of the manual.



Find the positions of the spectra ridges on the CCD

One step further : you will now find the lines along which all the spectra are stretched on the CCD, and their limits. For that, use the Mask function of the main menu, then the Find maxima subfunction. Enter the Preprocessed continuum frame p999200F.fits as Input contiunuum frame, and set the Output maxima file as M105MR3max, for instance. Click on Accept, and a file M105MR3max.max will be created to record the results.
A complete description of this process can be found in the Extraction mask section of the manual.



Create the spectra extraction mask

You have now at hand all the information to complete the extraction mask creation. For that, use the Mask function of the main menu, then the Create mask subfunction. Enter the Maxima file name (M105MR3max.max), the Calibration frame, which is the GUMBALL spectral lamp frame associated with the GUMBALL continuum lamp frame just used to find the spectra ridges (the Preprocessed continuum frame p999200F.fits). The Reference wavelength table is a special data file located somewhere within the directory structure of XOasis; just use the browse folder icon at the end of the input field, and choose the file of the element you used in the spectral lamp illumination. For instance, here, Neon_ref.fits. Enter the name of the Mask table already created (M105MR3mask.fits). Click on Accept. This last file is updated (may be quite long on slow systems), and the extraction mask is now ready.
A complete description of this process can be found in the Extraction mask section of the manual.



Extract the spectra from the CCD frame

It is now time to obtain the set of one thousand and one hundred M105 spectra; this is called extracting the spectra. Click on Extract in the main menu, then on Extract spectra. Give the Input frame name, that is the name of the frame from which you are going to extract spectra, the Input calibration frame name, which is the name of the calibration frame associated with the Input frame. For instance, here, the input frame maybe p999200F.fits, and the associated calibration frame p999201c.fits, if you want to extract the continuum spectra. They may be p999300o.fits and p999301c.fits, to extract the Messier 105 spectra, p999400o.fits and p999401c.fits to extract the HD88888 spectra, p999500c.fits and p999501c.fits to extract the sky spectra. And it may be p999201c.fits and p999201c.fits (twice the same name) if you want to extract the spectra from the calibration frame, which should be considered as associated with itself. Same thing with p999301c.fits and p999301c.fits, p999401c.fits and p999401c.fits, p999501c.fits and p999501c.fits. In fact, the eight extractions must be conducted. Give the name of the Extraction mask table, which has just been constructed during the previous step : M105MR3mask.fits. Give the name you like to the spectra set produced; it may be continuum for the continuum spectra file, M105MR3 for the Messier 105 spectra file, HD88888 for the standard star file, sky for the sky file, cont_neon for the calibration associated with the continuum, M105_neon for the calibration associated with the M105 exposure, HD_neon for the calibration associated with the HD88888 exposure, and sky_neon for the calibartion associated with the sky exposure, for instance. Keep the Summation full width at the default value (5), uncheck the Optimal extraction option (better but slower) and click on Accept. The files holding the resulting spectra set are special TIGER files called datacube, as they contain (alpha,delta,lambda) information. They belong to four new classes : Raw continuum datacube, for continuum.tig, Raw calibration datacube, for cont_neon.tig, M105_neon.tig, HD_neon.tig and sky_neon.tig, Raw object datacube, for M105MR3.tig and HD88888.tig, and Raw sky datacube for sky.tig. Here, "raw" means "not yet fully wavelength calibrated". At the same time, auxiliary "mother" tables are created, one for each created datacube, bearing the same name but with extension .fits against .tig for the datacubes. They are displayed as Table, nothing more.
A complete description of this process can be found in the Extraction mask section of the manual.



Prepare the wavelength calibration coefficients

The just extracted spectra are already wavelength-calibrated to the first order. There remains only a second order calibration to be made to get the final calibrated spectra. The first step is to compute, for each spectra the numerical coefficients of this second-order correction. Click on Wavelength in the main menu, then on Compute calibration. Give the name of the Associated calibration datacube, that is the name of the Raw calibration datacube just obtained which is associated with the Raw object datacube (or Raw continuum datacube) you are going to calibrate. Here, it may be cont_neon.tig, if you plan to calibrate the continuum spectra set, or M105_neon.tig, if you plan to calibrate the Messier 105 spectra, or HD_neon.tig, if you plan to calibrate the HD88888 spectra, or sky_neon.tig if you plan to calibrate the sky spectra; bad luck, you must do all of them.
Give the name of the Input wavelength reference table. This is a table with a single column containing all the wavelengths of the lines produced by the spectral lamp(s) used for the calibration exposures. Here, it may be the celebrated Neon_ref.fits table. A set of such tables is provided with XOasis, and you may use the browse icon at the end of the field to choose the right one. Keep the other parameters to their default value (remember that we are dealing with second order corrections) and click Accept. For each calibration The calibration coefficients are recorded into the mother tables created during the preceding step, ready for use in the next step; To keep trace of that, those tables change class, and each become a Calibrated table.
A complete description of this process can be found in the Wavelength calibration section of the manual.



Calibrate the spectra

It is now time to get the wavelength-calibrated spectra. Click on Wavelength in the main menu, then on Calibrate spectra. Give the name of the Input datacube, that is the spectra set to be calibrated; here, it may be continuum.tig, or M105MR3.tig, or HD88888.tig, or sky.tig. Give the name of the Input calibration table, that is the name of the calibrated table associated with the input datacube. Give the name of the Output datacube, that is the name of the resulting wavelength-calibrated datacube. It may be, here, continuum_lbda, or M105MR3_lbda, or HD88888_lbda, or sky_lbda, for instance. Keep the other parameters at their default values, and click Accept.
A complete description of this process can be found in the Wavelength calibration section of the manual.



Prepare the low frequency flat field reference spectra

The next step is the correction of the spatial variations of the spectral response, that is flat-fielding. For TIGER data, this is divided into two steps : the first one corrects for the high frequency variations, that is the CCD pixel-to-pixel variation, the second corrects for the low-frequency variations, that is the local spectral response variation. The first step has marginal effects on TIGER data, and is not yet implemented. The second is dealt with here.
In the main menu, click on Flat, then on Compute flat field. Give the name of the Input continuum datacube, that is the wavelength-calibrated datacube holding the GUMBALL continuum spectra set. It is displayed as Wavelength calib continuum datacube; in our example, it is continuum_lbda.tig. Give the name of the Input skyflat datacube, that is the wavelength-calibrated datacube holding the dawn or dusk sky spectra set; in our example, sky_lbda.tig. Give the name you want for the Output reference flat datacube, that is the datacube which will be created to hold the low-frequency correction spectra set. Here, it may be FLAT_M105MR3, for instance. Let the other parameters as they are, click on Accept; this new datacube is created, and displayed in the Reduction folder as Flat correction datacube.
A complete description of this process can be found in the Low frequency flat-fielding section of the manual.



Apply the low frequency flat field spectra correction

In this step, all the spectra recorded into the object datacube are multiplied by their corresponding correction spectra stored into the just created flat datacube. In the main menu, click on Flat, then on Apply flat field correction. Give the name of the Input datacube to be flat-fielded (here, M105MR3_lbda, or HD88888_lbda; both must be corrected), of the Input flat correction datacube (just created, FLAT_M105MR3 in the present case), of the Output datacube to record the result; it may be M105MR3_flat or HD88888_flat in the present case. Click on Accept, and M105MR3_flat.tig or HD88888_flat.tig is created, and displayed as Flat-fielded object datacube in the Reduction folder.
A complete description of this process can be found in the Low frequency flat-fielding section of the manual.



Remove cosmic ray impacts from the spectra

Tiger data are affected by cosmic ray impacts, and correction is done after spectra extraction and calibration. Cosmic ray signatures are searched into the spectra, and separated from possible emission lines using spatial and spectral morphological arguments.
In the main menu. click on Cosmics, then on Remove cosmics. Give the Input datacube name, that is the name of the datacube to be cleaned from cosmic impacts (here, M105MR3_flat or HD88888_flat), and of the Output datacube which will hold the result (for instance, M105MR3_cosm or HD88888_cosm). Let the other parameters at their default values, and click on Accept. The datacube M105MR3_cosm.tig or HD88888_cosm.tig is created, and displayed as Cosmics removed object datacube in the Reduction folder.
A complete description of this process can be found in the Cosmic ray impacts removal section of the manual.



Obtain the sky spectrum

The sky spectrum can be retrieved from the object exposure using a statistical algorithm; this was how it was done with TIGER (before OASIS) data. This feature is provided in XOasis since release 4.2.
To obtain the night sky spectrum as recorded on exposure 999300o.fits (Messier 105), click on Sky in the main menu, then on Compute sky spectrum. Enter M105MR3_cosm.tig as the name of the Input datacube, and, for instance, skyM105MR3 as the name of the Output sky spectrum. Click then on Accept. Looks like magic... In fact, you will learn later how to restrict the area devoted to sky spectrum computation to some clean one; but the histogram algorithm used is able to find the sky spectrum even in a quite populated field. Let suppose M105 fills only, say, 20% of the field, and there is no point in bothering about clean areas for sky determination, in this tutorial... A complete description of this process can be found in the Sky spectrum subtraction section of the manual.



Apply the sky spectrum subtraction

In the main menu, click on Sky, then on Apply sky subtraction. Give the name of the Input datacube, which is M105MR3_cosm in our example, of the Sky spectrum, which is skyM105MR3, and of the Output sky-subtracted datacube, say M105MR3_sky. Click on Accept.
A complete description of this process can be found in the Sky spectrum subtraction section of the manual.



Prepare the absolute flux reference spectrum

One step remains to be completed : obtain the absolute outside atmosphere spectral flux of every spatial element. This is done using the observation of the flux standard star (HD88888 in our example). In the main menu, click on Flux. The Copy reference table sub-function allows you to import into your work directory a copy of one of the absolute star flux tables provided with XOasis. Use it if you observed one of these stars. If not, you have to provide a fits file of your own, and copy it into your work directory right now. Then, click again on Flux, then on Compute throughput. Give the name of the Input reference star datacube, which should be the cosmic-cleaned datacube (we suppose that the night sky spectrum is absolutely negligible on the short exposure frame of the standard star. Remember that each spectrum receive only the energy from a fraction of an arcsecond squared); here, HD88888_cosm. Give the name of the Input flux reference table we refer to at the beginning of this section. In this table, you must specify the Wavelength column and the Flux column names. In the tables provided with XOasis, they are lambda and f_lambda. Give the name of the Output flux correction spectrum, where the result will be recorded; it may be M105MR3flux_corr, for instance. Keep the Spatial summation radius at the default 3 arcsec value; it is the radius within which the standard star spectra will be summed up, and 3 arcsec seems enough for the usual CFHT seeing. In the same way, the Summation center at 0,0 should be OK, as the standard star is supposed not to be wildly off-centered into the CCD frame. Click on Accept, and spectrum M105MR3flux_corr.fits will be created, and displayed as Flux calibration spectrum in the Reduction folder.
A complete description of this process can be found in the Flux calibration section of the manual.



Apply the absolute flux spectral calibration

Click on Flux in the main menu, then on Apply flux correction. Give the name of the Input object datacube, which should be the cosmic-cleaned object datacube obtained earlier. In our example, it will be M105MR3_sky. Give the name of the Input flux correction spectrum just computed (here, M105MR3flux_corr), and of the Output flux-calibrated object datacube to hold the final result. For instance, M105MR3_abs. Click on Accept, and the datacube M105MR3_abs.tig will be created, and displayed as Flux calib object datacube.
A complete description of this process can be found in the Flux calibration section of the manual.

That's all, folks...

MORE...

Sum energy over a spectral interval

This feature is provided to allow the user to map the energy content of a datacube within a certain wavelength range. Click on Analyse in the main menu, then on Integrate spectra. Give the name of the Input datacube, that is the object datacube from which you plan to extract the data; here, it must be M105MR3_abs. Choose the Spectral range, either complete or within given limits. Choose the Output medium : either a table column, and the column (you have to give the name) will be created in the table associated at spectra extraction with the original datacube (table M105MR3.fits in this case), or an image, and you will be given the opportunity to display this image by using View result. Note that you may choose to compute, over the wavelength interval you specifiy, either the Integral, or the Average, or the RMS dispersion of the datacube.
A complete description of this process can be found in the Analyse section of the manual.



Average spectra over an aperture of any shape

This function allows you to compute the global (average) spectrum of an area of any shape within the observed field. It may be used in release 4.3, as a side bonus of the sky subtraction function, and will be implemented cleanly in release 4.4. For the moment, let us compute the average spectrum of a circular zone within the observed field, Click on Analyse, then Sum aperture. Give the name of the Input datacube, for instance M105MR3_abs in the present case, of the Output spectrum where to put the sum of the spectra of the given aperture. Enter the Center of the circular zone, and the Radius. You may choose between pixel coordinates (CCD) or alpha,delta coordinates (Sky) (sky coordinates are recorded in the associated table, they have been computed at spectra extraction time). Click Accept, and the sum spectrum is created. It may be displayed using the Display, Spectrum or Table function from the main menu.
A complete description of this process can be found in the Analyse section of the manual.



Separate the continuum and line components in the spectra

Click on Analyse in the main menu, then Continuum subtraction. Choose Spectrum as the object to be processed, and enter the name of the Input datacube. Here, it may be M105MR3_abs, as Messier 105 is a well-known Seyfert 4.23 galaxy. Give the name of the result datacube, for instance M105MR3_line. Let the polynomial degree which is supposed to fit the continuum at the default value of 7, choose Subtract for the Operation option, and click Accept. The resulting spectra, held into M105MR3_line.tig, may be displayed using the Display, Spectrum or Table function from the main menu.
A complete description of this process can be found in the Analyse section of the manual.



Compute any spectral quantity

This feature is not yet implemented, but will be part of some future (4.4?) release. It will allow to combine freely columns in the tables associated to datacubes to map quantities such as line widths, line ratios, line-to-continuum ratio, and so on, as it was done for years with TIGER data... You may use the reduction hotline to know the present status of the next release implementing this feature.



Reconstruct an image from any spectral quantity

In the main menu, click on Image, then Reconstruct image. Give the name of the Input table where is the data you want to map, and the name of the Output image. Choose Sky coordinates. Give the Input independant column name, that is the name of the column containing the quantity to be mapped. Click on Accept, and the image is created. It may be displayed using the Display function>
A complete description of this process can be found in the Image reconstruction section of the manual.

Back to top of chapter.

Back to OASIS home page.