Generating Bad Pixel Mask using DS9 Region Files

1. Introduction
This document describe how to use existing bad pixel mask and ds9 region file to produce a better flagged bad pixel mask.  This method is developed because the original method of thresholding dark and flat field images can not pick up structure around bad pixel.  Therefore, human eye balls are needed to pick bad pixels in the corner and grouped bad pixels.
2. Related Program
Few programs are used to do this task.  The first one is the perl script, wircampolyregion. Users can check if this program is installed in your host.  Typically, this program is installed in wena, halea and ula.  Basically, this program takes the ds9 region file as input and output file will contain rearranged coordinate for each region.  The output file can be used in IDL script to flag the bad pixels.  To use this program, simply type the command without any input parameter.  The example syntax will be displayed on screen.   Here is the example.

[chyan@halea script]% wircampolyregion
 USAGE: wircampolyregion <ds9.reg> <output.txt>


 EXAMPLES:
  wircampolyregion ds9.reg output.txt

Another program will be used is maskregion.pro, which is an IDL script.  Since this is my own routine.  The source code is attached in the end of this document.  

3. Procedures

In this section, I will describe the detailed procedures for producing the new bad pixel mask using ds9 and existing bad pixel mask.  First of all, one should produce a masked flat filed or dark image.  For the dark image, one should use the one with 20-second exposure.  Based on these images,  you can see where are the bad pixels that did not pick up using thresholding.  

badpix
Fig 1.  The bad pixel mask

Fig 1. shows the original bad pixel mask.  As you may see, these are some bad pixels grouped all over the chips.  We are looking these bad pixels that are not masked perfectly.  Therefore, we use flat field and dark image to check.  In order to see the bad pixels that are not masked from these masked, select red from menu Edit->Preferences->Blank/Inf/NaN color

badpixred
Fig 2. (Left) The way changing the color of bad pixels. (right) Resulting image

After selecting the red color, the bad pixels that are not selected using thresholding method can be easily seen by looking at each chip very closely.  Once you identify the bad regions, you can mask it using ds9 regions.  Notice, please POLYGON and BOX only.   Fig 3. show an example to select these region.  As you may see, the bad pixels are still there (white points) even you have masked some of them (red points).   The region shown here are masked using polygon with green line.  When you done the region selecting, you can save the regions you defined to ds9 region file and these file(s) will be used to flagged bad pixels.  Here are examples of ds9 regions.

region
Fig 3. Region selected using ds9 polygon region.
 
Now it is time to using IDL to produce the final bad pixel mask.  Here is a simply IDL script does this job.  In this program. We take 3 region files as input and output FITS file of bad pixel mask.  So that one can check the result of each  region file.

PRO scriptbadpix
path='/data/halea/wircam/chyan/Badpix/'
conerregion='corner.reg'
darkregion='dark_bigstructure.reg'
flatrefion='flat_bigstructure.reg'
badpix='badpix_20061219.fits'


maskregion,badpix=path+badpix,region=path+conerregion,newbadpix=path+'1.fits'
maskregion,badpix=path+'1.fits',region=path+flatrefion,newbadpix=path+'2.fits'
maskregion,badpix=path+'2.fits',region=path+darkregion,newbadpix=path+'final.fits'
END

4. Possible Problem

One problem has been found that if your region file contains a small region.  This means that you can not define the region using 4 coordinates, that have the same x and y coordinate.

5. IDL Source Code

; NAME:
;       MASKREGION
; PURPOSE:
;       Produce a new bad pixel mask based on an exist one and ds9 region file
;
; CALLING SEQUENCE:
;       MASKREGION, BADPIX = badpix, REGION = ds9.reg, NEWBADPIX= newbadpix
;
; INPUTS:
;       BADPIX    -  string scalar, the name of the original bad pixel mask.
;       REGION    -  DS9 region file, only polygon and box allowed.
;       NEWBADPIX -  string scalar, the new bad pixel mask.
;
;
; REVISION HISTORY:
;       initial version   Chi-hung Yan   March 2007
;-



PRO RDWIRCAMCUBE, file, im
    im=fltarr(2048,2048,4)
    for i=0,3 do begin
        im[*,*,i]=mrdfits(file,i+1)
    endfor
END
;

PRO MASKPOLYREGION, im, ext, x, y
    ind=polyfillv(x,y,2048,2048)
    tim=im[*,*,ext-1]
    tim[ind]=!values.f_nan
    im[*,*,ext-1]=tim[*,*]
END

PRO WRWIRCAMCUBE, im, fits, new
    main_hd=headfits(fits,exten=0)
    mwrfits, 3,new , main_hd, /create
    for i=1,4 do begin
        hdr = headfits(fits, exten=i)
        mwrfits,im[*,*,i-1],new,hdr
    endfor
END


PRO MASKREGION, badpix=badpix, region=region, newbadpix=newbadpix
    file=region
    fits=badpix   
       
    ; Read image       
    rdwircamcube,fits,im
       
       
    spawn,"wircampolyregion "+file+" /tmp/region.txt"
    readcol,"/tmp/region.txt",id,ext,x,y
    n=0
   
    xx=fltarr(1)
     yy=fltarr(1)
    for i=0,n_elements(x)-1 do begin
        if i eq 0 then begin
            temp=id[i]
        endif else begin
            temp=id[i-1]
        endelse
       
        if i eq 0 then begin
            xx=x[i]
            yy=y[i]
        endif else if id[i] eq temp then begin
            xx=[xx,x[i]]
            yy=[yy,y[i]]
        endif else begin
            maskpolyregion,im,ext[i-1],xx,yy
            xx=x[i]
            yy=y[i]
            n=n+1
        endelse
       
    end
    spawn,'rm -rf /tmp/region.txt'
    wrwircamcube, im, fits, newbadpix
   
END

6. Perl Script

#! /usr/bin/perl
#----------------------------------------------------------------------------
#                             wircampolyregion
#----------------------------------------------------------------------------
# Contents: This script parse the ds9 region file into better format.  The out
#                out file can be easily used for IDL program.
#----------------------------------------------------------------------------
# Part of:     WIRcam C pipeline                              
# From:        ASIAA                   
# Author:      Chi-hung Yan(chyan@asiaa.sinica.edu.tw)                      
#----------------------------------------------------------------------------

use warnings;
use strict;

# Define global variable
our $SCRIPT_NAME = 'wircampolyregion';

sub trim {
   my $string=$_;
   for ($string) {
       s/^\s+//;
       s/\s+$//;
    }
   return $string;
}

# This finction is used to return program usage.
sub usage(){

print<<EOF
 USAGE: $SCRIPT_NAME <ds9.reg> <output.txt>


 EXAMPLES:
  $SCRIPT_NAME ds9.reg output.txt

EOF
}

if(@ARGV == 0 || @ARGV != 2){
   usage();
   exit 0;
}

my $file = $ARGV[0];
my $i=0;
my $j=0;
my $ext;
my $reg=0;

open(OUT,">$ARGV[1]");
open(FILE,$file);
while(<FILE>){
    if (/^# tile/){
        my @chip=split(" ",$_);
        $ext = $chip[2];
    }
    if (/^polygon/){
        $reg=$reg+1;
        my @word=split(/[\(\)]/,$_);
        my @coord=split(",",$word[1]);
        my $n=@coord/2;
        for ($i=0;$i<$n;$i++){
            printf(OUT "%d %d %7.2f %7.2f\n",$reg,$ext,$coord[2*$i],$coord[2*$i+1]);
        }   
    }
    if (/^box/){
        $reg=$reg+1;
        my @word=split(/[\(\)]/,$_);
        my @para=split(",",$word[1]);
        printf("$word[1]\n");
         printf(OUT "%d %d %7.2f %7.2f\n",$reg,$ext,$para[0]-$para[2]/2,$para[1]-$para[3]/2);
         printf(OUT "%d %d %7.2f %7.2f\n",$reg,$ext,$para[0]+$para[2]/2,$para[1]-$para[3]/2);
         printf(OUT "%d %d %7.2f %7.2f\n",$reg,$ext,$para[0]+$para[2]/2,$para[1]+$para[3]/2);
         printf(OUT "%d %d %7.2f %7.2f\n",$reg,$ext,$para[0]-$para[2]/2,$para[1]+$para[3]/2);
    }   


}

close(OUT);