Skyprobe Support
This page explains the components of the SkyProbe system, and
discusses trouble shooting.
Overview
The SkyProbe camera is mounted on the Caisson Central and controlled
by a iopener computer in the nearby cabinets. The iopener has no
on-board storage, so all images are immediately sent from this machine
via the ssh connection shell to a second computer at the Summit,
wekiu. All stages of the skyprobe pipeline, including image analysis,
storage of the resulting data products, and the generation of summary
plots, is performed on wekiu. The disk on wekiu has 100GB, enough
space for about 3 months of skyprobe data from a single camera. The
entire data partition is mirrored to a location in Waimea every
morning. On a monthly basis, this mirror is used to make a tape
backup of the old images, which are then purged from the online
system.
A daemon running on wekiu requests an image from the iopener once per
minute. This same daemon is responsible for turning on the camera
cooling system at the start of the night, turning it off at the end of
the night, and checking that the data directory for each night is
created. A separate daemon running on wekiu generates the skyprobe
plot, and saves a copy of the plot in the archive at the end of each
night.
Camera Control
The iopener which controls the skyprobe camera runs the program
sp_ccdcontrol which communicates with the skyprobe camera
directly. This program is in turn called by a program from the summit
computers. The general command to run the camera is
sp_command, which takes several options:
[skyprobe@wekiu:~/] sp_command -h
USAGE: sp_command init (path)
USAGE: sp_command cool (temp)
USAGE: sp_command warm
USAGE: sp_command test
USAGE: sp_command dark (exptime) (filename)
USAGE: sp_command expose (exptime) (filename)
The init command checks for the data directory and creates it
if needed. The cool command tells the camera to cool to the
specified temperature, usually set to -20C. The warm command
turns off the cooling system. The test command runs a
detailed test suite, discussed below. The dark command takes
a single dark exposure of the specified exposure time, writing the
output to the given file. Finally, the expose command is
used to take a single image of the specified exposure time, saving the
output to the specified file. Any of these commands can take an
optional flag -iopener name@address. This option tells
sp_command to send the commands to a different host computer,
with both the machine ip address and the camera account specified. In
addition, the expose command can take an option
-noprocess, which tells sp_command NOT to send the
image to the analysis system. The sp_command program is
usually run by the skyprobe daemon on wekiu at the summit.
The sp_command test suite performs a variety of useful checks
on the skyprobe camera: it checks that the iopener is alive, it turns
on the cooling system, waits 1 minute for the camera to cool somewhat,
takes a single dark frame, checks that the temperature information is
consistent with a cooling camera, checks the flux in the dark frame,
takes a single light frame, and measures the flux in the light frame.
It generates a postscript image with the current date and sends it to
the printer 'sps'. The entire process runs for about two minutes.
This should normally be run during the daytime, when skyprobe will not
be controlled by the daemon. Although no harm will come by running
the test when the daemon tries to take an image, the results will
likely be mis-interpreted. If it is necessary to run the skyprobe
test suite at night, it is best to halt the skyprobe daemon for the
duration (see below). A typical output from the test suite is given
here:
[skyprobe@wekiu:~/] sp_command test
starting skyprobe test suite
this will take about 2 minutes
iopener is OK
cooling camera & waiting for 1 minute
taking 1 sec dark frame test.dark.fits
progress: Exposure complete (100%)
progress: Writing FITS data to stdout00%)
logonly: Wrote 390150 pixels, min=84, max=234
progress: sp_ccdcontrol done.
temperatures: camera: -19.5, outside: 16.3
camera temperature OK
dark flux: 104
taking 0.1 sec light frame test.light.fits
logonly: ccd_status=2 controller (100%)
logonly: ccd_status=2
logonly: ccd_status=2
warning: Exposure took an extra 0.15 seconds
progress: Exposure complete (100%)
progress: Writing FITS data to stdout00%)
logonly: Wrote 390150 pixels, min=2726, max=36145
progress: sp_ccdcontrol done.
light flux: 13996.2
approx. expected counts:
100 for dark dome
15000 for dome lights on
30000 for dome open daytime
Skyprobe Daemon
The skyprobe daemon which runs the camera is called skyprobe
and is run by the user 'skyprobe' on the computer 'wekiu' at the
summit. This daemon is automatically started when this machine boots.
The skyprobe daemon uses the Elixir nightd daemon
system to run regular events during the night, as well as initial and
final nighttime events. The skyprobe daemon (simply a link
to the nightd daemon) is defined by the .skyproberc file in
skyprobe home directory. This configuration file is given here:
HOME /h/skyprobe
DATA_PATH $HOME/data/rawdir
PID_FILE $HOME/log/skyprobe.pid
LOG_FILE $HOME/log/sp_daemon.log
CCD_TEMP -20
EXPTIME 30
INIT_COMMAND sp_command cool $CCD_TEMP
INIT_COMMAND sp_command init $DATA_PATH/&DATE
MAIN_COMMAND sp_command expose $EXPTIME $DATA_PATH/&DATE/sp_&DATE_&TIME.fits
DONE_COMMAND sp_command warm
NIGHT_START 18:00
NIGHT_STOP 06:00
PERIOD 60
TIMEOUT 300
In this configuration file, there are a few keywords which are
required to define the daemon, and the rest simply define values to be
used elsewhere in the file. The required keywords are:
INIT_COMMAND - command executed at the start of the night
MAIN_COMMAND - command executed regularly during the night
DONE_COMMAND - command executed at the end of the night
NIGHT_START - local time of the start of a night
NIGHT_STOP - local time of the end of a night
PERIOD - MAIN commands are started on module PERIOD seconds
TIMEOUT - processes which don't complete in this time are killed
PID_FILE - process ID file, used for locking and communication
LOG_FILE - process log file.
Multiple entries of the three _COMMAND keywords are all
performed in sequence at the appropriate time. A single day runs for
24 hours from 12:00 noon local time. The special variables &DATE and
&TIME are converted by the daemon to the current value of the date
(the UT date for 09:00 of this 'day'; ie, if it is after 12:00 local
time, &DATE will reflect the UT date at 09:00 the next morning).
&DATE is provided in the format YYYYMMDD. &TIME is the current local
time in the format HHhMMm. In the case of skyprobe, these two are
used to generate unique names for each image.
The skyprobe daemon can be easily started and stopped using the
following command-line options:
skyprobe start : start the daemon
skyprobe stop : stop a running daemon
skyprobe status : check the status of a running daemon
skyprobe config : reload the configuration file
Skyprobe plotting daemon
The daemon which generates the skyprobe plots is called srt
and is run on 'wekiu' as user 'skyprobe'. This daemon is based on the
nightd system with the following configuration file
.srtrc
INIT_COMMAND sp_plots init
MAIN_COMMAND sp_plots vnccheck
MAIN_COMMAND sp_plots plot
DONE_COMMAND sp_plots done &DATE
PID_FILE /h/skyprobe/log/srt.pid
LOG_FILE /h/skyprobe/log/srt.log
NIGHT_START 18:00
NIGHT_STOP 06:00
PERIOD 60
TIMEOUT 300
Skyprobe analysis
After sp_command has succesfully taken a picture and obtained
TCS and logger information, the image is ready for processing. This
is perfomed by the program sp_phot. This program performs a
few major tasks:
the raw image is flattened by the program skyproberedccd
(/h/skyprobe/data/rawdir/DATE/filename.fits to
/h/skyprobe/data/smpdir/DATE/filename.flt)
photometry is performed on the flattened image by
sp_sexphot, a script which calls sextractor.
astrometry is performed on the resulting list by gastro
using the Tycho database. The output file is smpdir/DATE/filename.flt.
the stellar photometry is matched and compared to the Tycho
photometry, and a zero-point is measured (addstar). These
measurements are added to the photometry database
(/h/skyprobe/data/catdir).
These programs are part of the Elixir system and are discussed in
other pages.
The program sp_phot also has a test mode called with the
command sp_phot test. In this mode, sp_phot uses a
sample image (/h/skyprobe/data/Refs/sample.fits) to test each
of the stages above. The image is sent through all of the processing
stages, and the resulting entry is searched in the photometry
database. If successful, it deletes the entry and reports the
success.
Odds and Ends
TCS and data logger information are added to the skyprobe header using
the /cfht/bin/tcsh and /cfht/bin/loggerh programs.
These programs, part of the general CFHT observing software system,
add specified information to the FITS header of an image defined by
the environment variable FFTEMPLATE.
Logs from most of the programs go to the skyprobe home directory. Log
messages from the skyprobe daemon are written to
~skyprobe/log/sp_daemon.log. Output from sp_command
expose is written to ~skyprobe/log/sp_expose.log, unless
the -noprocess option has been chosen. Output from the image
analysis (sp_phot) is written to
~skyprobe/log/sp_phot.log. The two daemons send their log
entries to ~skyprobe/log/ert.log and
~skyprobe/log/srt.log.
|