CFHT CCD/IRFPA Detector Interface

  Introduction to DetI
  DetI Overview : Notes and Slides
Jean-Charles Cuillandre [Home Page] [jcc@cfht.hawaii.edu] - [CFHT Home Page]

Introduction to DetI

November 1997

The long-needed upgrade of the CFHT detector host software is under final testing and nearing completion. The software was known initially as GenIIIV4, an evolution of GenIIIV3, but was renamed to DetI since it eventually turned into a complete revision designed to operate with the director program. deticli, the command line interface module (called an agent) became the major bulk of the system (90%) while providing powerful capabilities for scripting and GUI.

Requirements for KIR (efficiency of the instrument on the sky) were met by using an architecture where the VME memory is accessed in DMA mode (whereas the CFHT approach back in the early 90's had been entirely Unix-based). DetI also uses all the default hardware and DSP software from Leach (VME Interface Board, Timing and Utility Boards) instead of using in-house designed boards (as was the case for the Utility Board).

Using DetI, the observer can control the detector either from the GUI, from a script, or from a command line interface. DetI-SAOimage (a modified version of SAOimage that can handle data cube format) is the default display tool for DetI, in the future some other tool might be used (ximtool, skycat,...). DetI can run within a Pegasus session (fully compatible with some specific Pegasus configuration files) or in standalone mode (for engineering purposes).

DetI itself is the main layer of the whole system consisting of the session level software (GUI, Director), the detector software (DetI) and the DSP code. DSP codes stands on three different boards : the VME Interface Board (writing data to VME image memory + descrambling), the Timing Board (clocking of the detector, sequencing of the digital conversion) and the Utility Board (exposure timing, temperature control,...). All three Leach boards have their own DSP code downloaded which allows great modularity in the functionality of the whole system.

DetI consists of three software modules : deticli, detio and a SDSU interface driver. deticli is the core of DetI. detio is dedicated to the heavy duty of writing data from the VME memory to disk. There is a driver to communicate with the VME Interface Board. On top of that, director just handles the way commands are sent to deticli.

Filling FITS cards in the header can be time consuming when done sequentially as it was the case in Pegasus. This has been evolved into a parallel approach to reduce the overhead between exposures. In engineering mode, only the detector related FITS cards are updated. Hence reducing the overhead and making the detector acquisition system completely independent from the rest of the telescope, something very useful during the setup of instruments at the summit when for example the detector needs to be tested whereas MOS (for example) has not been hooked up yet.

Also to improve observing efficiency, DetI proposes some features of great interest : full support of data cube format (and in a near future support of the FITS extension format), combination of frames in VME memory during data acquisition (co-adds) and for the infrared detectors the multi-sampling readout.

DetI commands can be combined together in complex scripts to allow efficient optimization of the detectors during observations (mixing TCS and detector commands) or during the test phase in Waimea. Indeed, high level commands like taking exposures are at the same level as setting a voltage on one pin of the CCD for example.

Options on the server (deticli), the VME Interface Board, the Timing and Utility boards can be set by the user either using a text menu, or for some of them (CUBE, MSR, STARING) with dedicated commands from DetI. This allows a great modularity in the functionality of the code. As an example, the ``TURBO'' mode will be a command (for CCDs only) which will make field recognition and optic setup tests much faster.

All commands have on-line help which is automatically displayed when arguments are omitted or invalid syntax is used. There is a complete raster interface in text mode in deticli allowing DetI to run as efficiently at the command line without having to use a GUI. Upon completion of a command, feedback tells the user what the status of the command was and uses the color features of director : green if it was a success, yellow for a warning and red for a error.

director allows a clone window of the main Director window to be opened by anybody. This clone window is an exact copy of the observer window. The engineer can open from any place (Waimea, home) the clone window. Several clone windows can exist at the same time, allowing multiple troubleshooting by several people located in different places. The GUI and scripts commands are all passing through the Director window. This window will log anything requested by the user on the detector as well as on other elements of the instrument since the Director window acts as the standard Pegasus feedback window. Troubleshooters have easy access to a complete log of what's happening on the observer account at the summit! Observing assistant will have such a clone window up on their screen to allow them monitoring what the observers are doing. At first login, engineers only have the observing commands available but they can switch to the set of engineering commands by providing a password.

The DetI GUI is a command/status window which remains on the screen without disappearing when the ``go'' command is requested. This keeps the information about the exposure available to the user : exposure type, exposure time, number of exposures, options for the exposure (staring mode, MSR, data cube format,...) , raster size, observer FITS cards fields. It acts also as a status window for the exposure. As the exposure runs through its different phases, a very obvious and colorful icon tells the user what is happening : cleaning, exposing, reading and then idle. All important options are associated with an icon, making the form highly meaningful by the tired observer who does not have to read anything.

DetI also has some internal automatic warning functionality. This is used to check if the detector temperature is increasing or if the account runs out of disk space. These warnings appear as obvious red icons on the GUI and also as warnings (yellow) in the Director window.

The abort, stop, break actions are fully operational in DetI. When used from the GUI, a confirm window will ask the user first before sending the action command. When used from director (CLI), a confirm prompt also asks the user.

DetI Overview : Notes and Slides

November 1997

Based on the presentation made to the CFHT staff in October 1997.


[Postscript file]  (Notes only)

CONTENTS

  • What people want and need
  • Available hardware
  • Functionality
  • Performance
  • DetI features
  • What is DetI?
  • DetI architecture
  • DetI software structure
  • FITS file creation
  • Noise Reduction Techniques with Reduced Overhead
  • How DetI interacts with an instrument
  • DetI engineering commands
  • DetI observing commands
  • DetI option bits
  • DetI on-line help and command feedback
  • DetI parameter file
  • DetI detector setup file
  • Troubleshooting
  • The DetI GUI
  • What's next



    1.
    What people want and need  [SLIDE]
    • This is a summary of some basic requirements for a data acquisition system meant to be used both as an observing tool and as an engineering tool (including troubleshooting).

    2.
    Available hardware  [SLIDE]
    • This is the existing hardware. The software was known initially as GenIIIV4, an evolution of GenIIIV3, but was renamed to DetI since it eventually turned into a complete revision designed to operate with the director program. deticli, the command line interface module (called an agent) became the major bulk of the system (90%) while providing powerful capabilities for scripting and GUI.
    • Requirements for KIR (efficiency of the instrument on the sky) were met by using an architecture where the VME memory is accessed in DMA mode (whereas the CFHT approach back in the early 90's had been entirely Unix-based). DetI also uses all the default hardware and DSP software from Leach (VME Interface Board, Timing and Utility Boards) instead of using in-house designed boards (as was the case for the Utility Board).

    3.
    Functionality  [SLIDE]
    • Some specific features depending on the context : observing or engineering world.

    4.
    Performance  [SLIDE]
    • The evolution of detector instrumentation at CFHT in the last years brought the initial acquisition system (CCD controller and computer) to its limits. Observing efficiency started being badly limited. This is what the efficiency numbers quoted here for the UH8K and KIR are showing. In contrast, it is not as critical on MOS since the exposure times are much longer than those typically used on infrared detectors.

    5.
    DetI features  [SLIDE]
    • The major features of DetI.

    6.
    What is DetI?  [SLIDE]
    • A short description (text and visual) of what DetI is. The observer can control the detector either from the GUI, from a script, or from a command line interface. DetI-SAOimage (a modified version of SAOimage that can handle data cube format) is the default display tool for DetI, in the future some other tool might be used (ximtool, skycat,...).

    7.
    DetI architecture  [SLIDE]
    • This slide shows the global architecture of the hardware that DetI uses. Data paths and command paths are indicated as well as measured data rates through the communication links. When reading the detector, the data go from the Timing Board to the VME board which writes to VME memory (DMA mode) and then DetI writes to the local disk (it could be a NFS disk too).

    8.
    DetI software structure  [SLIDE]
    • DetI itself is the main layer of the whole system consisting of the session level software (GUI, Director), the detector software (DetI) and the DSP code. DSP code stands on three different boards : the VME Interface Board (writing data to VME image memory + descrambling), the Timing Board (clocking of the detector, sequencing of the digital conversion) and the Utility Board (exposure timing, temperature control,...). All three Leach boards have their own DSP code downloaded which allows great modularity in the functionality of the whole system.
    • DetI consists of three software modules : deticli, detio and a SDSU interface driver. deticli is the core of DetI. detio is dedicated to the heavy duty of writing data from the VME memory to disk. There is a driver to communicate with the VME Interface Board. On top of that, director just handles the way commands are sent to deticli.
    • At startup, deticli checks a few files to load the configuration : the parameter file (.,deti.par) to load the previous user setup (detector, exposure time,...), the setup file (kir.stp) to load the specific configuration for the selected detector (name, size,...), the DSP load files which will be automatically downloaded to the controller (if the IDs mismatch), the .,config file (a Pegasus file) to get the list of handlers to be requested to fill the FITS headers, the .,psm.par (another Pegasus file) to get the observing run ID and the PI name. All these files are read only once. The odometer file is read every time a new filename has to be created. The parameter file will be updated every time the user enters a command that affects one of the values saved in it. This information will keep the GUI updated, even if the user is currently operating the system from the command prompt.
    • director only manages the terminal window. It does not parse the commands that it is sending to deticli, but it does gather simple completion status for each command (either PASS or FAIL). deticli is the core of DetI. This diagram indicates the data path for commands and responses and image data.
    • The signal sigusr1 is sent by the driver to detio when there is some data in VME memory to be written to disk. In the case of KIR, the pixel rate coming down the fiber is so high that the VME Interface Board can only catch the pixels and write them to VME image memory. There is no time to communicate with the driver to tell it that a given number of pixels has been descrambled and written in VME memory. With STIS2 it's a different story since the detector is read out by one output (then no descrambling is required) and the pixel rate is low (40 us per pixel whereas it is 13 us per 4 pixels in KIR!) so the VME board can tell the driver to wake up detio once in a a while to write data to disk. This allows a great reduction of the overhead associated with the memory to disk write time as the image is being written to disk while data are flowing from the detector.
    • There is another data path for the FITS file : the FITS cards have to be filled by deticli and the instrument related handlers. Here is not shown that when interacting in the observing environment, other devices (MOS filter wheel, AOB or TCS for example) have to fill cards in the FITS file. Those activities are launched by DetI without having DetI have a detailed knowledge about it, i.e. DetI does not control anything other than the detector, it launches some predefined tasks in the Pegasus file .,config for a given setup (MOS, AOBIR,...) without having to know what it is. See next slide for a short description of this concept. Currently, the steps for filling the FITS cards are sequential : DetI has to wait on the completion of one call before starting the next one (i.e. wait for AOB to be done before asking TCS to fill his cards). This is done to avoid conflicts on the FITS file access, but it can result in significant time overhead for some instruments (currently 14 seconds on KIR mainly due to get status from AOB hardware, see slide 4). This problem is being addressed.

    9.
    FITS file creation  [SLIDE]
    • There is currently some overhead associated with the sequential way FITS cards are updated. By making this activity run in parallel with the exposure, time can be saved. Currently in Pegasus, there are some actions resulting from the call for the FITS card filling. For example, on MOS for a calibration exposure, the command ``mosh -B'' will fill the FITS cards related to MOS but also check that the calibration mirror is in. If not it will move it. This should be made independent so that FITS cards filling is a unique task which could be run when the detector is exposing/reading while the actions to be made would take place before starting the exposure.
    • In engineering mode, only the detector related FITS cards are updated. Hence reducing the overhead and making the detector acquisition system completely independent from the rest of the telescope, something very useful during the setup of instruments at the summit when for example the detector needs to be tested whereas MOS (for example) has not been hooked up yet.

    10.
    Noise Reduction Techniques with Reduced Overhead  [SLIDE]
    • In order to reduce the overhead during observations, on-line and off-line detector readout techniques can be used by the observer to achieve his goal : to get a given signal-to-noise ratio on a astronomical field as quickly as possible.
    • The data cube format is the simplest way to avoid the overhead related to the FITS cards update. This is done only once per cube and any number of detector frames can be appended to the file. This is valid only if the telescope always points at the same place on the sky (otherwise FITS header field coordinates would not be correct for some frames). The user will have to combine the frames later.
    • Combining frames in VME memory is faster than keeping all the independent frames in a data cube FITS file. It is very useful when taking several calibration data (bias, dark, flat-field) since users generally don't want to keep all independent frames. When dealing with CCDs, a special algorithm (sigma clipping) for the combination has to be used to efficiently remove the cosmic rays. This is not a concern for the IRFPA where a simple sum (average) is the best approach.
    • IRFPA detectors have the great advantage over the CCDs of being non-destructive readout devices. The same frame can be read several times. This is called Multi-Sampling Readout. It is a way to reduce the readout noise for images where the readout noise is dominant (narrow band imaging) versus broad band imaging where the sky background photon noise dominates. In this second case, the only way around that is the use of one of the two previous techniques.

    11.
    How DetI interacts with an instrument  [SLIDE]
    • Integrating DetI within an instrument session while providing common capability to the user to control various parts of the instrument (i.e. at the command line or in scripts for filter selection and telescope offsets) has been foreseen in director. At startup, director loads the command table from its agent (DetI) and also registers scripts specific to this account. When commands are entered by the user, director will first check if it is a builtin command, or a session script or at last a DetI command (error otherwise). These scripts will use existing control software (mosh, kirwh,...) while providing a simple syntax. These scripts can also redefine a command to the agent. For example the ``go'' command in MOS will become a script that will check that the calibration mirror is in the right place depending on the exposure type and then send the real ``go'' command to DetI. Or in KIR that the blank has been set in the filter wheel before taking dark exposures. In both sessions, these scripts would have the same name but would be specific to the account. These scripts can also be used to create aliases : for example ``fn test'' will be equivalent to the DetI `filename test.fits'' command.

    12.
    DetI engineering commands  [SLIDE]
    • This slide shows how the engineering DetI window looks and lists all the engineering commands available. Note the status of the instrument at the bottom and also the status bar on top of the prompt (``>'') showing the status of the next exposure. As the user is typing his commands, he will see these fields changing in real time.
    • These commands can be combined together in complex scripts to allow efficient optimization of the detectors during the test phase in Waimea. Indeed, high level commands like taking exposures are at the same level as setting a voltage on one pin of the CCD for example.

    13.
    DetI observing commands  [SLIDE]
    • Same as previous slide but for the observing DetI version (setup by starting deticli with the option ``-s'').
    • The infrared community is very demanding on scripting capabilities. DetI allows them to write their own scripts mixing detector commands and AOB/TCS commands (see notes on slide 8 and 9).

    14.
    DetI option bits  [SLIDE]
    • Options on the server (deticli, the agent), the VME Interface Board, the Timing and Utility boards can be set by the user either using a text menu or for some of them (CUBE, MSR, STARING) with dedicated commands from DetI (see slides 12 and 13). This allows a great modularity in the functionality of the code. As an example, the ``TURBO'' mode will also become a special commands (for CCDs only) which will make field recognition and optic setup tests much faster. This complex menu is not available to observers, but only the dedicated commands quoted above. This is also available in the form of push buttons on the GUI.

    15.
    DetI on-line help and command feedback  [SLIDE]
    • All commands have on-line help which is automatically displayed when arguments are omitted or invalid syntax is used.
    • There is a complete raster interface in text mode in deticli allowing DetI to run as efficiently at the command line without having to use a GUI. Note that the correct range for the arguments are provided. Some rasters can't be modified (full detector unbinned, binned by 2 and 4).
    • Upon completion of a command, feedback tells the user what the status of the command was and uses the color features of director : green if it was a success, yellow for a warning and red for a error. director also writes these messages into the standard CFHT log files on the session host.

    16.
    DetI parameter file  [SLIDE]
    • The parameter file is used to keep track of the current DetI status between status, or in the event of a restart. It is also used to pass status informations to the hform GUI.

    17.
    DetI detector setup file  [SLIDE]
    • The detector setup file contains all the information required to run a given detector. DSP files to download, expected IDs, characteristics of the detector (size, number of amplifiers and relative positions, data for the FITS header,...) and all low-level values to run the Leach system. This includes memory location of all DSP variables to allow DetI to write exposure parameters at the right place during the pre-exposure sequence.
    • For safety reasons and simplicity, there is a unique copy of this file per system (Sparc1E) in a well defined place (``/usr/local/cfht/dev/conf'').

    18.
    Troubleshooting  [SLIDE]
    • director allows a clone window of the main Director window to be opened by anybody. This clone window is an exact copy of the observer window. On this slide, the engineer can open from any place (Waimea, home) the clone window. Several clone windows can exist at the same time, allowing multiple troubleshooting by several people located in different places. The GUI and scripts commands are all passing through the Director window. This window will log anything requested by the user on the detector as well as on other elements of the instrument since the Director window acts as the standard Pegasus feedback window. Troubleshooters have easy access to a complete log of what's happening on the observer account at the summit!
    • At first login, engineers only have the observing commands available but they can switch to the set of engineering commands by providing a password.
    • The observing assistant should always have a clone window (read only mode) up on his terminal. A great source of information that would improve efficiency. For example, it often happens that the telescope has to be moved at the end of the exposure, the observing assistant will just have to check for the green feedback rather than waiting for the observer to see the green message announcing the end of the exposure and then telling about it...

    19.
    The DetI GUI  [SLIDE]
    • The DetI GUI is an hform. This command/status window remains on the screen without disappearing when the ``go'' command is requested. This keeps the information about the exposure available to the user : exposure type, exposure time, number of exposures, options for the exposure (staring mode, MSR, data cube format,...), raster size, observer FITS cards fields. It acts also as a status window for the exposure. As the exposure runs through its different phases, a very obvious and colorful icon tells the user what is happening : cleaning, exposing, reading and then idle. Another status field provides the name of the current exposure (in this example, ``C[1]'' means first frame of the cube). All important options are associated with an icon, making the form highly meaningful by the tired observer who does not have to read anything.
    • DetI also has some internal automatic warning functionality. This is used to check if the detector temperature is increasing or if the account runs out of disk space. These warnings appear as obvious red icons on the GUI and also as warnings (yellow) in the Director window.
    • The abort, stop, break actions are fully operational in DetI. When used from the GUI, a confirm window will ask the user first before sending the action command. When used from director (CLI), a confirm prompt also asks the user.
    • The DetI Raster Form can be opened when needed by clicking on the ``Edit rasters'' button on the DetI Exposure Form. If the raster form was iconified then it automatically pops up.

    20.
    What's next  [SLIDE]
    • Upgrading the current hardware (computers and controllers) is the only way to keep up with the requirements set by the evolution of the new detectors. This slide compares what is achieved today compared to what our goals are for the coming years (CFH12K and KIR upgraded should run by next spring). There are still many other important issues for the software to address, but the hardware is now becoming the real bottleneck for our present system.


    Acronyms

    AOB : CFHT Adaptive Optic Bonnette
    AOBIR : Adaptive Optic Bonnette Infrared
    CFH12K : Canada-France-Hawaii 12Kx8K CCD mosaic camera
    CLI : Command Line Interface
    DMA : Direct Memory Access
    DetI : CFHT CCD/IRFPA Detector Interface
    GUI : Graphical User Interface
    ID : Identificator
    IRFPA : Infrared Focal Plane Array
    KIR : CFHT 1Kx1K Infrared camera
    MSR : Multi-Sampling-Readout
    SDSU : San Diego State University
    STIS2 : CCD coming from the HST Imaging Spectrograph developments
    TCS : Telescope Control System
    UH8K : University of Hawaii 8Kx8K CCD mosaic camera

    Jean-Charles Cuillandre [Home Page] [jcc@cfht.hawaii.edu] - [CFHT Home Page]