[Date Prev][Date Next][Date Index]
Meeting approaching + data flow
Hello!
The meeting is approaching, and by now you have probably
booked your flights for our meeting in Paris, Jan. 22 to 24.
Let me use this reminding mail to comment on another issue we
have to address: the access to the data for the communities
involved in the surveys (I already commented on the access
for the others in my previous email).
- Data are processed through Elixir in near real time for
data quality assessment. Elixir is the pipeline running with
the Queued Service Observing (QSO) which will be used for the
survey(s).
- Date are processed at CFHT to get the instrument signature removed
(bias, flat and fringe pattern) after a run (a period in which
MegaCam is on the sky, typically the 15/20 days around New Moon).
It again follows the QSO strategy.
- Data are sent to Terapix on a monthly basis for further processing
(resampling, stacking, object extraction and catalog).
- Terapix products are made available as they are produced (after
the required data quality assessment).
For the real time scientific application (SNe, moving objects, ...),
I would like to propose the approach we have started for the French
SNe group: data to be used for these programs are flagged and sent
to linux boxes in Waimea, where they are processed in the way needed
for the programs. Terapix is not designed for real time operations
and won't provide the software for that. Those who are interested in
real time processing have to get together and work on that... As there
is only a very limited data flow needed between cfht and those monitoring
the processing from their institution if the process is made at cfht,
the programs can be monitored remotely in an easy way. Disk space and
processing power is now cheap enough to make this approach feasible.
Now, the main question is: do we rely only on one processing center
for the object extraction and catalog building? I would have said yes
for sure five years ago, as the cost of data storage and processing
power would have made a duplication very expensive. Today, I'm not that
convinced that a single center is the best way to go.
The Hipparcos way of doing things, with two processing centers working
in parallel with their own algorithms, but comparing their results for
the publication of a single catalog at the end ,is quite appealing.
Well, France has already worked a lot for Terapix.
Would Canada for example be interested and have the
resources to have a Terapix-bis? Is it wise? Think about it!
In any case, the raw data will be saved and linked to the calibration
data (bias, flat and fringe pattern) to be used for the preprocessing.
To make them widely available to everybody in the communities participating
in the survey(s) is just too much. Having them available in each participating
"country" could make sense, though the model right now is to go through
Terapix first.
Have a nice day
Aloha
Christian
--
************************************************************
Dr. Christian Veillet, CFHT Senior Resident Astronomer
Phone: (808) 885-3161 http://www.cfht.hawaii.edu/~veillet/
************************************************************