Re: Reprocessing procedure and timing

From: Jean-Charles Cuillandre <jcc_at_cfht.hawaii.edu>
Date: Thu, 14 Feb 2008 14:57:38 -1000

 Hi JJ,

> > One question for CADC: we could start the reprocessing on 07A and
> > uploading to CADC tomorrow/next week, but do we want to replace old
> > files yet?
>
> Well.. If you are not 100% certain that this is a straight replace,
> then we best not remove the old stuff just yet.
Yes, I agree, let's play safe (and even after we remove them, Elixir
can still recreate the old versions - as long as we keep the headers
to track down the processing history).
 
> > Since time is of the essence and download time is the
> > limiting factor, I recommend we move on with the reprocessing of
> > the LS asap for 07A, then 06B, then 07B. My feel is that it'll
> > take more than the small test set to really validate this Elixir
> > B4 release and having 06B/07A/07B at Terapix through CADC would be a
> > good thing.
>
> OK. So, how much space is needed for this 'flow through' from CFHT-
> >CADC->Terapix? We could makeup something pretty quickly that would
> handle this problem but we have to be careful with storage issues. If
> the total space requirement [for LS 06B/07A/07B ] is less then ~ 10TB
> we are likely fine.. if the value is up at 20 or so, then we will need
> to deploy some additional storage and currently we are not planning to
> schedule that until mid April.
>
>
> JCC: Can you estimate how much storage is needed? I'm really not
> very comfortable with removed the current ELIXIR data from the
> archive... certainly not until we are completely comfortable that the
> the new stuff is correct.

Just the LS data for 07B/07A/06B represents 5Tb. Please let us know
when we can start pushing those runs to CADC. By the time 07B/07A/06B
Elixir B4 is at CADC, we should have the conclusion of the analyses
from Terapix (and others as well - I am asking Stephane Arnouts to
study some Deep data for Photometric Redshifts). If for some reason
we see something really bad - I would just have to revert the recipie
since Nov 07, while Terapix would retrieve the "old" (current) versions
of 07B/07A/06B that you're keeping until then.

                                                Jean-Charles.
Received on Thu Feb 14 2008 - 14:57:38 HST