I agree that it is too early to do a global reprocessing. Current elixir
has significant variations over the camera field and it really should be
brought into the SDSS/CALSPEC system more fully. The variations are
enough to cause problems with photo-z's
SNLS does not need anything to be done, although using SNLS calibration
will vastly increase the CFHTLS legacy value. SNLS does its own
post-processing of elixir data, although a number of details remain
under debate.--which is one strong reason to wait until there is a
publication on these issues.
Jean-Charles Cuillandre wrote:
> Hi,
>
> This has been in the air for some time: integrating the latest findings from
> SNLS on the photometry accuracy of the photometric flat to be applied for a
> global reprocessing of the LS data since first light, while addressing other
> issues such as zero points stability and global recipe consistency.
>
> We are however facing issues that are forcing us to take some steps back on
> this plan. I already consulted Yannick, Christian and Chris today about what's
> best, and it appears wise to postpone to T0006 the global reprocessing. Please
> feel free as a SG member to voice your view on this decision. Here are the
> elements that dictated the choice:
>
> 1) Bandwidth:
> The southern cross network access that was supposed to bring a
> 30 to 40Mb/sec CADC-CFHT connection did not materialize: we're
> still getting only 3Mb/sec max. With 25000 frames of 700Mb each
> to reprocess and transfer both ways, that's a total of 120 days,
> not counting in there the overheads for running Elixir and the
> DADS distribution. So, we'd be looking at 5 or even 6 months to
> get all the data from/to CADC, that is not acceptable as we
> must stick to our commitment to our community and release T0005
> this spring (May, or June at the latest).
>
> 2) Flat recipes:
> SNLS has strong requirements on the photometry and they have
> addressed some limitations caused by the Elixir native processing.
> But all the other programs can fully deliver with the 3% photometry
> precision we have right now. There is more work needed to validate
> the SNLS recipes as they get integrated in Elixir: preliminary
> results (by Terapix and S. Arnouts on photometric redshifts) with the current understanding of the flats strategy show no gain over
> the latest Elixir release.
>
> 3) Data consistency:
> The current set of data at Terapix used for T0004 has various
> flavors of Elixir flats: B1, B2, and B3. The global reprocessing
> had the advantage of making all data processed the same way (B4).
> But then again, this is not (or yet anyway) a limitation for programs
> other than SNLS which has its own data set under control. B1 was
> however not the best, and I believe the B1 data set (which is
> pretty minimal, 10 or 12 runs) could be reprocessed with B3.
> B4 has been active since December, hence the past 3 runs will be
> reprocessed with B3 as well and pushed at CADC (Terapix has yet
> to retrieve any data from the second half of 06B, all of 07A, all
> of 07B and 08A - and they can go ahead right away with 06B/07A and
> the first four runs of 07B - this is urgent as the Wide observing
> strategy needs data quality assesment to define the Wide patching
> in 08A).
>
> 4) Zero points:
> Clearly this was an issue and my recent message explained some
> of the gains on that one. It is still important to address this for
> T0005, hence I will re-apply that method to the ZPs of data
> processed with the B3 Elixir data. Terapix will be provided with
> an updated set of ZPs for all runs since first light, and they will
> have to integrate this somehow in their pipeline: the Elixir data
> will not reflect for now that change (i.e. nothing will change on
> the CADC files, or even the Elixir files already at Terapix).
>
> In summary:
> o The global reprocessing will not happen, at most it will be the
> first year of LS data (03B/04A) that will be reprocessed with the
> best flat Elixir recipe to date (B3). The past three runs since B4
> was activated will be reprocessed with B3.
> o All runs zero points will be updated but not within the FITS files.
>
> What we are facing for T0006:
> 1) Address the bandwidth issue. 6 months simply isn't reasonable
> to consider for a reprocessing.
> 2) Fully characterize and understand why the gain achieved by SNLS
> does not realize when integrated in Elixir. There is a lot of
> valuable work there and the data will eventually benefit from it.
> 3) Fringing, fringing, fringing...
> 4) Amplifier stability: that is a hard one, but it is becoming a
> limitation and it needs to be looked at.
> 5) Use the Deep fields as the new photometric standards. The absolute
> calibration program is about to be analyzed by CFHT and SNLS.
>
> By taking this decision now, not much time from any of us has been wasted yet.
> Hopefully you will all agree this is the best path to take at this point to
> serve our users.
>
> Please let me know if you have any comments,
>
> Jean-Charles.
>
>
Received on Wed Mar 12 2008 - 16:44:44 HST