Hi JJ,
Thanks for the quick answer ! :)
On Tue, 20 Mar 2007, JJ Kavelaars wrote:
> > >I can add the CRC values we produce to the megaprime_proxy table.
> >
> >Yes, that would be great. I can then move to new only the needed files.
> >Could you please also explain again how to use it (and its output format),
> >as I guess it might have changed a bit ?
> >
> >The list of weight images is ready, just waiting to be filtered to eliminate
> >already transfered files. This time, the links will point to NFS mounted
> >filesystems. That's why, I asked, for the near future, to have one
> >e-transfer daemon per machine hosting the data.
> >
> >The proxy http://cadcwww.hia.nrc.ca/cadcbin/cfhtInfo seems to still be
> >active, could you please explain how to use it, if it can be useful ? It
> >looks like it can return the CRC too.
>
> If you use the cfhtInfo proxy then you can get all the info you need and I
> don't need to bother with the megaprime_table changes. To get the CRC for a
> file [such as file 789054p.fits] use the command:
>
> curl
> 'http://cadcwww.hia.nrc.ca/cadcbin/cfhtInfo?file=789054p&options=-fileCrc'
>
> this will return the CRC of the file. [you just take the .fits of the filename
> in the URL] So, you can use cfhtInfo to check the CRC.
OK, I'll use the cfhtInfo proxy to filter the step1 weight images.
But could you include the crc into the megacam_proxy anyway ? The way we
check the availability of new images is to download the whole table, and
compare with the local images. If an image is upgraded on your side, its
name won't change, only the crc will be different. So it makes sense to
gather those information in a single table downloadable through a single
wget, rather than a table and 15000+ cross-correlated other wget...
Cheers,
Fred.
Received on Tue Mar 20 2007 - 19:26:09 HST