Hi Fred,
At your end, cadcCRC of the uncompressed files should match what we
have locally. The switched to storing only the uncompressed versions
of T000X was made for T0004.
Also, I've started my script that creates an MD5 of the files that we
have stored locally, these should be available soon. The users will
then have the md5 value to ensure the version they have matches the
version we have.
cheers,
JJ
On 21-Aug-08, at 7:40 PM, Frederic Magnard wrote:
>
> Hi all,
>
> Thereis a problem with the CRC checksums of all T05 data transfered to
> CADC: none match !
> Exemple:
> /W1/chi2/CFHTLS_W_gri_020241-041200_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x634941c0 TPX: 40ca6006
> ./W1/chi2/CFHTLS_W_gri_020241-041200_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x523803d0 TPX: 003c3686
> ./W1/chi2/CFHTLS_W_gri_020241-050800_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x135f1cf4 TPX: 2243fe3c
> ./W1/chi2/CFHTLS_W_gri_020241-050800_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x82ed680f TPX: 95a195ea
> ./W1/chi2/CFHTLS_W_gri_020241-060400_T0005.fits.fz ERROR: CRC does
> not match CADC: 0xdc3749d3 TPX: 07c635bb
> ./W1/chi2/CFHTLS_W_gri_020241-060400_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x76dab408 TPX: 957f5deb
> ./W1/chi2/CFHTLS_W_gri_020241-070000_T0005.fits.fz ERROR: CRC does
> not match CADC: 0xf97a108e TPX: e7aa687b
> ./W1/chi2/CFHTLS_W_gri_020241-070000_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x8161fe62 TPX: a8088c8b
> ./W1/chi2/CFHTLS_W_gri_020241-075600_T0005.fits.fz ERROR: CRC does
> not match CADC: 0xd614a77b TPX: dc523bef
> ./W1/chi2/CFHTLS_W_gri_020241-075600_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x75e32f3c TPX: 8d7c01f2
> ./W1/chi2/CFHTLS_W_gri_020241-085200_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x8a67a22e TPX: a108da31
>
> ...
>
> ./W1/stacks/CFHTLS_W_g_020241-104400_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0xabc4bc77 TPX: 04a4fad8
> ./W1/stacks/CFHTLS_W_g_020631-041200_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x8da292b4 TPX: 156cd997
> ./W1/stacks/CFHTLS_W_g_020631-041200_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0xdc65b23f TPX: 29ac45bb
> ./W1/stacks/CFHTLS_W_g_020631-050800_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x01e09831 TPX: c6ff6484
> ./W1/stacks/CFHTLS_W_g_020631-050800_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x3d518baf TPX: bad646a4
> ./W1/stacks/CFHTLS_W_g_020631-060400_T0005.fits.fz ERROR: CRC does
> not match CADC: 0xc988f6e0 TPX: ee38ba54
> ./W1/stacks/CFHTLS_W_g_020631-060400_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x6a8c49cd TPX: 70d7a622
> ./W1/stacks/CFHTLS_W_g_020631-070000_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x45adc572 TPX: 3981837c
> ./W1/stacks/CFHTLS_W_g_020631-070000_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0x53fd56dc TPX: f406454b
> ./W1/stacks/CFHTLS_W_g_020631-075600_T0005.fits.fz ERROR: CRC does
> not match CADC: 0x6a956aa4 TPX: 0e0fd94d
> ./W1/stacks/CFHTLS_W_g_020631-075600_T0005_weight.fits.fz ERROR: CRC
> does not match CADC: 0xb95726f9 TPX: 5f0d9d1a
> ./W1/stacks/CFHTLS_W_g_020631-085200_T0005.fits.fz ERROR: CRC does
> not match CADC: 0xc92f43f3 TPX: a72dd9cd
>
> ...
>
> ./D1-25/stacks/CFHTLS_D-25_i_022559-042940_T0005.fits.fz ERROR: CRC
> does not match CADC: 0x9c2c2846 TPX: 696e5b96
> ./D1-25/stacks/CFHTLS_D-25_i_022559-042940_T0005_weight.fits.fz
> ERROR: CRC does not match CADC: 0x3a969ab5 TPX: 2de60e76
> ./D1-25/stacks/CFHTLS_D-25_r_022559-042940_T0005.fits.fz ERROR: CRC
> does not match CADC: 0x3caef134 TPX: 7a0c4dc7
> ./D1-25/stacks/CFHTLS_D-25_r_022559-042940_T0005_weight.fits.fz
> ERROR: CRC does not match CADC: 0x4f237cb6 TPX: 5e57c685
> ./D1-25/stacks/CFHTLS_D-25_u_022559-042940_T0005.fits.fz ERROR: CRC
> does not match CADC: 0x4fb9721a TPX: a63c184a
> ./D1-25/stacks/CFHTLS_D-25_u_022559-042940_T0005_weight.fits.fz
> ERROR: CRC does not match CADC: 0x77781bfc TPX: e1c193ef
> ./D1-25/stacks/CFHTLS_D-25_z_022559-042940_T0005.fits.fz ERROR: CRC
> does not match CADC: 0x67787fc2 TPX: 0fb6de4d
> ./D1-25/stacks/CFHTLS_D-25_z_022559-042940_T0005_weight.fits.fz
> ERROR: CRC does not match CADC: 0x5e4cb523 TPX: f1b9d9cc
> ./D1-85/chi2/CFHTLS_D-85_gri_022559-042940_T0005.fits.fz ERROR: CRC
> does not match CADC: 0x45c6abb6 TPX: a7b0d13b
>
> etc...
>
> If the file stored at cadc is the same as the one we have moved to
> /data/fcix3/raid/cadc/new/, both checksums should be the same.
>
> I've used ~cadc/bin/cadcCRC to get the CRC checksum of local files,
> and
> curl -s --location-trusted -g "http://www.cadc.hia.nrc.gc.ca/cadcbin/cfhtInfo?file=$
> {cadc_name}&options=-fileCrc"
> where ${cadc_name} is the local file name, minus extensions
> like .fits or .fz, and substituting any + with %2B.
> Could you please investigate this problem, as for now, we have no
> way to be
> sure that the data available at cadc are correct.
>
>
>
> Moreover, a few files appear in /data/fcix3/raid/cadc/reject/name
> for no visible reason:
> CFHTLS_D-85_r_221531-174356_T0005.fits_weight.fz
> CFHTLS_W_z_090137-012700_T0005_weight.fits.fz
>
>
>
> In not_new, there are 3 files which should not be there, but I'll be
> able to
> solve this problem when the CRC checksums will be working:
> CFHTLS_W_g_022929-060400_T0005.fits.fz
> CFHTLS_W_z_085011-031900_T0005.fits.fz
> CFHTLS_W_z_141754+552631_T0005_weight.fits.fz
>
> Cheers,
> Fred.
>
>
Received on Thu Aug 21 2008 - 19:25:44 HST