rawspec failing on ATA_guppi_59730_78085_225768981_J0332+5434_off_0001-beam0000

[texadactyl]

rawspec  -f 1 -t 32    -g 3  -d .  /mnt_blpd20/scratch/rawspec_testing/baseline//ATA_guppi_59730_78085_225768981_J0332+5434_off_0001-beam0000
rawspec 3.1.1+11@g188981a using librawspec 3.1.1+11@g188981a and cuFFT 10.2.1.245
using requested GPU: 3
writing output files in SIGPROC Filterbank format
working stem: /mnt_blpd20/scratch/rawspec_testing/baseline//ATA_guppi_59730_78085_225768981_J0332+5434_off_0001-beam0000
opening file: /mnt_blpd20/scratch/rawspec_testing/baseline//ATA_guppi_59730_78085_225768981_J0332+5434_off_0001-beam0000.0000.raw
bad obsnchan/nants: 192 % 20 != 0
output product 0: 0 spectra

It must be one of the 2 commits after June 1st because regression testing was successful that day.
See https://github.com/UCBerkeleySETI/rawspec/commits/master

@radonnachie Is this file no longer valid? I got it from you: UCBerkeleySETI/rawspec_testing@9a36ae2

[texadactyl]

@wfarah I have tried to assign @radonnachie but he does not appear in the assignee list (?). Basically, it seems like a file that Ross gave me is no longer valid because obsnchan/nants: 192 % 20 != 0. Maybe we can replace this file?

[radonnachie]

Yeah it is invalid because it required the NBEAMS override of NANTS that I've recently redacted from #58. The NANTS value in this beam file should be 1.

[texadactyl]

@radonnachie Is a replacement available?

[radonnachie]

I have just run rsync -av /mnt/buf1/blade/GUPPI/./guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw rsync://blpd18.ssl.berkeley.edu/datax/ and manually inspected it to have NANTS=1, NBEAM=2 and BEAM_ID=1.

Prior to that I transferred rsync -av /mnt/buf0/blade/GUPPI/./guppi_59811_38073_118274414_AzEl_0001-beam0000.0001.raw rsync://blpd18.ssl.berkeley.edu/datax/, which has NANTS=1 and NBEAMS=1, so I'd ask you to discard it please (I've yet to actually log in to the Berkeley servers.

[texadactyl]

@radonnachie
Downloaded:

rsync -v rsync://blpd18.ssl.berkeley.edu/datax/guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw .
guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw

sent 43 bytes  received 8,525,911,930 bytes  215,845,872.73 bytes/sec
total size is 8,523,830,784  speedup is 1.00

Prepended "ATA_" to file name.
Tried to use it:

rawspec  -f 1 -t 32  -j  -g 3  -d .  ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001
rawspec 3.1.1+11@g188981a-dirty using librawspec 3.1.1+11@g188981a-dirty and cuFFT 10.2.1.245
using requested GPU: 3
writing output files in FBH5 format
working stem: ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001
opening file: ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw
Number of spectra per dump (Nd) = 256
Number of fine channels per coarse channel (nfpc) = 1
Chunk dimensions = (256, 1, 1)
Cache nslots = 257, nbytes = 196609
 BLOCSIZE not found in header
error getting obs params from ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw [No such file or directory]
opening file: ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001.0001.raw [No such file or directory]
output product 0: 1536 spectra

"BLOCSIZE not found in header"?

watutil -i ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001.rawspec.0000.h5

--- File Info ---
DIMENSION_LABELS : [b'time' b'feed_id' b'frequency']
        az_start :                              0.0
       data_type :                                1
            fch1 :                       6240.0 MHz
            foff :                          0.5 MHz
           ibeam :                                1
      machine_id :                               20
          nbeams :                                1
           nbits :                               32
          nchans :                              192
            nfpc :                                1
            nifs :                                1
     rawdatafile : ATA_guppi_59811_38723_118314086_AzEl_0001-beam0001.0000.raw
     source_name :                             AzEl
         src_dej :                    67:15:07.0502
         src_raj :                    12:33:08.4039
    telescope_id :                                9
           tsamp :                          6.4e-05
   tstart (ISOT) :          2022-08-20T10:45:23.000
    tstart (MJD) :                59811.44818287037
        za_start :                              0.0

Num ints in file :                             1536
      File shape :                   (1536, 1, 192)
--- Selection Info ---
Data selection shape :                   (1536, 1, 192)
Minimum freq (MHz) :                           6240.0
Maximum freq (MHz) :                           6335.5

nfpc=1? Number of fine channels per coarse channel = FFT length = 1? This bothered me with the previous file too. How could dedoppler search a coarse channel of width 1? Of course, for this testing, we are not doing dedoppler searching.

How does that Waterfall info display look to you other than the nfpc value?

[texadactyl]

@radonnachie Was my rawspec -f value wrong? How about the -t value? I was using the same values that you recommended for the previous file (ATA_guppi_59730_78085_225768981_J0332+5434_off_0001-beam0000) which is now discarded.

[texadactyl]

@radonnachie The new file successfully replaces the old one in rawspec_testing. I will try out the floating_point PR now that this is resolved.

[radonnachie]

The waterfall header-info displayed looks good to me. The nfpc is surely the result of -f, as you said the FFT length. In reality we'd probably really want to get to 1 Hz wide fine channels, so -f 524288. That would require much more coarse time though, for even a single fine-spectra output. It seems I only provided you with 32*1536=49152 time samples. Thanks @texadactyl!