Skip to content

Frequency values extracted not commensurate with expectated values #5

@eagoetz

Description

@eagoetz

When extracting a transfer function from a DTT file using dttxml, the frequency values do not match the ASCII exported values from DTT. Below is a print comparison

dttxml freq = 7.0; ASCII freq = 7.0
dttxml freq = 7.652900218963623; ASCII freq = 7.6529
dttxml freq = 8.366600036621094; ASCII freq = 8.3666
dttxml freq = 10.0; ASCII freq = 10.0
dttxml freq = 12.115300178527832; ASCII freq = 12.1153
dttxml freq = 14.678000450134277; ASCII freq = 14.678
dttxml freq = 15.600000381469727; ASCII freq = 15.6
dttxml freq = 16.399999618530273; ASCII freq = 16.4
dttxml freq = 17.100000381469727; ASCII freq = 17.1
dttxml freq = 17.600000381469727; ASCII freq = 17.6
dttxml freq = 19.100000381469727; ASCII freq = 19.1
dttxml freq = 21.544300079345703; ASCII freq = 21.5443
dttxml freq = 26.101600646972656; ASCII freq = 26.1016
dttxml freq = 31.622800827026367; ASCII freq = 31.6228
dttxml freq = 38.3119010925293; ASCII freq = 38.3119
dttxml freq = 46.41590118408203; ASCII freq = 46.4159
dttxml freq = 56.234100341796875; ASCII freq = 56.2341
dttxml freq = 68.12920379638672; ASCII freq = 68.1292
dttxml freq = 82.54000091552734; ASCII freq = 82.54
dttxml freq = 100.0; ASCII freq = 100.0
dttxml freq = 134.2239990234375; ASCII freq = 134.224
dttxml freq = 177.8280029296875; ASCII freq = 177.828
dttxml freq = 237.13699340820312; ASCII freq = 237.137
dttxml freq = 292.21099853515625; ASCII freq = 292.211
dttxml freq = 340.65399169921875; ASCII freq = 340.654
dttxml freq = 433.70001220703125; ASCII freq = 433.7
dttxml freq = 490.3269958496094; ASCII freq = 490.327
dttxml freq = 543.2130126953125; ASCII freq = 543.213
dttxml freq = 625.7420043945312; ASCII freq = 625.742
dttxml freq = 699.4650268554688; ASCII freq = 699.465
dttxml freq = 749.8939819335938; ASCII freq = 749.894
dttxml freq = 824.2650146484375; ASCII freq = 824.265
dttxml freq = 962.7639770507812; ASCII freq = 962.764
dttxml freq = 1221.5699462890625; ASCII freq = 1221.57

Note that some frequency offsets are 1/2^18. Not sure if that is a clue as to the origin of the differences, as they are much larger (and in some cases consistent with neighbors) than floating point errors

I am running dttxml-1.1.4 installed via pip with python-3.7. If I can provide any other details, just let me know.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions