Skip to content

testbench1.cpp: undefined behavior in signed saturation reference code for testcases 610 and 612 #1

Description

@oltolm

The reference implementation for compress_saturated in testbench1.cpp is incorrect for signed destination types in testcases 610 and 612.

It computes the saturated result with:

(RT(1) << (sizeof(RT) * 8 - 1)) - (elements1[i] > 0)

This has undefined behavior for signed RT because it left-shifts into the sign bit, and for positive overflow cases it can overflow again when subtracting 1.

As a result, the testbench reference code can produce incorrect expected results even when compress_saturated itself is correct.

Expected behavior

For signed narrowing saturation, the reference code should clamp to std::numeric_limits<RT>::max() for positive overflow and std::numeric_limits<RT>::min() for negative overflow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions