Precision argument for float string formatting

[JakeHagen]

This function works when the precision value is hardcoded but I am having trouble getting the precision argument from python to the c function. I also added a python implementation and tests that I will remove if this gets pulled.

While writing this I started thinking there might be better way. What if the function only added precision when it would round the decimals down to zero using only three decimals. I think the below function would accomplish something like that. It can probably be further optimized to only do extra work when needed.

int
vcz_ftoa(char *restrict buf, float value)
{
    int p = 0;
    int64_t i;

    max_precision = 9;

    if (!isfinite(value) || fabs(value) > INT32_MAX + 1LL) {
        return sprintf(buf, "%.*f", precision, value);
    }

    if (value < 0) {
        buf[p] = '-';
        p++;
        value = -value;
    }

    /* integer part */
    int magnitudes[] = {1, 10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000, 1000000000};

    i = (int64_t) round(((double) value) * magnitudes[max_precision]);
    p += vcz_itoa(buf + p, i / magnitudes[max_precision]);

    int d[precision];
    for (int j = 0; j < precision; j++) {
        d[precision - 1- j ] = (i / magnitudes[j]) % 10;
    }

    buf[p] = '.';
    p++;

    int r = 0;
    bool nonzero = false;
    for (int j = 0; j < precision; j++) {
        int sum_ = 0;
        for (int h = r; h < precision; h++) {
            sum_ += d[h];
        };
        if (sum_ > 0) {
            buf[p] = (char) d[r] + '0';
            if (d[r] > 0) {
                nonzero = true;
            }
            p++;
        };
        r++;
        if (r >= 3 && nonzero) {
            break;
        }
    }

    if (buf[p-1] == '.') {
        buf[p] = '0';
        p++;
    }

    buf[p] = '\0';
    return p;

[jeromekelleher]

This looks great, thanks @JakeHagen! The "glue" side of this is a bit tricky, and I'm happy to step in to sort that out. I think the main thing required now is some thorough testing of ftoa itself. This would be done here.

There's a lot of corner cases to consider I'm afraid. What might be a useful thing to do is to use you python implementation to compare with the equivalent string generated by Python's fixed precision formatting of numbers for a range of values, and then choose some representative cases from that to generate the C test cases. This is a fundamental function, so we do want to make sure we've got all the bases covered.

Regarding your more sophisticated proposal above - I'd rather keep things simple for now and make sure that the semantics we have are well defined and in-line with other implementations (so we can say "a precision of 5 e.g. is equivalent to f"{x:.5f}" in Python"). I don't think there's much point in trying to save a few zeros in the output as vcztools s primarily going to be used to generate input for other programs, where the size isn't all that important (and float cols that need more than 3 decimals of precision are rare anyway).