Skip to content

Prevent XLS precision loss for large integers#637

Open
ViceSaber wants to merge 1 commit into
jazzband:masterfrom
ViceSaber:fix/xls-large-int-precision
Open

Prevent XLS precision loss for large integers#637
ViceSaber wants to merge 1 commit into
jazzband:masterfrom
ViceSaber:fix/xls-large-int-precision

Conversation

@ViceSaber
Copy link
Copy Markdown

Summary

  • prevent precision loss when exporting large integers to XLS format
  • integers exceeding XLS float precision (2^53 - 1) are now stored as text strings

Problem

XLS (Excel 97-2003) stores numeric values as 64-bit IEEE 754 floats with a 53-bit mantissa. Integers larger than 9,007,199,254,740,991 (2^53 - 1) lose precision when stored as numbers. For example, 568883383628111872 becomes 568883383628112000.

Fix

In dset_sheet, detect integers that exceed the float precision limit and convert them to strings before writing. Small integers continue to be stored as numbers for normal behavior.

Testing

Verified that:

  • Large integers (e.g., 568883383628111872) are stored as text (cell type 1) and read back with full precision
  • Small integers (e.g., 42) continue to be stored as numbers (cell type 2)
  • Boolean values are unaffected (not coerced to string)

Fixes #197

XLS stores numbers as 64-bit floats with 53-bit mantissa,
so integers above 2^53-1 lose precision. Convert such
integers to text strings to preserve exact values.

Fixes jazzband#197
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 27, 2026

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.12%. Comparing base (7d6c58a) to head (bff868a).

Files with missing lines Patch % Lines
src/tablib/formats/_xls.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #637      +/-   ##
==========================================
- Coverage   93.14%   93.12%   -0.03%     
==========================================
  Files          29       29              
  Lines        3226     3229       +3     
==========================================
+ Hits         3005     3007       +2     
- Misses        221      222       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hugovk hugovk added the changelog: Fixed For any bug fixes label Mar 27, 2026
@hugovk
Copy link
Copy Markdown
Member

hugovk commented Mar 27, 2026

Thanks for the PR, please add tests.

I've just merged a fix for the lint teyit failure, please also rebase/merge main to include that.

@hugovk hugovk changed the title fix: prevent XLS precision loss for large integers Prevent XLS precision loss for large integers Mar 27, 2026
@claudep
Copy link
Copy Markdown
Contributor

claudep commented Mar 27, 2026

Another approach would have been to document that the XLS format is not adequate to store large integers. Is that format not on a disappearing trend?

@hugovk
Copy link
Copy Markdown
Member

hugovk commented Mar 27, 2026

I'm fine with either approach.

This fix seems simple enough, but I expect XLS use is very low, and outputting XLS even lower.

A docs change is safer, and we've not exactly had a huge number of requests for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: Fixed For any bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong long int value exported

3 participants