https://archives.boost.io/release/1.91.0/source/boost_1_91_0.tar.bz2 is a broken file

[vobloeb]

https://archives.boost.io/release/1.91.0/source/boost_1_91_0.tar.bz2 is a broken file (bz2 crc error)

SHA256 de5e6b0e4913395c6bdfa90537febd9028ea4c0735d2cdb0cd9b45d5f51264f5 is correct as published on https://www.boost.org/releases/1.91.0/

.tar.gz has no problems

[mclow]

I just download 'boost_1_91_0.tar.bz2' from https://archives.boost.io/release/1.91.0/source

% shasum -a 256  ~/Downloads/boost_1_91_0.tar.bz2
de5e6b0e4913395c6bdfa90537febd9028ea4c0735d2cdb0cd9b45d5f51264f5  /Users/marshall/Downloads/boost_1_91_0.tar.bz2

The file expands for me w/o error.

[vobloeb]

[vobloeb]

[sdarwin]

Use these commands:

root@server:/tmp# sha256sum boost_1_91_0.tar.bz2 
de5e6b0e4913395c6bdfa90537febd9028ea4c0735d2cdb0cd9b45d5f51264f5  boost_1_91_0.tar.bz2
root@server:/tmp/test# bunzip2 boost_1_91_0.tar.bz2 

[vobloeb]

Use these commands:

root@server:/tmp# sha256sum boost_1_91_0.tar.bz2 
de5e6b0e4913395c6bdfa90537febd9028ea4c0735d2cdb0cd9b45d5f51264f5  boost_1_91_0.tar.bz2
root@server:/tmp/test# bunzip2 boost_1_91_0.tar.bz2 

#1147 (comment)

[sdarwin]

@vobloeb do you see the year in the version of bunzip2. How many years ago was it released.

[vobloeb]

@vobloeb do you see the year in the version of bunzip2. How many years ago was it released.

Do you expect the CRC constant got out of fashion?

Also, it is the only such file: 1.89, 1.90 have no problem

And the more recent 7zip consider the file broken as well #1147 (comment)

[vobloeb]

bzip2, a block-sorting file compressor. Version 1.0.8, 13-Jul-2019. is also unable to unpack the file

I believe you guys use a particular platform (Apple maybe?) whose bzip2 is able to create and unpack archives, which all the available Linux archivers cannot unpack.

[sdarwin]

Which exact operating system and version are you using?

If I run "7z t boost_1_91_0.tar.bz2" it says "Everything is Ok". This is Ubuntu 24.04 which is based on Debian, so both of those would likely work.

[vobloeb]

I said all the archiver versions.

AI hints the bz2 file may have "Trailing bytes / padding" which some archivers may ignore

1.0.8 is effectively the latest official bzip2 https://sourceware.org/bzip2/ and it is unable to unpack your file

[sdarwin]

I asked "Which exact operating system and version are you using?" and your answer is "I said all the archiver versions."

[vobloeb]

I asked "Which exact operating system and version are you using?" and your answer is "I said all the archiver versions."

Linux, the exact distro seems as relevant as for example timezone - just a stupid L1 support question to keep people online.
There is no problem like "I need help to unpack" - I solved that by switching to .tar.gz, and just point you to a regression in your release cycle.

So far, lbzip2 version 2.5 (year 2014) is the only one who unpacked without error messages, official bzip2 (even year 2019) cannot.

[vobloeb]

This is diagnosable. The bz2 format has two CRC layers:

1. Per-block CRC32 — each ~900KB block has its own CRC over the block's decompressed data. Stored in the block header.
2. Combined CRC at end of stream — an XOR-rolling (with rotate) aggregate of all per-block CRCs. Written into the trailer after the 0x177245385090 magic.

Reference bzip2 checks both. lbzip2 — being parallel — verifies per-block CRCs in worker threads, and treats the combined-CRC either differently (XOR-aggregating in arbitrary order) or more leniently. There's a known class of bugs in parallel bz2 compressors (pbzip2 / lbzip2 / historical mpibzip2): when streaming blocks and aggregating the
combined-CRC in parallel, in rare cases they produce a value that doesn't match what a sequential reference decoder computes. Per-block CRCs stay correct.

Then:

• reference bunzip2 reaches end-of-stream, verifies combined-CRC → mismatch → data integrity error when decompressing — CRC error (even though every block decompressed cleanly).
• lbzip2 either computes combined-CRC as a commutative XOR-fold (no rotate) and matches, or just doesn't fail-hard — all blocks succeeded.

Plus a second common source of the same picture: multi-stream bz2 (several BZh…footer concatenated). lbzip2 finds the next BZh marker and continues, tolerant of bit-padding between streams. Reference bzip2, if the last stream is short / footer isn't where expected / there are stray bits — yells "CRC error".

So the fact that lbzip2 decompresses but reference bzip2 doesn't actually confirms "the archive is slightly broken from birth" — i.e., packed with a parallel tool that wrote combined-CRC / stream layout in a non-canonical way. SHA256 matches for everyone because it's that same slightly-broken file — that's how upstream published it. It's a bug
in boost's release toolchain, not in-flight corruption.

[sdarwin]

You are right: lbzip2.

At the same time, I am testing multiple environments and they are working:

• 7z on Ubuntu 24.04.
• bunzip2 on Ubuntu 24.04

Next, MacOS.

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H1715

$ bunzip2 --version
bzip2, a block-sorting file compressor.  Version 1.0.6, 6-Sept-2010.

• bunzip2 from 2010 on macos x86_64 OK
• bunzip2 from 2026 on macos ARM processor OK

What is the relevant factor then? What can explain the difference? Of course it must be the platform.

"as relevant as for example timezone". "a L1 support question". That is quite disingenuous. It is the most relevant factor. The executable itself (bzip2 Version 1.0.6, 6-Sept-2010), literally the same version of bzip, is able to succeed.