Bound tar entry preallocation to available archive data to prevent disk-exhaustion - #131794
Bound tar entry preallocation to available archive data to prevent disk-exhaustion#131794alinpahontu2912 with Copilot wants to merge 2 commits into
Conversation
…chive data Co-authored-by: alinpahontu2912 <56953855+alinpahontu2912@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR changes TarEntry extraction to avoid using the tar header’s declared entry size directly for FileStreamOptions.PreallocationSize when extracting regular files, by attempting to cap preallocation to the actual amount of archive data available (via a new SubReadStream helper). It also adds regression tests for mismatched size fields during ExtractToFile.
Changes:
- Add
SubReadStream.AvailableLengthInSuperStreamto expose how many bytes remain in the underlying seekable stream from the substream’s window start. - Use a new
TarEntry.GetPreallocationSize()helper to compute preallocation size instead of blindly usingTarEntry.Length. - Add tests covering extraction when the tar header “size” is much larger/smaller than the actual archive content.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/libraries/Common/src/System/IO/SubReadStream.cs | Adds an internal helper to estimate remaining bytes in the underlying (seekable) stream. |
| src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.cs | Switches file extraction preallocation to a helper that can cap to available archive bytes. |
| src/libraries/System.Formats.Tar/tests/TarEntry/TarEntry.ExtractToFile.Tests.cs | Adds regression coverage for mismatched header size vs actual archive content. |
| using var extractStream = new MemoryStream(archive); | ||
| using var extractReader = new TarReader(extractStream); | ||
| TarEntry entryToExtract = extractReader.GetNextEntry(copyData: false); | ||
| Assert.NotNull(entryToExtract); | ||
|
|
||
| // Should complete without hanging or exhausting disk trying to preallocate 500 MB | ||
| // for a few bytes of real data; the resulting file is truncated to the real data available. | ||
| entryToExtract.ExtractToFile(destination, overwrite: false); | ||
|
|
||
| long extractedSize = new FileInfo(destination).Length; | ||
| Assert.True(extractedSize < HugeDeclaredSize); | ||
| } |
| private long GetPreallocationSize() | ||
| { | ||
| long length = Length; | ||
|
|
||
| if (length > 0 && _header._dataStream is SubReadStream subReadStream) | ||
| { | ||
| long? availableLength = subReadStream.AvailableLengthInSuperStream; | ||
| if (availableLength.HasValue && availableLength.Value < length) | ||
| { | ||
| return availableLength.Value; | ||
| } | ||
| } | ||
|
|
||
| return length; | ||
| } |
|
Tagging subscribers to this area: @dotnet/area-system-formats-tar |
Co-authored-by: alinpahontu2912 <56953855+alinpahontu2912@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (2)
src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.cs:790
- GetPreallocationSize can return a negative value when the SubReadStream has been advanced/seeks past the declared Length (SubReadStream.Position setter/Seek don’t enforce an upper bound). A negative PreallocationSize will throw via FileStreamHelpers.ValidateArguments (preallocationSize < 0), breaking ExtractToFile. Also, AvailableLengthInSuperStream is measured from the window start, so when comparing against the remaining declared bytes, it should be reduced by the current SubReadStream.Position to avoid over-preallocating relative to the current read position.
if (length > 0 && _header._dataStream is SubReadStream subReadStream)
{
long remainingDeclared = length - subReadStream.Position;
long? availableLength = subReadStream.AvailableLengthInSuperStream;
if (!availableLength.HasValue)
{
// Unseekable archive stream: the declared size cannot be validated against the
// actual remaining data, so don't preallocate based on it at all.
return 0;
}
return Math.Min(remainingDeclared, availableLength.Value);
}
src/libraries/System.Formats.Tar/tests/TarEntry/TarEntry.ExtractToFile.Tests.cs:61
- These assertions use FileInfo.Length as a proxy for whether preallocation happened. However, FileStreamOptions.PreallocationSize typically affects allocated blocks (e.g., posix_fallocate / Windows allocation size) without changing the file’s logical length. That means this test can pass even if extraction still preallocates the huge declared size (and could still exhaust disk), making the regression coverage unreliable and potentially environment-dependent (disk capacity). Consider asserting on allocated size (similar to System.Runtime/tests/System.IO.FileSystem.Tests/FileStream/ctor_options.* helpers) or otherwise validating the computed preallocation size directly.
long extractedSize = new FileInfo(destination).Length;
Assert.True(extractedSize < HugeDeclaredSize);
}
| // Preallocating to the expanded size would reserve disk space that bears no | ||
| // relation to the archive contents and can fail surprisingly on small volumes. | ||
| PreallocationSize = _header._gnuSparseDataStream is null ? Length : 0, | ||
| PreallocationSize = _header._gnuSparseDataStream is null ? GetPreallocationSize() : 0, |
There was a problem hiding this comment.
You can put the _gnuSparseDataStream check also into GetPreallocationSize so that all logic is in one place
| // Unseekable archive stream: the declared size cannot be validated against the | ||
| // actual remaining data, so don't preallocate based on it at all. | ||
| return 0; |
There was a problem hiding this comment.
This might regress perf because it disables preallocation in cases where it was allowed before.
The application already authorized writing Length amount of data to the disk by calling the ExtractToFile or ExtractToDirectory method, so verifying that there is enough data in the archive does not introduce any meaningful protection.
Extracting a non-sparse regular file entry set
FileStreamOptions.PreallocationSizeto the entry'sLength, sourced directly from the tar header's attacker-controlledsizefield, with no check against real data present in the archive. A crafted archive declaring a huge size but containing little/no actual data could trigger an arbitrarily large preallocation, causing disk exhaustion.Changes
SubReadStream(src/libraries/Common/src/System/IO/SubReadStream.cs): addedAvailableLengthInSuperStream, reporting bytes actually remaining in the underlying seekable stream from the entry's data window start, ornullwhen the stream isn't seekable.TarEntry(src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.cs): preallocation size is now computed via a newGetPreallocationSize()helper, which caps the declaredLengthto the real available data when smaller. Falls back to prior behavior for unseekable streams, user-suppliedDataStream, or when the archive genuinely contains the declared amount of data. GNU sparse entries remain excluded from preallocation, unchanged.TarEntry.ExtractToFile.Tests.cs): crafted tar entries with headersizefields mismatched against actual archive content, verifying extraction is bounded by real data rather than the declared size.