Skip to content

Conversation

@inolen
Copy link
Collaborator

@inolen inolen commented Dec 18, 2025

I ran into this when writing a bunch of files from a tar archive to the file system, where each file was a subarray of the main buffer.

@inolen inolen force-pushed the nodefs_byte_offset branch 3 times, most recently from e592e31 to 3ff825e Compare December 18, 2025 15:17
@inolen
Copy link
Collaborator Author

inolen commented Dec 18, 2025

The change started out by adding buffer.byteOffset to offset (hence the description) but then ended by just getting rid of the additional array view. Digging through the history, it seems that may have originated from the original code, where it seems the node.js fs APIs didn't accept a TypedArray - just a node Buffer instance.

const buf = Uint8Array.from('c=3\nd=4\ne=5', x => x.charCodeAt(0));
FS.writeFile("testfile", "a=1\nb=2\n");
FS.writeFile("testfile", new Uint8Array([99, 61, 51]) /* c=3 */, { flags: "a" });
FS.writeFile("testfile", buf.subarray(4, 7), { flags: "a" });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe leave the comment here but change it do /* d=4 */?

const buf = Uint8Array.from('c=3\nd=4\ne=5', x => x.charCodeAt(0));
FS.writeFile("testfile", "a=1\nb=2\n");
FS.writeFile("testfile", new Uint8Array([99, 61, 51]) /* c=3 */, { flags: "a" });
FS.writeFile("testfile", buf.subarray(4, 7), { flags: "a" });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this new test code not work with the old nodefs code? (i.e. could you land the test change first without it breaking?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test fails on both nodefs / noderawfs.

This change started out by adding buffer.byteOffset to offset (hence the
description) but ended by just getting rid of the additional array view.
Digging through the history, it seems that may have originated from the
original code, when the node.js fs APIs didn't accept a TypedArray -
just a node Buffer instance
@inolen inolen force-pushed the nodefs_byte_offset branch from 3ff825e to 0708a96 Compare December 18, 2025 20:02
@inolen inolen merged commit c77be74 into emscripten-core:main Dec 18, 2025
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants