Incremental Update Implementation (cleaner) by adnsistemas · Pull Request #111 · cantoo-scribe/pdf-lib

adnsistemas · 2025-06-03T21:27:45Z

What?

Ability to make incremental updates to PDFs

Why?

Required for proper electronic signing of documents.

How?

Making an snapshot of the original PDF and marking changed Refs that need to be saved in the "increment"

Testing?

All tests and Node app.

New Dependencies?

No

Screenshots

N/A

Anything Else?

Had to make a couple of small changes to package.json in order to be able to build, test, and commit the changes.

Checklist

…-scribe-master

SamiSaves · 2025-07-01T07:37:33Z

@Sharcoux Hey, is this PR in a clean enough state to be merged or worked on (the previous pr was #110) ? I am also interested in this as we need it for PAdes long term validation support.

I could probably try and help with this to make it merge ready

benjinus · 2025-07-01T13:30:07Z

I think this is a very important feature update, everyone needs this👏

Sharcoux · 2025-07-01T14:45:46Z

Ok, I'll try to have a look. It's quite complex. If one of you want to have a second look at the change, that would be great.

SamiSaves · 2025-07-02T04:33:00Z

I'll see if I have time to check the pr later this week and I also should be able to test it out and validate that it works for our scenario too 👍

SamiSaves · 2025-07-04T06:07:32Z

Here is some reading related to this PR:

PDF spec 7.5.6 Incremental Updates (pdf page 52)
https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/PDF32000_2008.pdf#page=52

ETSI has a document "Part 1: Building blocks and PAdES baseline signatures" in which part 5.4 mentions incremental updates
https://www.etsi.org/deliver/etsi_en/319100_319199/31914201/01.02.01_60/en_31914201v010201p.pdf#page=10

SamiSaves · 2025-07-07T12:49:13Z

I was able to create an incrementally updated pdf with the branch:

async function addIncrementalUpdate(signedPdfBuffer: Buffer, update: string): Promise<Buffer> {
  // Load for incremental update
  const pdfDoc = await PDFDocument.load(signedPdfBuffer, {    forIncrementalUpdate: true  })

  // Add the update string as a text annotation to the first page to demonstrate any update
  const pages = pdfDoc.getPages()
  if (pages.length > 0) {
    pages[0].drawText(update, { x: 50, y: 700, size: 12 })
  }

  // Save incrementally (creates a new revision at the end)
  const newPdf = await pdfDoc.save({ useObjectStreams: false })
  return Buffer.from(newPdf)
}

I checked that increment didn't modify any of the old pdf bytes with this test:

const testPdf = await readFile(path.join(__dirname, './test.pdf'))
const incrementedPdf = addIncrementalUpdate(testPdf, 'testing incremental update')
assert(incrementalPdf.subarray(0, testPdf.length).equals(testPdf))

And I also observed from the pdf bytes that the new increment contains a new catalog, trailer and EOF.

adnsistemas · 2025-07-07T23:16:06Z

I added some tests in the PR, unit and app tests. One tests that the original PDF is unchanged.
There is not one that checks the increment, maybe is worth adding it.

SamiSaves

I went through the PR and didn't find much to say as I am not familiar with the codebase.

To me it looks good, it doesn't seem to break existing functionality and this is confirmed by existing tests. It has test coverage for the new feature as well.

I think the only thing missing here is some documentation what the incremental update is, what does it do and how to use it.

SamiSaves · 2025-07-03T12:02:50Z

package.json

-      "prettier --check || (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)",
-      "eslint --fix || (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)"


Probably should revert these changes. I wonder what here caused issues 🤔

I had to make changes in order to be able to run the tests. Don't know why, but in my environment the original failed with an error. Should be "reverted" to merge, but I can't do it in my branch.

SamiSaves · 2025-07-08T05:20:19Z

apps/deno/tests/test20.ts

+  let tw = font.widthOfTextAtSize('Electronic Signature Test #20', tamLetra);
+  let lw = fontBold.widthOfTextAtSize(usarLibreOffice, tamAdvertencia);
+  if (lw > tw) tw = lw;


I'd personally prefer more descriptive names for variables. As I am new to this project and the world of pdfs, it takes quite a bit of effort to understand what these are for.

I agree. And the code seems unnecessarily hard to read. What about:

const textWidth = Math.max( font.widthOfTextAtSize('Electronic Signature Test #20', tamLetra), fontBold.widthOfTextAtSize(usarLibreOffice, tamAdvertencia) ) const textHeight = Math.max( font.heightAtSize(tamLetra), fontBold.heightAtSize(tamAdvertencia) )

SamiSaves · 2025-07-08T06:13:35Z

tests/api/PDFDocument.spec.ts

+        expect(pdfIncrementalBytes.byteLength).toBeGreaterThan(0);
+        expect(finalPdfBytes.byteLength).toBeLessThanOrEqual(
+          pdfSaveBytes.byteLength,
+        );


If we are expecting same output in both cases, would it make sense to compare that the actual bytes are equal?

Well, then do it, no?

Sharcoux · 2025-07-08T07:37:57Z

package.json

  "description": "Create and modify PDF files with JavaScript",
  "author": "Andrew Dillon <andrew.dillon.j@gmail.com>",
-  "packageManager": "yarn@1.22.19+",
+  "packageManager": "yarn@1.22.19",


Why this change?

Original don't run in my environment. Should be equivalent, but in my case npm complains about it.

Cantoo scribe master has

"packageManager": "yarn@1.22.19+",

This pr has

"packageManager": "yarn@1.22.19",

And your forks master has:

"packageManager": "yarn@1.22.22+sha512.a6b2f7906b721bba3d67d4aff083df04dad64c399707841b7acf00f6b133b7ac24255f2652fa22ae3534329dc6180534e98d17432037ff6fd140556e2bb3137e",

Have you tried with the cantoo master version? Perhaps the issue was the yarn with the full sha?

Yes, I try a lot of combinations that didn't work. But I'm not keen at all in configs.. so probably is something simple that I don't know about. Config and package changes shouldn't be necessaries, but I don't know how to exclude them from the PR.

Could you do somthing like this:

Revert the changes (no commiting yet)

Temporarily remove this from package.json

"husky": { "hooks": { "pre-commit": "lint-staged" } },

Stage and commit the reverted code (excluding the husky change above)

I think is done..
Also added a fix in largestObjectNumber determination, when XREF is a Stream. For that I had to fix some PDFparser tests expectations, a couple of which I'm not totally sure about, but it seems that all the expectations in those tests are "taken" from the test result, and not from the PDF actual content, so I'm guessing is "all right".

Sharcoux · 2025-07-08T07:38:04Z

package.json

    "{src,tests,apps}/**/*.{ts,js,json,html,css}": [
-      "prettier --check || (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)",
-      "eslint --fix || (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)"
+      "prettier --check",


Why those changes?

Can't run it without the change.

For me, in your forks branch cantoo-scribe-master eslint gets stuck trying to lint /dist/pdf-lib.esm.min.js. So it seems to be trying to scan files should be ignored.

It seems to be due to this change: adnsistemas@417a8a6#diff-a32a0887ed9d1d707bbb3b845b7df7fd40e673c47e7b60a3ebd896b68d3b8839R10

Could this be the reason why it didn't work earlier with these changes?

It didn't got stuck for me, it always failed with the message after the ||
I have the same problem with both projects (the one I forked from, and this). I guess is the versions of node/npm/Linux I'm running in, but I really don't know (and to be honest I don't care much about it so I didn't dig into the actual problem).

What happens when you run yarn lint?

In @cantoo/pdf-lib master branch (node 20.17.0, npm 10.8.2):
Invalid package manager specification in package.json (yarn@1.22.19+); expected a semver version
Then, changing that line to yarn@1.22.19:
yarn run v1.22.19 $ yarn lint:prettier && yarn lint:eslint $ prettier --write "./{src,tests,apps}/**/*.{ts,js,json,html,css}" --log-level error $ eslint .
And there it hungs, after running only yarn lint:eslint --verbose, and then again yarn lint it works.
Then I changed a file and staged it. Running yarn lint-staged gives:
`yarn run v1.22.19
$ lint-staged
✔ Backed up original state in git stash (d85c0fa)
⚠ Running tasks for staged files...
❯ package.json — 1 file
❯ {src,tests,apps}/**/*.{ts,js,json,html,css} — 1 file
✖ prettier --check || (echo '
⚠️ Linting issues detected. Please run "yarn lint" to fix them and stage the changes.' && exit 1) [FAILED]
◼ eslint --fix || (echo '
⚠️ Linting issues detected. Please run "yarn lint" to fix them and stage the changes.' && exit 1)
↓ Skipped because of errors from tasks.
✔ Reverting to original state because of errors...
✔ Cleaning up temporary files...

✖ prettier --check || (echo '
⚠️ Linting issues detected. Please run "yarn lint" to fix them and stage the changes.' && exit 1):
[error] No files matching the pattern were found: "||".
[error] No files matching the pattern were found: "(echo".
[error] No files matching the pattern were found: "
[error] ⚠️ Linting issues detected. Please run "yarn lint" to fix them and stage the changes.".
[error] No files matching the pattern were found: "&&".
[error] No files matching the pattern were found: "exit".
[error] No files matching the pattern were found: "1)".
Checking formatting...
All matched files use Prettier code style!
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.`

But yarn lint finds no problems.

Sharcoux · 2025-07-08T07:42:52Z

src/core/objects/PDFObject.ts

 import PDFContext from '../PDFContext';

 class PDFObject {
+  registerChange() {


Throwing an error will probably break something PDFPageLeaf, will it not? Shouldn't we just do nothing?

Every PDFObject should 'registerChange' (do noting is an option for specific descendants).
Otherwise you can't guarantee the increment will be correct.
If you just "forgot" to implement it, and the base implementation does nothing, will be difficult to trace why you have missing elements in the increment.

Sharcoux · 2025-07-08T07:45:42Z

rollup.config.mjs

  'Circular dependency: es/api/PDFPage.js -> es/api/PDFDocument.js -> es/api/form/PDFForm.js -> es/api/form/PDFOptionList.js -> es/api/PDFPage.js',
  'Circular dependency: es/api/PDFPage.js -> es/api/PDFDocument.js -> es/api/form/PDFForm.js -> es/api/form/PDFRadioGroup.js -> es/api/PDFPage.js',
  'Circular dependency: es/api/PDFPage.js -> es/api/PDFDocument.js -> es/api/form/PDFForm.js -> es/api/form/PDFTextField.js -> es/api/PDFPage.js',
+  'Circular dependency: es/core/index.js -> es/core/writers/PDFWriter.js -> es/api/snapshot/index.js -> es/api/snapshot/IncrementalDocumentSnapshot.js -> es/core/index.js',


IncrementalDocumentSnapshot should not depend on core/index.js.

Sharcoux · 2025-07-08T07:45:47Z

rollup.config.mjs

  'Circular dependency: es\\api\\PDFPage.js -> es\\api\\PDFDocument.js -> es\\api\\form\\PDFForm.js -> es\\api\\form\\PDFOptionList.js -> es\\api\\PDFPage.js',
  'Circular dependency: es\\api\\PDFPage.js -> es\\api\\PDFDocument.js -> es\\api\\form\\PDFForm.js -> es\\api\\form\\PDFRadioGroup.js -> es\\api\\PDFPage.js',
  'Circular dependency: es\\api\\PDFPage.js -> es\\api\\PDFDocument.js -> es\\api\\form\\PDFForm.js -> es\\api\\form\\PDFTextField.js -> es\\api\\PDFPage.js',
+  'Circular dependency: es\\core\\index.js -> es\\core\\writers\\PDFWriter.js -> es\\api\\snapshot\\index.js -> es\\api\\snapshot\\IncrementalDocumentSnapshot.js -> es\\core\\index.js',


Sharcoux · 2025-07-08T07:46:28Z

src/api/snapshot/DefaultDocumentSnapshot.ts

@@ -0,0 +1,29 @@
+import { PDFObject, PDFRef } from '../../core';


Do you want to import only the type? That might solve the cyclic dependency.

Sharcoux · 2025-07-08T07:46:34Z

src/api/snapshot/DocumentSnapshot.ts

@@ -0,0 +1,14 @@
+import { PDFObject, PDFRef } from '../../core';


Do you want to import only the type? That might solve the cyclic dependency.

Sharcoux · 2025-07-08T08:35:51Z

src/api/snapshot/IncrementalDocumentSnapshot.ts

@@ -0,0 +1,59 @@
+import { PDFContext, PDFObject, PDFRef } from '../../core';


Do you want to import only the type? That might solve the cyclic dependency.

Those imports are "inherited" from the project I took the incremental update base functionality. Not sure about the circular dependency issue, and solution.

I think this solution would work with DocumentSnapshot and DefaultDocumentSnapshot, in both all imports can be switched to import type I think. But in this file we have obj instanceof PDFRef ? obj : this.context.getObjectRef(obj) and that requires a normal import from core

Would it help if we import PDFRef straight from the file itself like PDFWriter? 🤔

import PDFRef from '../../core/objects/PDFRef';

Or maybe this.context.getObjectRef(obj) could handle the case of PDFRef and return the obj in that case?

Implemented most of the suggested changes and all seems to be working as before, no more circular dependency message, and no more '!' after getObjectRef.
Had to add a method to context, to be able to import only the type for PDFRef.
Is there a way to update this PR with the changes?

To answer my own question: Yes, by pushing to the branch the PR comes from.
Updated.

adnsistemas · 2025-07-08T17:04:35Z

I think the only thing missing here is some documentation what the incremental update is, what does it do and how to use it.
Didn't include it because makes the PR "messy". You can take the documentation from my project @adnsistemas/pdf-lib, the README.md has the documentation.

Sharcoux · 2025-07-10T15:11:15Z

src/api/snapshot/IncrementalDocumentSnapshot.ts

@@ -0,0 +1,59 @@
+import { PDFContext, PDFObject, PDFRef } from '../../core';


Or maybe this.context.getObjectRef(obj) could handle the case of PDFRef and return the obj in that case?

Sharcoux · 2025-07-10T15:17:21Z

src/api/snapshot/IncrementalDocumentSnapshot.ts

+  markObjsForSave(objs: PDFObject[]): void {
+    this.markRefsForSave(
+      objs.map((obj) =>
+        obj instanceof PDFRef ? obj : this.context.getObjectRef(obj)!,


It's probably not a good idea to assume this.context.getObjectRef(obj)!. It might be better to handle the case where it won't return what you expect. Does it make sense to accept filter the map? .filter((obj): result is PDFObject => !!obj)

I really don't like that '!' there, but as I didn't wrote that, I supposed it was just for TS not complaining. I like the filter a lot more, and makes more sense: if you can't get a Ref, then there is nothing to update in the increment.

…f deletions. Deletion of pages objects on page removal. Snapshots ref deletion registration

… of genNumber

…ify saving consistency

- Add commit() method to PDFDocument enabling chained incremental updates - Fix font duplication: change 'modified' flag to 'alreadyEmbedded' in PDFFont - Use PDFWriter instead of PDFStreamWriter for incremental saves - Add comprehensive tests for commit() functionality (17 test cases)

Previously, inline objects (arrays/dicts embedded directly in other objects, not registered as indirect objects) would not be tracked for incremental saves. This caused changes to inline arrays like MediaBox to be lost. The fix enhances PDFContext.registerObjectChange() to: 1. First check if the object is directly registered (indirect object) 2. If not, search through all indirect objects to find which one contains the inline object, and mark that parent for save Also fixed lint-staged config that had broken shell commands. Added test: 'tracks inline array modifications for incremental saves' All 673 tests pass.

…e and incremental generation. Parameter adding for incremental update saving, to force useObjectStreams preference

… fully saved (rewritted). Added test for this scenario.

Sharcoux · 2026-02-19T11:13:46Z

I have this: "This branch cannot be rebased due to conflicts". This is in GitHub. I could work around and manually apply the patch locally, but the change would lose paternity, I'd like to avoid if possible.

Maybe you can just update your branch to the top of master? That should be enough.

Ah! Maybe you didn't fetch the last version of master locally and that's why you don't see the conflict?

adnsistemas · 2026-02-19T17:13:36Z

I have this: "This branch cannot be rebased due to conflicts". This is in GitHub. I could work around and manually apply the patch locally, but the change would lose paternity, I'd like to avoid if possible.
Maybe you can just update your branch to the top of master? That should be enough.

Ah! Maybe you didn't fetch the last version of master locally and that's why you don't see the conflict?

Ok, I'll finish the rebase and update this branch. I'm not good at git, so no idea what that message means, as I can checkout cantoo-scribe/master and merge this branch onto it without conflicts.
I'll have to check the rebase result, so it will take me some time.

… into cantoo-scribe-master

adnsistemas · 2026-02-20T04:04:13Z

Ok, I'm not sure I did it properly. This is what I did:

Added this project as an additional remote origin with name 'cantoo'
Checked out master branch of this project as 'cantoo-master'
git checkout cantoo-scribe-master
git rebase cantoo-master
Fixed all the conflicts
Checked the resulting changes
Added the corrections requested and fixed a problem with XRef Streams skipping process
Tried to push to this pull-request branch and git complained that I needed to pull first
Pulled remote cantoo-scribe-master into local cantoo-scribe-master branch
Rechecked changes and rerun tests and apps
Pushed my local cantoo-scribe-master to this branch
Everything seems ok to me..

Hopding and others added 14 commits November 6, 2021 17:28

Bump version to 1.17.1

4beebbe

Merge branch 'master' of github.com:Hopding/pdf-lib

93dd36e

add incremental save functionality

c6cbda3

make snapshot mechanism more generic

c3ea300

Incremental Update Change Automatic Registration

05376db

Generación de paquete npm

7d61af0

Merge branch 'master' of github.com:cantoo-scribe/pdf-lib into cantoo…

0f2abc3

…-scribe-master

Tests conciliation

40dcac3

Fixed merge with cantoo-scribe

872fbdc

Readme and package changes

b4e7323

Lint minor fix

d264a49

ELint exclussions

4b22104

Correcciones al proceso de publicación del paquete

b783161

Adecuación con la rama master de cantoo-scribe

417a8a6

github-actions bot added the needs-triage label Jun 3, 2025

SamiSaves approved these changes Jul 8, 2025

View reviewed changes

Sharcoux reviewed Jul 8, 2025

View reviewed changes

Sharcoux reviewed Jul 10, 2025

View reviewed changes

adnsistemas added 4 commits July 10, 2025 17:00

Circular dependency correction. Changed ref registration improvement

ce366d7

Circular dependency fix. Changed Refs registration improvement

3aac5c5

Version bump

2e21e49

Object number detection correction

cce6de3

adnsistemas and others added 24 commits February 18, 2026 16:28

Added incremental update documentation

02a80f3

Readme and package changes

62a27c0

ELint exclussions

7554a37

Correcciones al proceso de publicación del paquete

51e0698

Fix for release-it

21f588b

Release 2.4.4

1694e00

Remove innecesary parameter to release-it

4bc64d4

Fix in inclusions paths, to fix package building

d64e3d1

Reimplementation of XREF objects registration for propper rearrange o…

2d5c405

…f deletions. Deletion of pages objects on page removal. Snapshots ref deletion registration

Incremental update tests improved. Linting changes.

53ed57f

Fix in new ref registration when incremental updating

0f173d4

Fix in StreamXref, in the generation and in the reading interpetation…

0c639f2

… of genNumber

Improvement in originalbytes preservation test

e430dd7

Fix in XREFStreams saving of document. Added some saving tests to ver…

33bfdd4

…ify saving consistency

Add signature preservation tests and PR documentation

a79abde

Fix in object streams usage on XREF generation, for incremental updat…

0017f18

…e and incremental generation. Parameter adding for incremental update saving, to force useObjectStreams preference

Minor optimizations

a1895cc

Fix in file version

c576b05

Updated documentation

a9ffc4a

Fix in XRef Stream output when PDF, opened for incremental update, is…

882429c

… fully saved (rewritted). Added test for this scenario.

Fixes based on master branch

27770f4

Examples improvements for code clarity

21ecc44

adnsistemas added 2 commits February 20, 2026 00:28

Fix in last XRef Stream skipping on save

7f91f76

Merge branch 'cantoo-scribe-master' of github.com:adnsistemas/pdf-lib…

eaf4b69

… into cantoo-scribe-master

Merge branch 'master' into cantoo-scribe-master

2aad561

		"prettier --check \|\| (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)",
		"eslint --fix \|\| (echo '\n⚠️ Linting issues detected. Please run \"yarn lint\" to fix them and stage the changes.' && exit 1)"

		@@ -0,0 +1,29 @@
		import { PDFObject, PDFRef } from '../../core';

		@@ -0,0 +1,14 @@
		import { PDFObject, PDFRef } from '../../core';

		@@ -0,0 +1,59 @@
		import { PDFContext, PDFObject, PDFRef } from '../../core';

Comments

Conversation

adnsistemas commented Jun 3, 2025

What?

Why?

How?

Testing?

New Dependencies?

Screenshots

Suggested Reading?

Anything Else?

Checklist

Uh oh!

SamiSaves commented Jul 1, 2025

Uh oh!

benjinus commented Jul 1, 2025

Uh oh!

Sharcoux commented Jul 1, 2025

Uh oh!

SamiSaves commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SamiSaves commented Jul 4, 2025

Uh oh!

SamiSaves commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adnsistemas commented Jul 7, 2025

Uh oh!

SamiSaves left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SamiSaves commented Jul 2, 2025 •

edited

Loading

SamiSaves commented Jul 7, 2025 •

edited

Loading

Sharcoux commented Feb 19, 2026 •

edited

Loading