fix(docx): preserve manual line breaks - #186
Conversation
138e3b0 to
51f5909
Compare
mldangelo-oai
changed the title
|
Related DOCX parser change: #123 addresses malformed numeric XML references that can abort the knowledge-base load (issue #40), while this PR preserves manual line breaks. Both touch the same parser and tests but solve different defects, so they should be coordinated or rebased together rather than labeled duplicate. |
3fe6cf9 to
c7d6b92
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: effea4accb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return decodeXml( | ||
| xml | ||
| .replace(/<\/(?:\w+:)?p\s*>/gu, "\n") | ||
| .replace(/<(?:\w+:)?br\b[^>]*>/gu, "\n") |
There was a problem hiding this comment.
When a DOCX represents a manual text-wrapping break as the valid WordprocessingML element <w:cr/>, this replacement does not recognize it, and the later generic tag removal still turns <w:t>foo</w:t><w:cr/><w:t>bar</w:t> into foobar. Handle w:cr alongside w:br so these documents do not continue concatenating text across manual line breaks.
Useful? React with 👍 / 👎.
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
2 participants
Summary
<w:br/>Root cause
The DOCX extractor handled paragraph endings and tabs before stripping XML tags, but it did not handle WordprocessingML break elements. The generic tag removal therefore deleted
<w:br/>without inserting whitespace and concatenated text from either side.Impact
Prepared DOCX knowledge bases retain word boundaries and line structure around manual breaks, improving the text supplied to scans.
Overlap check
Searched open issues and pull requests for DOCX line breaks, manual knowledge-base breaks, and
w:br; no overlapping work was found.Validation
pnpm dlx bun test --timeout 30000 ./tests-ts/knowledge-base.test.ts(5 passed, 2 platform skips)pnpm exec tsc --noEmitpnpm exec prettier --check src/knowledge-base.ts tests-ts/knowledge-base.test.tsgit diff --check