feat: support standard YAML frontmatter via pre-AST replacement#700
feat: support standard YAML frontmatter via pre-AST replacement#700avivkeller merged 3 commits intonodejs:mainfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #700 +/- ##
==========================================
+ Coverage 75.40% 75.47% +0.06%
==========================================
Files 148 148
Lines 13586 13623 +37
Branches 1025 1031 +6
==========================================
+ Hits 10245 10282 +37
Misses 3336 3336
Partials 5 5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
moshams272
force-pushed
the
feat/standard-yaml-support
branch
from
cb6b9ed to
e86af8c
Compare
moshams272
changed the title
There was a problem hiding this comment.
Pull request overview
Adds support for standard Markdown --- YAML frontmatter by converting it into the tool’s existing legacy <!-- YAML ... --> format before generating the Markdown AST, keeping downstream metadata extraction unchanged.
Changes:
- Add a
standardYamlFrontmatterregex to detect top-of-file frontmatter blocks. - Add unit tests covering matching/non-matching cases and CRLF handling.
- Apply a pre-parse string replacement in the AST generator to transpile
---frontmatter into<!-- YAML ... -->.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/utils/queries/index.mjs | Introduces a new regex query for standard YAML frontmatter detection. |
| src/utils/queries/tests/index.test.mjs | Adds tests validating the new frontmatter regex behavior. |
| src/generators/ast/generate.mjs | Performs pre-AST replacement to convert standard frontmatter into legacy YAML comment blocks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const value = content | ||
| .replace( | ||
| QUERIES.standardYamlFrontmatter, | ||
| (_, yaml) => '<!-- YAML\n' + yaml + '\n-->\n' |
There was a problem hiding this comment.
The replacement for frontmatter appends an unconditional trailing newline after -->. If the original frontmatter is followed by a newline (the common case), this produces an extra blank line at the top of the document and shifts subsequent node position line numbers, which can affect diagnostics/logging that rely on those positions. Consider preserving the original spacing by not adding an extra newline (or by consuming/re-emitting the original post-delimiter newline in the regex/replacement).
| (_, yaml) => '<!-- YAML\n' + yaml + '\n-->\n' | |
| (_, yaml) => '<!-- YAML\n' + yaml + '\n-->' |
| // ReGeX for retrieving the inner content from a YAML block | ||
| yamlInnerContent: /^<!--[ ]?(?:YAML([\s\S]*?)|([ \S]*?))?[ ]?-->/, | ||
| // ReGeX for standard Markdown YAML frontmatter | ||
| standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---/, |
There was a problem hiding this comment.
standardYamlFrontmatter currently matches the closing delimiter as just \n--- without requiring it to be the whole line (newline/EOF immediately after). That means inputs like a closing line of ---- (or --- followed by other characters) will still match and be treated as frontmatter, which is not valid frontmatter syntax and can lead to incorrect metadata extraction. Consider tightening the pattern so the closing delimiter must be exactly --- followed by \r?\n or end-of-string (and update the related tests accordingly).
| standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---/, | |
| standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/, |
Description
Currently,
doc-kit's YAML metadata parser strictly expects legacy HTML comment blocks.This PR introduces native support for standard
---YAML frontmatter blocks while preserving 100% backward compatibility for existing Node.js documentation.Instead of adding heavy external dependencies like remark-frontmatter, I implemented a lightweight Pre-AST replacement strategy in src/generators/ast/generate.mjs.
It intercepts standard --- blocks at the beginning of files and transpiles them into the legacy `` format before the AST is generated.
This ensures that all existing downstream logic (metadata extraction, search indexing, etc.) works out-of-the-box without needing any modifications to the core AST traversal.
Developer Experience (DX) & Future-proofing:
As doc-kit is adopted by other ecosystems (like Webpack), developers naturally expect standard --- support. This PR future-proofs the tool for modern Markdown generators without bloating the dependency tree.
Validation
Added dedicated unit tests for the new
standardYamlFrontmatterregex insrc/utils/queries/__tests__/index.test.mjs.Verified that horizontal rules (---) in the middle of documents remain unaffected (via regex anchors).
Related Issues
This is a foundational fix for supporting modern markdown AST generators that rely on standard frontmatter injection.
Check List
node --run testand all tests passed.node --run format&node --run lint.