Skip to content

feat: support standard YAML frontmatter via pre-AST replacement #700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
avivkeller merged 3 commits into from
Mar 26, 2026
+43 −6
Merged

feat: support standard YAML frontmatter via pre-AST replacement #700

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e86af8c
feat(metadata): add support for standard YAML frontmatter blocks
moshams272 Mar 24, 2026
e2a2bd3
fix: resolve merge conflict in ast generator
moshams272 Mar 24, 2026
4a996f7
Merge branch 'main' into feat/standard-yaml-support
moshams272 Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions src/generators/ast/generate.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,15 @@ export async function processChunk(inputSlice, itemIndices) {

for (const [path, parent] of filePaths) {
const content = await readFile(path, 'utf-8');
const value = content.replace(
QUERIES.stabilityIndexPrefix,
match => `[${match}](${STABILITY_INDEX_URL})`
);
const value = content
.replace(
QUERIES.standardYamlFrontmatter,
(_, yaml) => '<!-- YAML\n' + yaml + '\n-->\n'
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The replacement for frontmatter appends an unconditional trailing newline after -->. If the original frontmatter is followed by a newline (the common case), this produces an extra blank line at the top of the document and shifts subsequent node position line numbers, which can affect diagnostics/logging that rely on those positions. Consider preserving the original spacing by not adding an extra newline (or by consuming/re-emitting the original post-delimiter newline in the regex/replacement).

Suggested change
(_, yaml) => '<!-- YAML\n' + yaml + '\n-->\n'
(_, yaml) => '<!-- YAML\n' + yaml + '\n-->'

Copilot uses AI. Check for mistakes.
)
.replace(
QUERIES.stabilityIndexPrefix,
match => `[${match}](${STABILITY_INDEX_URL})`
);

const relativePath = sep + withExt(relative(parent, path));

Expand Down
34 changes: 32 additions & 2 deletions src/utils/queries/__tests__/index.test.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,39 @@
import { strictEqual } from 'node:assert';
import { strictEqual, ok } from 'node:assert';
import { describe, it } from 'node:test';

import { u as createTree } from 'unist-builder';

import { UNIST } from '../index.mjs';
import { QUERIES, UNIST } from '../index.mjs';

describe('QUERIES', () => {
describe('standardYamlFrontmatter', () => {
it('matches standard YAML frontmatter at the beginning of the text', () => {
const content = '---\nintroduced_in: v1.0.0\ntype: module\n---';
ok(QUERIES.standardYamlFrontmatter.test(content));

const match = QUERIES.standardYamlFrontmatter.exec(content);
strictEqual(match[1], 'introduced_in: v1.0.0\ntype: module');
});

it('matches standard YAML frontmatter with Windows line endings (CRLF)', () => {
const content = '---\r\nintroduced_in: v1.0.0\r\n---';
ok(QUERIES.standardYamlFrontmatter.test(content));

const match = QUERIES.standardYamlFrontmatter.exec(content);
strictEqual(match[1], 'introduced_in: v1.0.0');
});

it('does not match horizontal rules or dashes not at the start of the string', () => {
const content = '# Hello\n\n---\n\nSome text';
strictEqual(QUERIES.standardYamlFrontmatter.test(content), false);
});

it('does not match unclosed frontmatter blocks', () => {
const content = '---\nintroduced_in: v1.0.0\nSome text...';
strictEqual(QUERIES.standardYamlFrontmatter.test(content), false);
});
});
});

describe('UNIST', () => {
describe('isStronglyTypedList', () => {
Expand Down
2 changes: 2 additions & 0 deletions src/utils/queries/index.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ export const QUERIES = {
stabilityIndexPrefix: /Stability: ([0-5](?:\.[0-3])?)/,
// ReGeX for retrieving the inner content from a YAML block
yamlInnerContent: /^<!--[ ]?(?:YAML([\s\S]*?)|([ \S]*?))?[ ]?-->/,
// ReGeX for standard Markdown YAML frontmatter
standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---/,
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

standardYamlFrontmatter currently matches the closing delimiter as just \n--- without requiring it to be the whole line (newline/EOF immediately after). That means inputs like a closing line of ---- (or --- followed by other characters) will still match and be treated as frontmatter, which is not valid frontmatter syntax and can lead to incorrect metadata extraction. Consider tightening the pattern so the closing delimiter must be exactly --- followed by \r?\n or end-of-string (and update the related tests accordingly).

Suggested change
standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---/,
standardYamlFrontmatter: /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/,

Copilot uses AI. Check for mistakes.
// ReGeX for finding references to Unix manuals
unixManualPage: /\b([a-z.]+)\((\d)([a-z]?)\)/g,
// ReGeX for determing a typed list's non-property names
Expand Down
Loading