Skip to content

Accept header-end and field-end lines with no space after the '#' - #7

Merged
AAAlvesJr merged 1 commit into
AAAlvesJr:mainfrom
smrgeoinfo:spec-conformant-separators
Aug 1, 2026
Merged

Accept header-end and field-end lines with no space after the '#'#7
AAAlvesJr merged 1 commit into
AAAlvesJr:mainfrom
smrgeoinfo:spec-conformant-separators

Conversation

@smrgeoinfo

Copy link
Copy Markdown
Contributor

The problem

The specification defines both separator lines as

Header-end line: comment token + header-end token + end-of-line token

where the header-end token is "three or more dash characters" and the field-end token is "three or more forward slash characters". No whitespace token sits between the comment token and the separator token.

The validator requires one:

regex_header_end = r"^#\s+---(-*)$"
regex_fields_end = r"^#\s+///(/*)$"

so a conforming file written #----- is rejected outright with Header-end line ('# ---') not found. Analysis can't proceed.

Why it matters in practice

258 of the 272 files in the XAS Data Library write the header-end line as #----- with no space. That is the reference collection published by the XDI working group. Every one of those files failed to parse.

Measured over the whole collection:

before after
header-end failures 272 0

The remaining validation errors in that collection are genuine data non-conformance — non-ISO Scan.start_time, Sample.temperature written as room temperature rather than float + units — and are untouched by this change.

What changed

regex_header_end = r"^#\s*-{3,}$"
regex_fields_end = r"^#\s*/{3,}$"

The spec's illustrative example does show a space, and that form keeps working — the change only stops requiring it. Fewer than three dashes, or a leading ##, are still rejected.

The field-end separator has the identical defect. No file I encountered writes it without a space, so nothing observed is fixed by that half; it is included because it is the same bug in the same grammar rule, and fixing one and not the other would be arbitrary. Happy to drop it if you would rather keep the diff to what is demonstrably broken.

Tests

Adds tests/separators_without_space.xdivalid.xdi with both separators unspaced — and asserts both the spaced and unspaced forms validate clean.

tests/test_WrongFields.py::test_invalid_column fails both before and after this change on unmodified main: it expects the error key column['1'] where the validator produces column.1. Left alone as unrelated.

🤖 Generated with Claude Code

The specification defines both separators as

    comment token + separator token + end-of-line token

where the separator token is "three or more" dash or slash characters.
No whitespace token sits between the comment token and the separator
token. The validator required one:

    regex_header_end = r"^#\s+---(-*)$"
    regex_fields_end = r"^#\s+///(/*)$"

so a conforming file written "#-----" was rejected outright, with
"Header-end line ('# ---') not found. Analysis can't proceed."

This is not a hypothetical reading of the grammar. 258 of the 272 files
in the XAS Data Library -- the reference collection published by the XDI
working group itself -- write the header-end line as "#-----" with no
space. Every one of them failed to parse. Running the validator over
that collection now yields zero header-end failures where before it
yielded 272.

The spec's illustrative example does show a space, and that form keeps
working; the change only stops requiring it. Fewer than three dashes,
or a leading "##", are still rejected.

The field-end separator has the identical defect. No file encountered
writes it without a space, so nothing observed is fixed by that half of
the change -- it is included because it is the same bug in the same
grammar rule, and leaving one of the two would be arbitrary.

Adds tests/separators_without_space.xdi, which is valid.xdi with both
separators unspaced, and asserts both forms validate clean.

Note: tests/test_WrongFields.py::test_invalid_column fails both before
and after this change, on unmodified main. It expects the error key
"column['1']" where the validator produces "column.1". Left alone as
unrelated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@AAAlvesJr

Copy link
Copy Markdown
Owner

Thank you. We can not reproduce the error you mentioned related to test tests/test_WrongFields.py::test_invalid_column either on the CI or in our local stack. I will investigate this further.
In the meanwhile let me accept your Pr.

@AAAlvesJr
AAAlvesJr merged commit 244e2e9 into AAAlvesJr:main Aug 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants