feat(arxjit): reject methods and make the rejection matrix exhaustive - #102
Conversation
Wiki Issue 4 lists methods among the initial unsupported constructs, but nothing caught them: a method is an ordinary function in the extracted ast, and its first parameter is only conventionally named self, so an instance method, a staticmethod and a classmethod all validated cleanly. __qualname__ does record the owning class, so extraction now carries it and validation uses it. A function nested inside another function is dotted too, but Python inserts the "<locals>" marker there, which distinguishes the two cases. Replace the per-construct rejection tests with a table-driven matrix over _UNSUPPORTED_MESSAGES, plus a test asserting the matrix covers every entry in the table, so it cannot quietly fall behind the validator (wiki Issue 12, "tests for unsupported syntax diagnostics"). Each case asserts the construct reports its own message rather than one belonging to an enclosing node. That exhaustiveness check found three entries whose message was shadowed: Await is reported as a standalone expression statement unless it is the value of an assignment, Starred is reported as the enclosing container unless it appears in a range() call, and Slice has no reachable position at all, because a slice can only appear inside a Subscript and Subscript is rejected without descending into it. The first two are now exercised in the positions that reach them; Slice is named in _SHADOWED_MESSAGES and kept as a safety net rather than silently dropped. The matrix builds its sources as text rather than as real nested functions, so a lint autofix cannot rewrite the construct under test, as happened previously with unused in-function imports. 148 tests, arxjit coverage 100%. Verified on CPython 3.10, 3.11 and 3.14.
c4b581b to
84b1a5a
Compare
|
thanks for working on that @Jaskirat-s7 some notes from my side: |
…ions Review found that __qualname__ records where a function was defined, not whether a class currently owns the descriptor, so the method check claimed more than it can determine. Probing the boundary, three cases are missed: a decorator that returns a different function without preserving __qualname__ or setting __wrapped__, a module-level function assigned in a class body, and a function attached to a class after it was created. Rename _is_method to _defined_in_class_body so the code states what it actually tests, and document the three gaps on it, on the qualname field and on validate. A functools.wraps chain stays covered, because wraps copies __qualname__ and sets __wrapped__ for extraction to follow; that is now pinned by a test rather than left to chance. Pin the descriptor forms the description advertised but did not test: an instance method and a staticmethod resolve to plain functions when accessed through the class, while a classmethod resolves to a bound method object, so all four forms are parametrized rather than assumed equivalent. Pin the two out-of-contract cases as accepted, with the reason in the test docstring, so the contract is explicit and a future fix has a test to flip. Verified on 3.10/3.11/3.14 that __set_name__ is called for anything bound in a class body, including a decorator's return value, so the decorator layer can close both; it is not called for assignment after class creation, so that case stays undetectable and is documented as such. 153 tests, arxjit coverage 100%. Verified on CPython 3.10, 3.11 and 3.14.
|
Good catch, thanks @xmnlab , You're right that I conflated "defined in a class body" with "is a method" — I went the narrowing route, plus the regression test you suggested. The predicate is called The four that work are a parametrized test now; I kept the classmethod in there separately since it comes back as a bound method rather than a plain function. The three that don't are pinned as accepted with the reasoning in the docstring, so the contract is explicit and there's something concrete to flip when we fix it. I left the The 153 tests, still 100% coverage, checked on 3.10/3.11/3.14. Updated the description to match as well. |
xmnlab
left a comment
There was a problem hiding this comment.
Thanks for addressing the earlier feedback, @Jaskirat-s7.
I reviewed the latest changes and the contract is much clearer now. Renaming the predicate to "_defined_in_class_body", narrowing the documented behaviour, and adding coverage for instance methods, static methods, class methods, "functools.wraps", and the known undetected cases resolves my previous concerns.
I did not find any remaining blocking issues, and all CI workflows are green.
Two small non-blocking suggestions:
1. The diagnostic currently says:
"methods are not supported (only module-level functions)"
Since the implementation specifically detects functions lexically defined in class bodies, a more precise message could be:
"functions defined in class bodies are not supported"
2. The PR description mentions that attaching a function to a class after class creation is also pinned as an accepted limitation. I found tests for class-body assignment and a decorator that drops "__qualname__", but I did not see an explicit post-creation assignment test. It would be useful to add one, though it does not need to block this PR.
The exhaustive rejection matrix also looks solid, including the explicit handling of the unreachable "ast.Slice" message.
Approved from my side. Thanks again for the thorough follow-up.
Continues the validation work from #99.
Methods defined in a class body were accepted
Wiki Issue 4 lists methods among the initial unsupported constructs, but nothing caught them. A method is an ordinary function in the extracted ast, and its first parameter is only conventionally named
self, so all forms validated cleanly. Extraction now carries__qualname__onExtractedSource(the same rationale asglobalnsandfreevars: runtime context the ast cannot supply) and validation uses it. A function nested inside another function is dotted too, but Python inserts the<locals>marker there, which distinguishes the two.Scope of the check — narrowed after review
__qualname__records where a function was defined, not whether a class currently owns the descriptor. The check is therefore deliberately named_defined_in_class_bodyrather than_is_method, and the contract is documented on the predicate, on thequalnamefield, and onvalidate. Measured boundary:__qualname__Calc.scaledCalc.statCalc.klassfunctools.wrapsdecorator chainCalc.wrapped__qualname__/__wrapped__replacement.<locals>.wrappedsample_module_levelsample_module_levelAll four detected forms are pinned by a parametrized test — the classmethod matters separately because it resolves to a bound method object rather than a function. The three undetected cases are pinned too, as accepted, with the reason in the test docstring, so the contract is explicit and a future fix has a test to flip.
Follow-up, verified not assumed: I checked on 3.10/3.11/3.14 that
__set_name__is called for anything bound in a class body, including a decorator's return value — so the@jitlayer can close the first two cases by carrying the owning class. It is not called for assignment after class creation, so that case stays undetectable at any layer. This lands in the wiring PR, which is wherecore.pygets touched; this PR does not modify the decorator. One wrinkle recorded for it: an exception raised from__set_name__is wrapped inRuntimeErroron 3.10/3.11 but propagates bare on 3.12+, whichstrict=Truewill have to account for.Not covered here for the same reason: the
@jit+@staticmethod/@classmethoddecorator-order combinations, which need the decorator.The rejection matrix is now table-driven and exhaustive
The per-construct rejection tests are replaced by a matrix over
_UNSUPPORTED_MESSAGES, plus a test asserting the matrix covers every entry in the table, so it cannot quietly fall behind the validator (wiki Issue 12, "tests for unsupported syntax diagnostics"). Each case asserts the construct reports its own message rather than one belonging to an enclosing node.That check immediately found three entries whose message was shadowed:
ast.Awaitawait xreports "standalone expression statements"v = await xast.Starredv = [*x]reports "list literals"range(*x)ast.Slicex[0:1]reports "subscripting"Subscriptand the container types reject without descending, so their children's messages are unreachable. On severity: every one of these constructs is still rejected, just with a less specific message — message quality plus one dead table entry, not a correctness hole.The first two are now exercised in the positions that reach them. Design point:
ast.Sliceis kept as a safety net and named in_SHADOWED_MESSAGESso the exhaustiveness test stays honest, rather than silently deleted. Happy to drop the entry instead if you'd prefer the table to contain only reachable messages.The matrix builds its sources as text rather than as real nested functions, so a lint autofix cannot rewrite the construct under test — that happened previously with unused in-function imports being stripped by
ruff --fix, leaving tests validating the wrong source.Verification
153 tests, arxjit coverage 100%. All gates green locally: pytest+cov, ruff format/check, mypy strict (
cd packages/arxjit && mypy src), bandit-iii -lll, vulture 80, mccabe 10,douki syncidempotent.Verified on CPython 3.10.19, 3.11.11 and 3.14.6. The
TryStarcase is version-gated the same way the validator guards its table entry, and appears only on 3.11+.