Skip to content

Fix xonsh integration breaking on backslash line-continuation#905

Open
anki-code wants to merge 1 commit into
evhub:developfrom
anki-code:fix-xonsh-line-continuations
Open

Fix xonsh integration breaking on backslash line-continuation#905
anki-code wants to merge 1 commit into
evhub:developfrom
anki-code:fix-xonsh-line-continuations

Conversation

@anki-code
Copy link
Copy Markdown
Contributor

@anki-code anki-code commented May 2, 2026

Fixes #906

Summary

When the Coconut xontrib is autoloaded into xonsh and the user enters a multiline subprocess command joined with \<newline>, xonsh raises SyntaxError: unexpected indent on input that works fine without Coconut.

@ echo a \
    b
SyntaxError: code: b

Root cause

Compiler.reind_proc intentionally splits backslash-continued source into two physical lines and indents the continuation tail by tabideal spaces "for readability". For standalone Coconut output that's fine — Python tolerates it through implicit string/expression continuation. But the xonsh xontrib feeds the compiled output back through xonsh's own parser as top-level statements, where a complete statement followed by an unexpected INDENT is a SyntaxError. Concretely, echo a \<nl>b becomes echo a #1: echo a \<nl> b #1: echo a \<nl>, which neither xonsh nor CPython will parse.

I confirmed via tokenize.tokenize that this output is invalid CPython too — not just xonsh-specific:

TokenInfo(type=NAME, 'echo'), TokenInfo(NAME, 'a'), COMMENT, NEWLINE,
TokenInfo(INDENT, '    '), TokenInfo(NAME, 'b'), COMMENT, NEWLINE

The INDENT after a complete echo a statement is the bug.

Fix

Collapse \<newline><indent> into a single space before handing the source to parse_xonsh. With the line continuation already collapsed, the compiler emits one physical line on the other side and the host parser is happy.

The collapse is a small manual scanner — string literals and comments are skipped so backslash sequences inside them keep their original meaning. (A plain re.sub would silently rewrite "a\<nl>b", which is a Python implicit-string line continuation.)

Test plan

Reproduced + verified against xonsh-dev (Python 3.13, xonsh main, coconut 3.2.0 + this patch):

  • @ echo a \⏎ba b (was SyntaxError)
  • @ print(range(5) |> list)[0, 1, 2, 3, 4] (Coconut features unchanged)
  • @ double = x -> x*2; print(double(5))10
  • @ print(sum <| [1,2,3])6
  • @ s = "a\⏎b"; print(repr(s))'ab' (Python implicit-string continuation preserved)
  • 14 unit-tests on _collapse_xonsh_line_continuations (no-op input, simple/multi continuations, strings/triple-quotes/comments preserved, trailing continuation, empty input)
  • Existing test_xontrib scenarios still pass when run against the patched compile path (single-quoted continuation, semicolon chains, env-var f-strings, multi-line strings)

Compatibility

No core compiler changes — patch is contained to coconut/integrations.py (xonsh-specific module). Other Coconut endpoints (parse_sys, parse_eval, parse_file, …) and the rest of the runtime are untouched.

Reported by xonsh users at #906 (the linked xonsh issue is about a related but separate xonsh-side bug; this fixes a Coconut-specific path that surfaces only with the xontrib loaded).

@nahoj
Copy link
Copy Markdown

nahoj commented May 2, 2026

Works for me, thanks @anki-code :)

@jorel6666
Copy link
Copy Markdown

This was answered by my ai arkhan by itself jorel666 be a part of the future hit me up

@evhub
Copy link
Copy Markdown
Owner

evhub commented May 4, 2026

This looks great! Can you rebase and retest on develop instead of master?

@evhub evhub added this to the v3.3.0 milestone May 4, 2026
@evhub evhub removed the tooling label May 4, 2026
When Coconut is loaded as a xonsh xontrib and the user enters a
multiline subprocess command joined by ``\<newline>``, xonsh raises
``SyntaxError: unexpected indent`` for input that worked fine without
Coconut.

Root cause: ``Compiler.reind_proc`` intentionally splits backslash-
continued source into two physical lines and indents the continuation
tail by ``tabideal`` spaces "for readability" of the compiled output.
That is fine for standalone Coconut output — Python tolerates it via
the implicit string/expression continuation rules — but the xonsh
xontrib feeds the compiled output back through xonsh's own parser as
top-level statements, where a complete statement followed by an
unexpected INDENT is a SyntaxError.  Concretely, ``echo a \<nl>b``
becomes ``echo a  evhub#1: echo a \<nl>    b  evhub#1: echo a \<nl>``, which
neither xonsh nor CPython will parse.

Fix: collapse ``\<newline><indent>`` into a single space before
handing the source to ``parse_xonsh``, so the compiler emits one
physical line on the other side and the host parser is happy.  The
collapse walks the source manually so backslash sequences inside
string literals and comments are left alone (a plain ``re.sub`` would
silently rewrite ``"a\<nl>b"``, which is a Python implicit-string
line continuation and must keep its meaning).

Reproducer (with the xontrib autoloaded):

    @ echo a \\
        b
    SyntaxError: code: b   ← before
    a b                    ← after

Other xonsh-side scenarios (``echo && cmd`` chains, Coconut features
like ``|>``/``->``/``<|``, capture forms, xonsh assignments) are
unaffected.
@anki-code anki-code force-pushed the fix-xonsh-line-continuations branch from f64e0fb to c1ae96f Compare May 4, 2026 05:12
@anki-code anki-code changed the base branch from master to develop May 4, 2026 05:12
@anki-code
Copy link
Copy Markdown
Contributor Author

@evhub Rebased onto develop (one commit, no merge conflicts — only coconut/integrations.py is touched, and develop made no changes there since v3.2.0). PR base updated to develop.

Retested against develop (Python 3.14, xonsh 0.23.3.dev12, cPyparsing 2.4.7.2.4.3):

  • 14/14 unit tests on _collapse_xonsh_line_continuations pass (no-op, simple/multi continuations, single/double/triple-quoted strings preserved, comments preserved, escaped quotes, trailing continuation, mixed string + continuation, empty input)
  • Tokenize check on the bug repro confirms 0 spurious INDENT tokens after collapse
  • Compile-path scenarios driven directly through the xontrib's compile_code path: echo a \⏎ b, print(range(5) |> list), double = x -> x*2; print(double(5)), print(sum <| [1,2,3]), !(ls -la) |> bool, '1; 2' |> print, env-var assign, triple-quoted multi-line len, semicolon chains, echo abc; print(1 |> (.+1)), execx("10 |> print") — all parse to expected output, no SyntaxError

The patch logic is byte-identical to before the rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xonsh multiline with coconut: SyntaxError

4 participants