[BUG] Test failures with pygments 2.19

[mweinelt]

• I've checked docs and closed issues for possible solutions.
• I can't find my issue in the FAQ.

Describe the bug

After upgrading pygments to 2.19 we are seeing the following test failures on nixpkgs with rich 13.9.4:

rich> _______________________________ test_inline_code _______________________________
rich> 
rich>     def test_inline_code():
rich>         markdown = Markdown(
rich>             "inline `import this` code",
rich>             inline_code_lexer="python",
rich>             inline_code_theme="emacs",
rich>         )
rich>         result = render(markdown)
rich>         expected = "inline \x1b[1;38;2;170;34;255;48;2;248;248;248mimport\x1b[0m\x1b[38;2;0;0;0;48;2;248;248;248m \x1b[0m\x1b[1;38;2;0;0;255;48;2;248;248;248mthis\x1b[0m code                                                                             \n"
rich>         print(result)
rich>         print(repr(result))
rich> >       assert result == expected
rich> E       AssertionError: assert 'inline \x1b[...           \n' == 'inline \x1b[...           \n'
rich> E         
rich> E         Skipping 51 identical leading characters in diff, use -v to show
rich> E         - [0m this code                                                                             
rich> E         ?           ^ ^ ^
rich> E         + [0m this code                                                                             
rich> E         ?           ^^^ ^^^ ^^^
rich> 
rich> tests/test_markdown.py:116: AssertionError
rich> ----------------------------- Captured stdout call -----------------------------
rich> 'inline \x1b[1;38;2;170;34;255;48;2;248;248;248mimport\x1b[0m\x1b[38;2;187;187;187;48;2;248;248;248m \x1b[0m\x1b[1;38;2;0;0;255;48;2;248;248;248mthis\x1b[0m code                                                                             \n'
rich> inline import this code                                                                             
rich> 
rich> 'inline \x1b[1;38;2;170;34;255;48;2;248;248;248mimport\x1b[0m\x1b[38;2;187;187;187;48;2;248;248;248m \x1b[0m\x1b[1;38;2;0;0;255;48;2;248;248;248mthis\x1b[0m code                                                                             \n'
rich> _______________________________ test_blank_lines _______________________________
rich> 
rich>     def test_blank_lines():
rich>         code = "\n\nimport this\n\n"
rich>         syntax = Syntax(
rich>             code, lexer="python", theme="ascii_light", code_width=30, line_numbers=True
rich>         )
rich>         result = render(syntax)
rich>         print(repr(result))
rich> >       assert (
rich>             result
rich>             == "\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m1 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m2 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m3 \x1b[0m\x1b[1;38;2;0;128;0;48;2;248;248;248mimport\x1b[0m\x1b[38;2;0;0;0;48;2;248;248;248m \x1b[0m\x1b[1;38;2;0;0;255;48;2;248;248;248mthis\x1b[0m\x1b[48;2;248;248;248m                   \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m4 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m5 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n"
rich>         )
rich> E       AssertionError: assert '\x1b[1;38;2;...    \x1b[0m\n' == '\x1b[1;38;2;...    \x1b[0m\n'
rich> E         
rich> E         Skipping 398 identical leading characters in diff, use -v to show
rich> E         - [0m this                   
rich> E         ?           ^ ^ ^
rich> E         + [0m this                   
rich> E         ?           ^^^ ^^^ ^^^
rich> E             4                               
rich> E             5                               
rich> 
rich> tests/test_syntax.py:54: AssertionError
rich> ----------------------------- Captured stdout call -----------------------------
rich> '\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m1 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m2 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m3 \x1b[0m\x1b[1;38;2;0;128;0;48;2;248;248;248mimport\x1b[0m\x1b[38;2;187;187;187;48;2;248;248;248m \x1b[0m\x1b[1;38;2;0;0;255;48;2;248;248;248mthis\x1b[0m\x1b[48;2;248;248;248m                   \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m4 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n\x1b[1;38;2;24;24;24;48;2;248;248;248m  \x1b[0m\x1b[38;2;173;173;173;48;2;248;248;248m5 \x1b[0m\x1b[48;2;248;248;248m                              \x1b[0m\n'
rich> ___________________ test_python_render_simple_indent_guides ____________________
rich> 
rich>     @pytest.mark.skipif(OLD_PYGMENTS, reason="Pygments changed their tokenizer")
rich>     def test_python_render_simple_indent_guides():
rich>         syntax = Syntax(
rich>             CODE,
rich>             lexer="python",
rich>             line_numbers=False,
rich>             theme="ansi_light",
rich>             code_width=60,
rich>             word_wrap=False,
rich>             indent_guides=True,
rich>         )
rich>         rendered_syntax = render(syntax)
rich>         print(repr(rendered_syntax))
rich>         expected = '\x1b[34mdef\x1b[0m \x1b[32mloop_first_last\x1b[0m(values: Iterable[T]) -> Iterable[Tuple[\x1b[36mb\x1b[0m\n\x1b[2;37m│   \x1b[0m\x1b[33m"""Iterate and generate a tuple with a flag for first an\x1b[0m\n\x1b[2m│   \x1b[0miter_values = \x1b[36miter\x1b[0m(values)\n\x1b[2m│   \x1b[0m\x1b[34mtry\x1b[0m:\n\x1b[2m│   │   \x1b[0mprevious_value = \x1b[36mnext\x1b[0m(iter_values)\n\x1b[2m│   \x1b[0m\x1b[34mexcept\x1b[0m \x1b[36mStopIteration\x1b[0m:\n\x1b[2m│   │   \x1b[0m\x1b[34mreturn\x1b[0m\n\x1b[2m│   \x1b[0mfirst = \x1b[34mTrue\x1b[0m\n\x1b[2m│   \x1b[0m\x1b[34mfor\x1b[0m value \x1b[35min\x1b[0m iter_values:\n\x1b[2m│   │   \x1b[0m\x1b[34myield\x1b[0m first, \x1b[34mFalse\x1b[0m, previous_value\n\x1b[2m│   │   \x1b[0mfirst = \x1b[34mFalse\x1b[0m\n\x1b[2m│   │   \x1b[0mprevious_value = value\n\x1b[2m│   \x1b[0m\x1b[34myield\x1b[0m first, \x1b[34mTrue\x1b[0m, previous_value\n'
rich> >       assert rendered_syntax == expected
rich> E       assert '\x1b[34mdef\...vious_value\n' == '\x1b[34mdef\...vious_value\n'
rich> E         
rich> E         - def loop_first_last(values: Iterable[T]) -> Iterable[Tuple[b
rich> E         ?             ^
rich> E         + def loop_first_last(values: Iterable[T]) -> Iterable[Tuple[b
rich> E         ?             ^^^^^^^^^^
rich> E           │   """Iterate and generate a tuple with a flag for first an
rich> E           │   iter_values = iter(values)...
rich> E         
rich> E         ...Full output truncated (10 lines hidden), use '-vv' to show
rich> 
rich> tests/test_syntax.py:123: AssertionError

[github-actions]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[stanislavlevin]

It was fixed in #3768 and released with Rich 14.1.0.

[mweinelt]

Confirmed. Thanks!

[github-actions]

I hope I helped!

Consider sponsoring my work on Rich. I give tech support for free, in addition to maintaining Rich and Textual.

If you like using Rich, you might also enjoy Textual.

Will McGugan

[sandrotosi]

It was fixed in #3768 and released with Rich 14.1.0.

im packagng rich/14.1.0 on debian (which still uses pygments/2.18.0) and it fails with a similar traceback. If i update it to pygments/2.19.0 the failure goes away.

Im wondering if there should be an update to the versioned dependency in pyproject.toml to require pygments/2.19.0 as that now seem to be the minimum version needed