feature: hooks for injecting hidden code (could be extended for other uses)

[benjamincburns]

Problem

In our docs for LangGraph we use tools like VCR and nock to cache and replay interactions between our code samples and the APIs with which our samples interact.

When we built our docs from Jupyter notebooks, we'd inject code for setting these up into the notebooks as non-rendered blocks. This allows us to have the setup/teardown defined centrally, and means less boilerplate in each notebook file.

markdown-exec sessions are throwing us a curve-ball, however. We'd have to process each block in the doc, catalogue the sessions, and insert a pre/post amble block for each one.

Proposed solution

It would be nice if I could register callbacks to markdown-exec that would be called at the start and end of each session, and perhaps also at the start and end of each block.

Configuration

The callbacks could be registered via the mkdocs.yml config in the following way:

plugins:
  - markdown-exec:
    hooks:
      python:
        pre_session: # called before the first code block of a python session runs
          - module.to.import:python_pre_session_callback_fn
        post_session: # called after the last code block of a python session runs
          - module.to.import:python_post_session_callback_fn
        pre_block: # called before the execution of each python block
          - module.to.import:python_pre_block_callback_fn
        post_block: # called after each successful execution of each python block
          - module.to.import:python_post_block_callback_fn
        block_error: # called after each erroneous execution of each python block
          - module.to.import:python_block_error_callback_fn
      typescript:
        pre_session: # called before the first code block of a typescript session runs
          - module.to.import:typescript_pre_session_callback_fn
        post_session: # called after the last code block of a typescript session runs
          - module.to.import:typescript_post_session_callback_fn
        pre_block: # called before the execution of each typescript block
          - module.to.import:typescript_pre_block_callback_fn
        post_block: # called after each successful execution of each typescript block
          - module.to.import:typescript_post_block_callback_fn
        block_error: # called after each erroneous execution of each typescript block
          - module.to.import:typescript_block_error_callback_fn

Hook arguments

As for what arguments to pass, I think these functions should ideally get the full context available. So that would include

• all of the arguments passed to the formatter on the invocation that triggered the hook
  • in addition, context about the filename & path of the document being rendered, if that's not already in the kwargs for the formatter calls
• in the case of post_* and error_* hooks, the result of the execution that triggered the call

You could also pass the code and output history, but our immediate use case doesn't require this.

Hook return values

To make this mechanism extensible, hook functions should return an object (or list of objects?) that instruct markdown-exec to carry out some operation.

Example operations could include:

• Execute some setup/teardown code before/after the block/session, but don't include it or its output in the rendered output for the block
  • This is the case that we have immediate need for
• pre_block only: Replace the code in the given block with the code returned in the object
• pre_block only: Skip execution of the block, and render the output as given (provides an extension point that would be useful for solving feature: only recompute dirty docs during editing #71, perhaps also Cache #4?)

[pawamoy]

Thanks for the request and ideas @benjamincburns 🙂

For session-scoped setup/teardown I'd recommend regular executed code blocks that don't print anything. It's easy to inject code from external files thanks to extensions like pymdownx.snippets:

Some Markdown document.

```python exec="1" session="some-session"
--8<-- "path/to/some-session-setup.py
```

```python exec="1" session="some-session"
print("Some Python code.")
```

```python exec="1" session="some-session"
--8<-- "path/to/some-session-teardown.py
```

I understand this creates a bit of boilerplate (with the example above: 2 code blocks / 6 lines per document).

For block-scoped setup/teardown, I can't think of any easy way. Injecting with pymdownx.snippets would be very repetitive. The alternative would be to write code in external files that deals with running setup/teardown thanks to functions/decorators or something. Error handling could probably be done the same way, though not exactly sure how.

What I'd like to see is actual examples of how you'd use such hooks: what code you would execute as setup/teardown/error handling.

[benjamincburns]

My project has around 200 docs files, all of them will use multiple sessions (at least two, often more). Your alternative proposal would require us to remember to inject 6 boilerplate lines times however many sessions we have across those 200 files. Bit past the "rule of three," don't you think?

[pawamoy]

Yup. Especially with multiple sessions per file, which I didn't think about. I'd still like to see some concrete hook examples 😄

[benjamincburns]

I'd still like to see some concrete hook examples

This code is a little janky at the moment (especially the thing where I'm injecting the document path as a fence attribute), but here is an example of a pre-session hook that we're using to inject code that sets up HTTP request recording/replay with VCR for python and nock for JS.

And here is the corresponding teardown hook.

There's probably a cleaner design to accomplish this, so squint a little (or a lot!) when you read this, but hopefully that conveys the gist.

I'm also treating blocks w/ no session attribute as both starting an ending a session in a single block. That was convenient for my case, but probably would be more predictable/explainable if pre/post block hooks were used for that.

[pawamoy]

Thank you!

I thought about specifying paths to setup/teardown functions in code block attributes, but this would have meant lots of boilerplate and redundancy again. We already set a session name on each code block, so it'd be a shame not to use it directly.

So, lets move forward with this 🙂 I'm all for making Markdown Exec much more powerful 😄

However right now I'm a bit torn between the two aspects of the project: it's both a MkDocs plugin and a Markdown extension. Well, not really: it's a SuperFences' extension. Ideally, everything we can do from the plugin perspective, except the stuff that is entirely specific to MkDocs, should be doable from the extension perspective. Therefore I might want to refactor the project first to make it a proper Markdown extension. The plugin would just forward the options to it (making any paths relative to the mkdocs.yml config file). Session-related hooks implemented in the plugin should be somehow exposed as public API so that simple Markdown users or other SSGs can manage sessions the same way.

Then all new features we implement will work both from the Markdown extension perspective as well as from the MkDocs plugin one (with just a tiny bit of additional code to forward stuff). WDYT?