leverage sphinx config info to improve parsing

[drammock]

@lagru I tried docstub on MNE-Python with the following results:

6241 matched annotations
511 grammar violations
1972 unknown doctypes

Two ideas for improvement:

• I think a lot of the grammar violations could be avoided by reading the package's conf.py and looking at numpydoc_xref_ignore (for us, this includes things like "instance of", "shape", "n_channels", "n_times", etc)
• I think a lot of the unknown doctypes could be resolved by looking at numpydoc_xref_aliases and/or leveraging intersphinx mappings.

This is something I'd be willing to work on (though maybe not right away), if you think it's in-scope for docstub. WDYT?

[lagru]

Oh, thanks for trying it! 😊

I wasn't yet aware of things like numpydoc_xref_ignore and numpydoc_xref_ignore so, very useful feedback.

Regarding the scope, I don't have a clear opinion yet to which extend I want to support things specific to Sphinx and NumPyDoc. Considering that I'm writing the tool for the SP community that seems practical. Do you have a feeling for how widely used these NumPyDoc specific variables are?

At the very least, I fully intend to make room for supporting these in the form of a plugin / hook system. I added that as an item in the early development roadmap.

If you want to invest time, I'm happy to bump the priority so that you can explore docstub more effectively in terms of MNE-Python. I'd be happy to review PRs, thanks for offering! Just be aware, that that the internal architecture of docstub hasn't settled yet. Some abstractions still feel off to me.

[lagru]

To add some context, "grammar violations" are type description whose syntax doesn't match the one defined in src/docstub/doctype.lark. "unknown doctypes" are names in type descriptions with a valid syntax that docstub doesn't know the origin for. That also means that it doesn't know how to import them and falls back to from typing import Any as unknown_name.

[drammock]

Do you have a feeling for how widely used these NumPyDoc specific variables are?

https://github.com/search?q=numpydoc_xref_aliases&type=code has 410 hits (though many are forks/duplicates), including ArViz, Astropy, pymc, arrow, something from NASA, etc. (and a few different packages in the MNE organization). I know SciPy uses it too. The corresponding search for numpydoc_xref_ignore has 428 hits.

At the very least, I fully intend to make room for supporting these in the form of a plugin / hook system.

Hopefully I'll convince you to make it part of core, not a plugin, as I think for projects that don't use numpydoc it will be a very fast no-op (i.e., "does conf.py exist? if so, is that variable defined in there? if not, do nothing")

the internal architecture of docstub hasn't settled yet. Some abstractions still feel off to me.

OK thanks for the tip. If I have ideas about that while hacking, I'll open other issues to discuss.

"grammar violations" are type description whose syntax doesn't match the one defined in src/docstub/doctype.lark. "unknown doctypes" are names in type descriptions with a valid syntax that docstub doesn't know the origin for.

Yep, I inferred that based on the names, but good to have confirmation that my mental model is correct, before I dive into the codbase :)

[OriolAbril]

I think that would be great but as a general use-case I think I'd want to use the aliases from conf.py then override some of them. At ArviZ for example we alias "hashable" to the glossary definition: https://docs.python.org/3/glossary.html#term-hashable instead of the collections.abc type (which is of little to no use to someone who doesn't already know what hashable means): https://docs.python.org/3/library/collections.abc.html#collections.abc.Hashable. But for docstubs I'd want the alias to be the collections.abc one.