✨ Enhance docstring highlighting: multi-word, complex types, NumPy params, Sphinx rtype

Author: rodolphebarbanneau

.vscode/launch.json

@@ -11,6 +11,7 @@
       "type": "extensionHost",
       "request": "launch",
       "args": [
+        "--disable-extension=rodolphebarbanneau.python-docstring-highlighter",
         "--extensionDevelopmentPath=${workspaceFolder}",
         "--folder-uri=${workspaceFolder}/tests",
         "${workspaceFolder}/tests/google.py",

README.md

@@ -11,17 +11,19 @@ This extension is designed to **highlight docstrings** in **Python code**, makin
 
 ![Demo](https://raw.githubusercontent.com/rodolphebarbanneau/python-docstring-highlighter/main/assets/docstring.gif)
 
+> **Note**: The highlighting appearance in your editor may differ from this demo. Your personal color theme and the latest extension updates can affect the visual result. This demo is intended to showcase the general functionality rather than an exact representation.
+
 > **Technical Note**: The highlighting uses the VScode TextMate grammar injection feature, which means that it is compatible with any color theme and should not lead to performance issues (i.e. no custom scripts is executed). The extension uses a custom  grammar to match the docstring patterns and inject the appropriate scopes into the code editor.
 
-> **Technical Note**: TextMate grammar injection can only match one-ligne regular expressions consistently. This means that the extension will not be able to match multi-line patterns in the docstring. This is a limitation of the TextMate grammar injection feature and cannot be worked around.
+> **Technical Note**: TextMate grammar injection can only match one-line regular expressions consistently. This means that the extension will not be able to match multi-line patterns in the docstring. This is a limitation of the TextMate grammar injection feature and cannot be worked around.
 
 ## Requirements
 
 This extension is **only compatible** with the standard [VSCode Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python). It is not compatible with other language server like _Python for VSCode (deprecated)_.
 
 ## Extension Settings
 
-This extensions does not provide specific settings. It is designed to work out of the box with the default color theme. However, you can customize the color theme by overriding the default color theme in your `settings.json` file (see the [Customization](#customization) section).
+This extension does not provide specific settings. It is designed to work out of the box with the default color theme. However, you can customize the color theme by overriding the default color theme in your `settings.json` file (see the [Customization](#customization) section).
 
 ### Available Scopes
 
@@ -31,7 +33,7 @@ The following list shows the available scopes with their available sub-scopes sp
 
 #### `docstring.heading`
 
-Section headings in the docstring (e.g. `Args`, `Returns`, or `Raises`)
+Section headings in the docstring (e.g. `Args`, `Returns`, or `Raises`).
 
 - `.python`
 - `.begin.python`
@@ -58,7 +60,7 @@ Identifiers in the docstring (e.g. `:meth:`, or `:attr:`).
 
 #### `docstring.literal`
 
-Inline literals in the docstring (e.g. ``` ``"Sucssefully executed"`` ```).
+Inline literals in the docstring (e.g. ``` ``"Successful Error"`` ```). 😭
 
 - `.python`
 - `.begin.python`
@@ -103,9 +105,9 @@ You can customize the color theme by overriding the default color theme in your
       "scope": "docstring.heading.placeholder",
       "settings": {
         "fontStyle": "bold underline",
-        "foreground": "#569cd6",
+        "foreground": "#569cd6"
       }
-    },
+    }
   ]
 }
 ```
@@ -114,6 +116,13 @@ You can customize the color theme by overriding the default color theme in your
 
 ## Release Notes
 
+### 0.2.4
+
+- Added colorization for multi-word capitalized sections (e.g., `See Also`)
+- Fixed highlighting for complex type annotations (e.g., `Sequence[str]`)
+- Improved support for NumPy-style multiple parameters (e.g., `param1, param2 : int`)
+- Fixed Sphinx-style return type highlighting (e.g., `:rtype: int`)
+
 ### 0.2.3
 
 - Updated base heading pattern end token to match end of line

syntaxes/base.tmLanguage.json

@@ -17,7 +17,7 @@
           "name": "punctuation.definition.tag.end.docstring.python, docstring.heading.end.python"
         }
       },
-      "match": "(?:^\\s*)()(_*[A-Z]\\w*)(:?$)",
+      "match": "(?:^\\s*)()(_*(?:[A-Z]\\w*(?:\\s+(?!$))?)+)(:?$)",
       "name": "docstring.heading.python"
     },
     {

syntaxes/google.tmLanguage.json

@@ -17,7 +17,7 @@
           "name": "punctuation.definition.tag.end.docstring.python, docstring.variable.end.google.python"
         }
       },
-      "match": "(?:^\\s*)((?:-\\s*)?)((?:\\*\\*|\\*)?[a-zA-Z_]\\w*)((?:\\s\\(.*\\))?:)(?=\\s)",
+      "match": "(?:^\\s*)((?:-\\s*)?)((?:\\*\\*|\\*)?[a-zA-Z_]\\w*(?:\\[.*\\])?)((?:\\s\\(.*\\))?:)(?=\\s)",
       "name": "docstring.variable.google.python"
     },
 		{

syntaxes/numpy.tmLanguage.json

@@ -17,7 +17,7 @@
           "name": "punctuation.definition.tag.end.docstring.python, docstring.variable.end.numpy.python"
         }
       },
-      "match": "(?:^\\s*)((?:-\\s*)?)((?:\\*\\*|\\*)?[a-zA-Z_]\\w*)((?:\\s:\\s.*)?)(?:$)",
+      "match": "(?:^\\s*)((?:-\\s*)?)((?:(?:\\*\\*|\\*)?[a-zA-Z_]\\w*(?:\\[.*\\])?(?:\\s*,\\s*(?=[a-zA-Z_]))?)+)((?:\\s+:\\s+.*)?)(?:$)",
       "name": "docstring.variable.numpy.python"
     },
 		{

syntaxes/sphinx.tmLanguage.json

@@ -23,7 +23,7 @@
           "name": "punctuation.definition.tag.end.docstring.python, docstring.variable.end.sphinx.python"
         }
       },
-      "match": "(?:^\\s*)(:)(\\w+)(?:(\\s|(?:.*\\s))?((?:\\*\\*|\\*|_)?\\w*)?)((?:=[^:]+)?:)(?=\\s)",
+      "match": "(?:^\\s*)(:)(\\w+)(?:(\\s|(?:.*\\s))?((?:\\*\\*|\\*|_)?\\w*(?:\\[.*\\])?)?)((?:=[^:]+)?:(?:(?:(?<=:rtype:)\\s.*)|(?=\\s)))",
       "name": "docstring.variable.sphinx.python"
     },
 		{

tests/google.py

@@ -24,13 +24,19 @@
         one convention to document module level variables and be consistent
         with it.
 
+See Also:
+    `Extension Documentation`_ for more details on the extension.
+
 .. todo::
     * For TODOs directive module
     * You have to also use ``sphinx.ext.todo`` extension;
 
 .. _Google Python Style Guide:
     https://google.github.io/styleguide/pyguide.html
     https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
+
+.. _Extension Documentation:
+    https://marketplace.visualstudio.com/items?itemName=rodolphebarbanneau.python-docstring-highlighter
 """
 
 module_level_variable1 = 12345
@@ -43,7 +49,7 @@
 """
 
 
-def function_with_types_in_docstring(param1, param2):
+def function_with_types_in_docstring(param1, param2, param3):
     """Example function with types documented in the docstring.
 
     `PEP 484`_ type annotations are supported. If attribute, parameter, and
@@ -52,7 +58,8 @@ def function_with_types_in_docstring(param1, param2):
 
     Args:
         param1 (int): The first parameter.
-        param2 (str): The second parameter.
+        param2 (int): The second parameter.
+        param3 (str): The third parameter.
 
     Returns:
         bool: The return value. True for success, False otherwise.
@@ -63,12 +70,15 @@ def function_with_types_in_docstring(param1, param2):
     pass
 
 
-def function_with_pep484_type_annotations(param1: int, param2: str) -> bool:
+def function_with_pep484_type_annotations(
+    param1: int, param2: int, param3: str
+) -> bool:
     """Example function with PEP 484 type annotations.
 
     Args:
         param1: The first parameter.
         param2: The second parameter.
+        param3: The third parameter.
 
     Returns:
         The return value. True for success, False otherwise.
@@ -248,7 +258,7 @@ def example_method(self, param1, param2):
             param2: The second parameter.
 
         Returns:
-            True if successful, False otherwise.
+            Sequence[str]: A sequence of strings.
         """
         return True
 

tests/numpy.py

@@ -15,7 +15,7 @@
 
 Section breaks are created with two blank lines. Section breaks are also
 implicitly created anytime a new section starts. Section bodies *may* be
-indented:
+indented.
 
 Notes
 -----
@@ -35,6 +35,9 @@
     Either form is acceptable, but the two should not be mixed. Choose one
     convention to document module level variables and be consistent with it.
 
+See Also
+--------
+`Extension Documentation`_ for more details on the extension.
 
 .. todo::
     * For TODOs directive module
@@ -43,6 +46,9 @@
 .. _NumPy Python Style Guide:
     https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt
     https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html
+
+.. _Extension Documentation:
+    https://marketplace.visualstudio.com/items?itemName=rodolphebarbanneau.python-docstring-highlighter
 """
 
 module_level_variable1 = 12345
@@ -55,7 +61,7 @@
 """
 
 
-def function_with_types_in_docstring(param1, param2):
+def function_with_types_in_docstring(param1, param2, param3):
     """Example function with types documented in the docstring.
 
     `PEP 484`_ type annotations are supported. If attribute, parameter, and
@@ -64,10 +70,10 @@ def function_with_types_in_docstring(param1, param2):
 
     Parameters
     ----------
-    param1 : int
-        The first parameter.
-    param2 : str
-        The second parameter.
+    param1, param2 : int
+        The first and second parameters.
+    param3 : str
+        The third parameter.
 
     Returns
     -------
@@ -80,15 +86,17 @@ def function_with_types_in_docstring(param1, param2):
     pass
 
 
-def function_with_pep484_type_annotations(param1: int, param2: str) -> bool:
+def function_with_pep484_type_annotations(
+    param1: int, param2: int, param3: str
+) -> bool:
     """Example function with PEP 484 type annotations.
 
     Parameters
     ----------
-    param1
-        The first parameter.
-    param2
-        The second parameter.
+    param1, param2
+        The first and second parameters.
+    param3
+        The third parameter.
 
     Returns
     -------
@@ -310,8 +318,8 @@ def example_method(self, param1, param2):
 
         Returns
         -------
-        bool
-            True if successful, False otherwise.
+        Sequence[str]
+            A sequence of strings.
         """
         return True
 

tests/sphinx.py

@@ -18,12 +18,18 @@
 The :attr:`module_level_variable1` module level variables must be documented
 in an inline docstring immediately following the variable.
 
+See Also:
+    `Extension Documentation`_ for more details on the extension.
+
 .. todo::
     * For TODOs directive module
     * You have to also use ``sphinx.ext.todo`` extension
 
 .. _Sphinx Python Style Guide:
     https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html
+
+.. _Extension Documentation:
+    https://marketplace.visualstudio.com/items?itemName=rodolphebarbanneau.python-docstring-highlighter
 """
 
 module_level_variable1 = 12345
@@ -36,31 +42,34 @@
 """
 
 
-def function_with_types_in_docstring(param1, param2):
+def function_with_types_in_docstring(param1, param2, param3):
     """Example function with types documented in the docstring.
 
     `PEP 484`_ type annotations are supported. If attribute, parameter, and
     return types are annotated according to `PEP 484`_, they do not need to be
     included in the docstring:
 
     :param int param1: The first parameter.
-    :param str param2: The second parameter.
-
+    :param int param2: The second parameter.
+    :param str param3: The third parameter.
+    :return: The return value. True for success, False otherwise.
     :rtype: bool
-    :returns: The return value. True for success, False otherwise.
 
     .. _PEP 484:
         https://www.python.org/dev/peps/pep-0484/
     """
     pass
 
 
-def function_with_pep484_type_annotations(param1: int, param2: str) -> bool:
+def function_with_pep484_type_annotations(
+    param1: int, param2: int, param3: str
+) -> bool:
     """Example function with PEP 484 type annotations.
 
     :param param1: The first parameter.
     :param param2: The second parameter.
-    :returns: The return value. True for success, False otherwise.
+    :param param3: The third parameter.
+    :return: The return value. True for success, False otherwise.
     """
     return True
 
@@ -92,15 +101,15 @@ def module_level_function(param1, param2=None, *args, **kwargs):
     :param kwargs: Miscellaneous.
 
     :rtype: bool
-    :returns: True if successful, False otherwise.
+    :return: True if successful, False otherwise.
 
         The return type is optional and may be specified using the ``:rtype:``
         directive.
 
-        The ``:returns:`` description may span multiple lines and paragraphs.
+        The ``:return:`` description may span multiple lines and paragraphs.
         Following lines should be indented to match the first line.
 
-        The ``:returns:`` description supports any reStructuredText formatting,
+        The ``:return:`` description supports any reStructuredText formatting,
         including literal blocks::
 
             {
@@ -218,8 +227,8 @@ def example_method(self, param1, param2):
 
         :param str param1: The first parameter.
         :param int param2: The second parameter.
-        :returns: True if successful, False otherwise.
-        :rtype: bool
+        :return: A sequence of strings.
+        :rtype: Sequence[str]
 
         .. note::
             Do not include the `self` parameter in the ``:param:``directives.