hashtags in fenced code python comments clash with markdown headers in tagbar

[joaoandreporto]

Hi, I’m getting python comments inside fenced code, which should not be visible, clashing with markdown headers and creating an incorrect display of headers in tagbar, e.g.:

I have placed the following inside my .vimrc, after following through the manual’s integration steps (https://github.com/vimwiki/vimwiki/blob/619f04f89861c58e5a6415a4f83847752928252d/doc/vimwiki.txt#L1988) and vwtags.py:

let g:tagbar_type_vimwiki = {                                                 
                        \   'ctagstype':'vimwiki'                             
                        \ , 'kinds':['h:header']                              
                        \ , 'sro':'&&&'                                       
                        \ , 'kind2scope':{'h':'header'}                       
                        \ , 'sort':0                                          
                        \ , 'ctagsbin':'/Library/Scripts/vwscripts/vwtags.py'
                        \ , 'ctagsargs':'markdown'                            
                        \ }                                                   

I’ve tried to alter vwtags.py, in order to omit fenced code blocks. Yet, the result remains the same, once i run :TagbarToggle inside vimwiki.

To no avail, I’ve also tried using markdown2ctags.py, as well as omitting g:tagbar_type_ in .vimrc, all producing the same result.

Q: How can this be fixed?

[joaoandreporto]

Testing the script, which is on #12 , from outside vimwiki, worked as expected, i.e.:

From this:

$ python3 vwtags.py "markdown" test.md
Heading 1	test.md	/^# Heading 1$/;"	h	line:1
Heading 2	test.md	/^## Heading 2$/;"	h	line:3	header:Heading 1
Heading 3.1	test.md	/^### Heading 3.1$/;"	h	line:9	header:Heading 1&&&Heading 2
Heading 3.2	test.md	/^### Heading 3.2$/;"	h	line:11	header:Heading 1&&&Heading 2
Heading 3.3	test.md	/^### Heading 3.3$/;"	h	line:13	header:Heading 1&&&Heading 2
Heading 3.4	test.md	/^### Heading 3.4$/;"	h	line:17	header:Heading 1&&&Heading 2
PYTHON COMMENT 1	test.md	/^# PYTHON COMMENT 1$/;"	h	line:21
PYTHON COMMENT 2	test.md	/^# PYTHON COMMENT 2$/;"	h	line:24
Heading 3.5	test.md	/^### Heading 3.5$/;"	h	line:28	header:PYTHON COMMENT 2
PYTHON COMMENT 3	test.md	/^# PYTHON COMMENT 3$/;"	h	line:38
PYTHON COMMENT 4	test.md	/^# PYTHON COMMENT 4$/;"	h	line:39
Heading 2.2	test.md	/^## Heading 2.2$/;"	h	line:42	header:PYTHON COMMENT 4
Heading 3.6	test.md	/^### Heading 3.6$/;"	h	line:44	header:PYTHON COMMENT 4&&&Heading 2.2
Heading 4	test.md	/^#### Heading 4$/;"	h	line:46	header:PYTHON COMMENT 4&&&Heading 2.2&&&Heading 3.6
Heading 5	test.md	/^##### Heading 5$/;"	h	line:48	header:PYTHON COMMENT 4&&&Heading 2.2&&&Heading 3.6&&&Heading 4
PYTHON COMMENT 5	test.md	/^# PYTHON COMMENT 5$/;"	h	line:51
Heading 6	test.md	/^###### Heading 6$/;"	h	line:55	header:PYTHON COMMENT 5

To this:

$ python3 vwtags.py "markdown" test.md
Heading 1	test.md	/^# Heading 1$/;"	h	line:1
Heading 2	test.md	/^## Heading 2$/;"	h	line:3	header:Heading 1
Heading 3.1	test.md	/^### Heading 3.1$/;"	h	line:9	header:Heading 1&&&Heading 2
Heading 3.2	test.md	/^### Heading 3.2$/;"	h	line:11	header:Heading 1&&&Heading 2
Heading 3.3	test.md	/^### Heading 3.3$/;"	h	line:13	header:Heading 1&&&Heading 2
Heading 3.4	test.md	/^### Heading 3.4$/;"	h	line:17	header:Heading 1&&&Heading 2
Heading 3.5	test.md	/^### Heading 3.5$/;"	h	line:22	header:Heading 1&&&Heading 2
Heading 2.2	test.md	/^## Heading 2.2$/;"	h	line:33	header:Heading 1
Heading 3.6	test.md	/^### Heading 3.6$/;"	h	line:35	header:Heading 1&&&Heading 2.2
Heading 4	test.md	/^#### Heading 4$/;"	h	line:37	header:Heading 1&&&Heading 2.2&&&Heading 3.6
Heading 5	test.md	/^##### Heading 5$/;"	h	line:39	header:Heading 1&&&Heading 2.2&&&Heading 3.6&&&Heading 4
Heading 6	test.md	/^###### Heading 6$/;"	h	line:43	header:Heading 1&&&Heading 2.2&&&Heading 3.6&&&Heading 4&&&Heading 5

Running ctags, directly, produced different tag kinds, because it is markdown specific, but still the result is what is intended, i.e. it now omits the fenced code and thus python comments are not considered has headers anymore:

$ ctags -f - --format=2 --excmd=pattern --extras= --fields=nksaSmt test.md
Heading 1	test.md	/^# Heading 1$/;"	c	line:1
Heading 2	test.md	/^## Heading 2$/;"	s	line:3
Heading 2.2	test.md	/^## Heading 2.2$/;"	s	line:42
Heading 3.1	test.md	/^### Heading 3.1$/;"	S	line:9
Heading 3.2	test.md	/^### Heading 3.2$/;"	S	line:11
Heading 3.3	test.md	/^### Heading 3.3$/;"	S	line:13
Heading 3.4	test.md	/^### Heading 3.4$/;"	S	line:17
Heading 3.5	test.md	/^### Heading 3.5$/;"	S	line:28
Heading 3.6	test.md	/^### Heading 3.6$/;"	S	line:44
Heading 4	test.md	/^#### Heading 4$/;"	t	line:46
Heading 5	test.md	/^##### Heading 5$/;"	T	line:48
Heading 6	test.md	/^###### Heading 6$/;"	u	line:55

Yet, this still produced no effect on tagbar, so I ran :TagbarDebug and discovered that it was calling vwtags.py from inside taskwiki:

(...)
2022-06-23 11:26:46.155974: typeinfo for file to process: {'ctagstype': 'default', 'ftype': 'vimwiki', 'kinddict': {'p': 1, 'v': 2, 'h': 0, '?': 3}, 'ctagsargs': 'markdown', 'sro': '&&&', 'kinds': [{'short': 'h', 'long': 'header', 'fold': 0, 'stl': 1}, {'short': 'p', 'long': 'preset', 'fold': 0, 'stl': 1}, {'short': 'v', 'long': 'viewport', 'fold': 0, 'stl': 1}, {'short': '?', 'long': 'unknown', 'fold': 0, 'stl': 1}], 'getKind': function('<SNR>114_getKind'), 'createKinddict': function('<SNR>114_createKinddict'), 'kind2scope': {'p': 'preset', 'v': 'viewport', 'h': 'header', '?': 'unknown'}, 'ctagsbin': '/Users/jporto/.vim/bundle/taskwiki/extra/vwtags.py', 'sort': 0, 'scope2kind': {'viewport': 'v', 'header': 'h', 'preset': 'p'}}
2022-06-23 11:26:46.156204: Caching file into: /var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/7.md
2022-06-23 11:26:46.180506: ExecuteCtagsOnFile called [/var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/7.md]
2022-06-23 11:26:46.181376: EscapeCtagsCmd called
2022-06-23 11:26:46.181633: ctags_bin: /Users/jporto/.vim/bundle/taskwiki/extra/vwtags.py
2022-06-23 11:26:46.181818: ctags_args (is a string):  markdown 
2022-06-23 11:26:46.182197: Escaped ctags command: '/Users/jporto/.vim/bundle/taskwiki/extra/vwtags.py'  markdown  '/var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/7.md'
2022-06-23 11:26:46.182471: Executing ctags command: '/Users/jporto/.vim/bundle/taskwiki/extra/vwtags.py'  markdown  '/var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/7.md'
2022-06-23 11:26:46.287400: Calling shell to execute: "('/Users/jporto/.vim/bundle/taskwiki/extra/vwtags.py'  markdown  '/var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/7.md')>/var/folders/3n/_tyk2bfx6kbb1ryzkxcpq1100000gs/T/vt7tlUD/8 2>&1"
2022-06-23 11:26:46.287784: Exit code: 0
2022-06-23 11:26:46.314059: Ctags executed successfully
2022-06-23 11:26:46.319006: Filetype tag kinds: ['p', 'v', 'h', '?']
(...)

Once taskwiki was disabled, the version of vwtags.py that I had assigned to ctagsbin, inside my .vimrc, worked.

I’ve found out that this issue tbabej/taskwiki#194 had been raised already, by @ranebrown, inside the taskwiki repo, but it still remains open.

I hope this can contribute, in any way, to help solve it.

[joaoandreporto]

Closed by #12