What happened?
When using salt-ssh, pkg.installed state cannot be found. After a bit of debugging, I found this:
[ERROR ] Failed to import module yumpkg, this is due most likely to a syntax error:
Traceback (most recent call last):
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 902, in _load_module
self.run(spec.loader.exec_module, mod)
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 1365, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 1380, in _run_as
ret = _func_or_method(*args, **kwargs)
File "<frozen importlib._bootstrap_external>", line 1023, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/var/tmp/.root_dd8a91_salt/running_data/var/cache/salt/minion/extmods/modules/yumpkg.py", line 25, in <module>
import configparser
File "/var/tmp/.root_dd8a91_salt/running_data/var/cache/salt/minion/extmods/utils/configparser.py", line 12, in <module>
class GitConfigParser(RawConfigParser):
^^^^^^^^^^^^^^^
NameError: name 'RawConfigParser' is not defined
The test call was:
salt-ssh f42xtest-target state.single pkg.installed mc
Note yumpkg tries to import configparser from Python's stdlib, but got one from salt.utils instead. The salt.utils.configparser seems to be specific to gitfs, and even that fails for similar reason:
[ERROR ] Failed to import fileserver gitfs, this is due most likely to a syntax error:
Traceback (most recent call last):
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 902, in _load_module
self.run(spec.loader.exec_module, mod)
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 1365, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader/lazy.py", line 1380, in _run_as
ret = _func_or_method(*args, **kwargs)
File "<frozen importlib._bootstrap_external>", line 1023, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/var/tmp/.root_dd8a91_salt/pyall/salt/fileserver/gitfs.py", line 52, in <module>
import salt.utils.gitfs
File "/var/tmp/.root_dd8a91_salt/pyall/salt/utils/gitfs.py", line 31, in <module>
import salt.utils.configparser
File "/var/tmp/.root_dd8a91_salt/pyall/salt/utils/configparser.py", line 7, in <module>
from configparser import * # pylint: disable=no-name-in-module,wildcard-import,unused-wildcard-import
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/.root_dd8a91_salt/running_data/var/cache/salt/minion/extmods/utils/configparser.py", line 12, in <module>
class GitConfigParser(RawConfigParser):
^^^^^^^^^^^^^^^
NameError: name 'RawConfigParser' is not defined
This all seems to be caused by prepending /var/tmp/.root_dd8a91_salt/running_data/var/cache/salt/minion/extmods/utils into sys.path, so modules there take precedence over Python's stdlib. Looking at the content of that directory, there are likely more potential conflicts: functools, itertools, json, systemd...
If I change insert_system_path to append the dirs, not prepend them, the issue is gone:
diff --git a/salt/config/__init__.py b/salt/config/__init__.py
index 1a5068328e8..24ca6fec7cc 100644
--- a/salt/config/__init__.py
+++ b/salt/config/__init__.py
@@ -2309,7 +2309,7 @@ def insert_system_path(opts, paths):
path_options = {"path": path, "root_dir": opts["root_dir"]}
prepend_root_dir(path_options, path_options)
if os.path.isdir(path_options["path"]) and path_options["path"] not in sys.path:
- sys.path.insert(0, path_options["path"])
+ sys.path.append(path_options["path"])
def minion_config(
Brief source inspection says things that really want to use modules from salt.utils, use them by full name, for example:
salt/utils/gitfs.py:import salt.utils.configparser
salt/utils/gitfs.py: conf = salt.utils.configparser.GitConfigParser()
salt/utils/gitfs.py: except salt.utils.configparser.NoSectionError:
salt/utils/gitfs.py: except salt.utils.configparser.NoOptionError:
salt/utils/gitfs.py: except salt.utils.configparser.NoSectionError:
salt/utils/gitfs.py: except salt.utils.configparser.NoOptionError:
Just in case, I'm attaching full log of a salt-call --local done by salt-ssh internally (out of stuff left on the target in /var/tmp), with full logging enabled:
Type of salt install
Official rpm
Major version
3007.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
fedora-42
salt --versions-report output
Salt Version:
Salt: 3007.13
Python Version:
Python: 3.10.19 (main, Feb 5 2026, 07:05:38) [GCC 11.2.0]
Dependency Versions:
cffi: 2.0.0
cherrypy: unknown
cryptography: 42.0.5
dateutil: 2.8.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.6
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 24.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.22.3
smmap: Not Installed
timelib: 0.3.0
Tornado: 6.5.4
ZMQ: 4.3.4
Salt Package Information:
Package Type: onedir
System Versions:
dist: fedora 42
locale: utf-8
machine: x86_64
release: 6.12.64-1.qubes.3.fc41.x86_64
system: Linux
version: Fedora Linux 42
What happened?
When using salt-ssh,
pkg.installedstate cannot be found. After a bit of debugging, I found this:The test call was:
Note yumpkg tries to import
configparserfrom Python's stdlib, but got one fromsalt.utilsinstead. Thesalt.utils.configparserseems to be specific to gitfs, and even that fails for similar reason:This all seems to be caused by prepending
/var/tmp/.root_dd8a91_salt/running_data/var/cache/salt/minion/extmods/utilsinto sys.path, so modules there take precedence over Python's stdlib. Looking at the content of that directory, there are likely more potential conflicts:functools,itertools,json,systemd...If I change
insert_system_pathto append the dirs, not prepend them, the issue is gone:Brief source inspection says things that really want to use modules from
salt.utils, use them by full name, for example:Just in case, I'm attaching full log of a
salt-call --localdone by salt-ssh internally (out of stuff left on the target in /var/tmp), with full logging enabled:Type of salt install
Official rpm
Major version
3007.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
fedora-42
salt --versions-report output
Salt Version: Salt: 3007.13 Python Version: Python: 3.10.19 (main, Feb 5 2026, 07:05:38) [GCC 11.2.0] Dependency Versions: cffi: 2.0.0 cherrypy: unknown cryptography: 42.0.5 dateutil: 2.8.2 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.6 libgit2: Not Installed looseversion: 1.3.0 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.7 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 24.0 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: Not Installed python-gnupg: 0.5.2 PyYAML: 6.0.1 PyZMQ: 25.1.2 relenv: 0.22.3 smmap: Not Installed timelib: 0.3.0 Tornado: 6.5.4 ZMQ: 4.3.4 Salt Package Information: Package Type: onedir System Versions: dist: fedora 42 locale: utf-8 machine: x86_64 release: 6.12.64-1.qubes.3.fc41.x86_64 system: Linux version: Fedora Linux 42