cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

[aloshbau]

Issue:

Error on from abodybuilder3.language.model import ProtT5

(base) root@:/# conda activate abodybuilder3
(abodybuilder3) root@:/# python
Python 3.9.23 | packaged by conda-forge | (main, Jun  4 2025, 17:57:12)
[GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from abodybuilder3.utils import string_to_input, output_to_pdb, add_atom37_to_output
/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/lightning_utilities/core/imports.py:14: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
>>> from abodybuilder3.lightning_module import LitABB3
>>> from abodybuilder3.language.model import ProtT5
Traceback (most recent call last):
  File "/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1382, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/opt/conda/envs/abodybuilder3/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/transformers/generation/utils.py", line 85, in <module>
    from accelerate.hooks import AlignDevicesHook, add_hook_to_module
  File "/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/accelerate/accelerator.py", line 34, in <module>
    from huggingface_hub import split_torch_state_dict_into_shards
ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (/opt/conda/envs/abodybuilder3/lib/python3.9/site-packages/huggingface_hub/__init__.py)

Resolution:

Update pinned-versions.txt huggingface-hub from 0.20.1 to 0.23.0
Import from abodybuilder3.language.model import ProtT5 now works

$ git diff pinned-versions.txt

-huggingface-hub==0.20.1
+huggingface-hub==0.23.0

Thanks to https://stackoverflow.com/questions/78920095/cannot-import-name-split-torch-state-dict-into-shards-from-huggingface-hub for the solution