Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Commit 38fd61d

Browse files
bfineranKSGulindependabot[bot]markurtzspacemanidol
authored
Sync release/0.10 with patches from main (#513)
* Update ViT Example + Fix Readme (#507) * Update: More straightforward ViT example and recipe * Bump numpy in /research/information_retrieval/doc2query (#503) Bumps [numpy](https://github.com/numpy/numpy) from 1.20.3 to 1.21.0. - [Release notes](https://github.com/numpy/numpy/releases) - [Changelog](https://github.com/numpy/numpy/blob/main/doc/HOWTO_RELEASE.rst.txt) - [Commits](numpy/numpy@v1.20.3...v1.21.0) --- updated-dependencies: - dependency-name: numpy dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mark Kurtz <mark@neuralmagic.com> * Bump pillow in /research/information_retrieval/doc2query (#502) Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.3.2 to 9.0.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@8.3.2...9.0.0) --- updated-dependencies: - dependency-name: pillow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mark Kurtz <mark@neuralmagic.com> * Bump nltk in /research/information_retrieval/doc2query (#501) Bumps [nltk](https://github.com/nltk/nltk) from 3.6.5 to 3.6.6. - [Release notes](https://github.com/nltk/nltk/releases) - [Changelog](https://github.com/nltk/nltk/blob/develop/ChangeLog) - [Commits](nltk/nltk@3.6.5...3.6.6) --- updated-dependencies: - dependency-name: nltk dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mark Kurtz <mark@neuralmagic.com> * Update trainer.py (#506) * Update trainer.py recipe saving fixes for ddp * fix style * simplifying Co-authored-by: Daniel Campos <dfcf93@hotmail.com> Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> * bugfix: pytorch train save by top-1 acc (#509) * [transformers] trainers to prioritize recipe num_epochs + 0.10 fixes (#505) * allow for underscores in transformers onnx export task names * split out checkpoint and current recipes, QA CSV data load fix * propagate chagnes to task-level trainers * remove broken epoch override * add check before apply * json to jsonl conversion tool for HF custom data ease of use * Install Sparseml-Transformers deps dyanmically on import (#511) * Update: install sparseml on setup (#512) * Update: install sparseml on setup * Fix: typo Co-authored-by: Konstantin Gulin <66528950+KSGulin@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mark Kurtz <mark@neuralmagic.com> Co-authored-by: spacemanidol <dcampos3@illinois.edu> Co-authored-by: Daniel Campos <dfcf93@hotmail.com>
1 parent 3444136 commit 38fd61d

File tree

16 files changed

+199
-75
lines changed

16 files changed

+199
-75
lines changed

‎integrations/pytorch/train.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -606,7 +606,12 @@ def train(
606606
val_metric = val_res.result_mean(target_metric).item()
607607

608608
if epoch >= train_args.save_best_after and (
609-
best_metric is None or val_metric <= best_metric
609+
best_metric is None
610+
or (
611+
val_metric <= best_metric
612+
if target_metric != "top1acc"
613+
else val_metric >= best_metric
614+
)
610615
):
611616
utils.save_model_training(
612617
model,

‎integrations/rwightman-timm/setup_integration.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@ git clone https://github.com/neuralmagic/pytorch-image-models.git
88
cd pytorch-image-models
99
git checkout release/0.10
1010
pip install -r requirements.txt
11+
pip install sparseml[torch]

‎research/information_retrieval/doc2query/requirements.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,9 @@ nbconvert==6.0.7
5555
nbformat==5.1.3
5656
nest-asyncio==1.5.1
5757
networkx==2.5.1
58-
nltk==3.6.5
58+
nltk==3.6.6
5959
notebook==6.4.1
60-
numpy==1.20.3
60+
numpy==1.21.0
6161
onnx==1.7.0
6262
onnxruntime==1.8.0
6363
packaging==20.9
@@ -68,7 +68,7 @@ pathtools==0.1.2
6868
pathy==0.5.2
6969
pexpect==4.8.0
7070
pickleshare==0.7.5
71-
Pillow==8.3.2
71+
Pillow==9.0.0
7272
preshed==3.0.5
7373
progressbar2==3.53.1
7474
prometheus-client==0.11.0

‎setup.py‎

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,6 @@
3030

3131
_PACKAGE_NAME = "sparseml" if is_release else "sparseml-nightly"
3232

33-
transformers_branch = "master" if not is_release else f"release/{version_major_minor}"
34-
transformers_requirement = (
35-
"transformers @ git+https://github.com/neuralmagic/transformers.git"
36-
f"@{transformers_branch}"
37-
)
38-
3933
_deps = [
4034
"jupyter>=1.0.0",
4135
"ipywidgets>=7.0.0",
@@ -69,13 +63,6 @@
6963
"tf2onnx>=1.0.0,<1.6",
7064
]
7165
_keras_deps = ["tensorflow~=2.2.0", "keras2onnx>=1.0.0"]
72-
_transformers_deps = [
73-
"torch>=1.1.0,<1.9.0",
74-
transformers_requirement,
75-
"datasets",
76-
"sklearn",
77-
"seqeval",
78-
] + _pytorch_deps[1:]
7966

8067
_dev_deps = [
8168
"beautifulsoup4==4.9.3",
@@ -122,7 +109,6 @@ def _setup_extras() -> Dict:
122109
"tf_v1": _tensorflow_v1_deps,
123110
"tf_v1_gpu": _tensorflow_v1_gpu_deps,
124111
"tf_keras": _keras_deps,
125-
"transformers": _transformers_deps,
126112
}
127113

128114

‎src/sparseml/transformers/__init__.py‎

Lines changed: 75 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -18,22 +18,87 @@
1818

1919
# flake8: noqa
2020

21+
import logging as _logging
22+
23+
2124
try:
2225
import transformers as _transformers
2326

24-
transformers_import_error = None
25-
except Exception as transformers_import_err:
26-
transformers_import_error = transformers_import_err
27+
_transformers_import_error = None
28+
except Exception as _transformers_import_err:
29+
_transformers_import_error = _transformers_import_err
2730

2831

29-
def _check_transformers_install():
30-
if transformers_import_error is None:
31-
return
32-
raise ImportError(
33-
"No installation of transformers found. It is recommended to use the "
34-
"sparseml fork of transformers which can be installed under "
35-
"sparseml[transformers] or git+https://github.com/neuralmagic/transformers.git"
32+
_LOGGER = _logging.getLogger(__name__)
33+
34+
35+
def _install_transformers_and_deps():
36+
37+
import pip as _pip
38+
import sparseml as _sparseml
39+
40+
transformers_branch = (
41+
"master"
42+
if not _sparseml.is_release
43+
else f"release/{_sparseml.version_major_minor}"
3644
)
45+
transformers_requirement = (
46+
"transformers @ git+https://github.com/neuralmagic/transformers.git"
47+
f"@{transformers_branch}"
48+
)
49+
50+
try:
51+
_pip.main(
52+
[
53+
"install",
54+
transformers_requirement,
55+
"datasets",
56+
"sklearn",
57+
"seqeval",
58+
]
59+
)
60+
61+
import transformers as _transformers
62+
63+
_LOGGER.info("sparseml-transformers and dependencies successfully installed")
64+
except Exception:
65+
raise ValueError(
66+
"Unable to install and import sparseml-transformers dependencies check "
67+
"that transformers is installed, if not, install via "
68+
"`pip install git+https://github.com/neuralmagic/transformers.git`"
69+
)
70+
71+
72+
def _check_transformers_install():
73+
if _transformers_import_error is not None:
74+
import os
75+
76+
if os.getenv("NM_NO_AUTOINSTALL_TRANSFORMERS", False):
77+
_LOGGER.warning(
78+
"Unable to import transformers, skipping auto installation "
79+
"due to NM_NO_AUTOINSTALL_TRANSFORMERS"
80+
)
81+
# skip any further checks
82+
return
83+
else:
84+
_LOGGER.info(
85+
"No installation of transformers found. Installing sparseml-transformers "
86+
"dependencies"
87+
)
88+
_install_transformers_and_deps()
89+
90+
# check NM fork installed with QATMatMul available
91+
try:
92+
import transformers as _transformers
93+
94+
_transformers.models.bert.modeling_bert.QATMatMul
95+
except Exception:
96+
_LOGGER.warning(
97+
"transformers.models.bert.modeling_bert.QATMatMul not availalbe. the"
98+
"neuralmagic fork of transformers may not be installed. it can be "
99+
"installed via "
100+
"`pip install git+https://github.com/neuralmagic/transformers.git`"
101+
)
37102

38103

39104
_check_transformers_install()

‎src/sparseml/transformers/train/language_modeling.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -615,7 +615,8 @@ def group_texts(examples):
615615
# Initialize our Trainer
616616
trainer = SparseMLMLMTrainer(
617617
model_args.model_name_or_path,
618-
[existing_recipe, new_recipe],
618+
new_recipe,
619+
checkpoint_recipes=[existing_recipe],
619620
teacher=teacher_model,
620621
model=model,
621622
args=training_args,

‎src/sparseml/transformers/train/question_answering.py‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -396,11 +396,14 @@ def main():
396396
if data_args.test_file is not None:
397397
data_files["test"] = data_args.test_file
398398
extension = data_args.test_file.split(".")[-1]
399+
400+
# unwrap data from json
401+
load_kwargs = {"field": "data"} if extension == "json" else {}
399402
datasets = load_dataset(
400403
extension,
401404
data_files=data_files,
402-
field="data",
403405
cache_dir=model_args.cache_dir,
406+
**load_kwargs,
404407
)
405408
# See more about loading any type of standard or custom dataset
406409
# (from files, python dict, pandas DataFrame, etc) at
@@ -742,7 +745,8 @@ def compute_metrics(p: EvalPrediction):
742745
# Initialize our Trainer
743746
trainer = SparseMLQATrainer(
744747
model_args.model_name_or_path,
745-
[existing_recipe, new_recipe],
748+
recipe=new_recipe,
749+
checkpoint_recipes=[existing_recipe],
746750
teacher=teacher_model,
747751
model=model,
748752
args=training_args,

‎src/sparseml/transformers/train/text_classification.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -624,7 +624,8 @@ def compute_metrics(p: EvalPrediction):
624624
# Initialize our Trainer
625625
trainer = SparseMLGLUETrainer(
626626
model_args.model_name_or_path,
627-
[existing_recipe, new_recipe],
627+
new_recipe,
628+
checkpoint_recipes=[existing_recipe],
628629
teacher=teacher_model,
629630
model=model,
630631
args=training_args,

‎src/sparseml/transformers/train/token_classification.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,8 @@ def compute_metrics(p):
556556
# Initialize our Trainer
557557
trainer = SparseMLNERTrainer(
558558
model_args.model_name_or_path,
559-
[existing_recipe, new_recipe],
559+
new_recipe,
560+
checkpoint_recipes=[existing_recipe],
560561
teacher=teacher_model,
561562
model=model,
562563
args=training_args,

‎src/sparseml/transformers/utils/export.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,12 +123,13 @@ def export_transformer_to_onnx(
123123
pipeline, it will look only for 'model.onnx'
124124
:return: path to the exported ONNX file
125125
"""
126-
if task.lower() not in _TASK_TO_CONSTRUCTOR:
126+
task = "-".join(task.lower().split("_"))
127+
if task not in _TASK_TO_CONSTRUCTOR:
127128
raise ValueError(
128129
f"task {task} unsupported for export_transformer_to_onnx. Supported "
129130
f"tasks include {list(_TASK_TO_CONSTRUCTOR.keys())}"
130131
)
131-
auto_model_constructor = _TASK_TO_CONSTRUCTOR[task.lower()]
132+
auto_model_constructor = _TASK_TO_CONSTRUCTOR[task]
132133

133134
if not os.path.isdir(model_path):
134135
raise ValueError(

0 commit comments

Comments
 (0)