Skip to content

Commit b91eb8e

Browse files
Alexandre LandeauAlexandre Landeau
authored andcommitted
changed default number of processes to prevent bug on MacOS and python 3.8
1 parent eee133d commit b91eb8e

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

python-lib/spacy_tokenizer.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,8 @@ class MultilingualTokenizer:
122122

123123
DEFAULT_BATCH_SIZE = 1000
124124
MAX_NUM_CHARACTERS = 10 ** 7
125-
DEFAULT_NUM_PROCESS = 2
125+
# Set to 1 to prevent pickling issues when spawning multiple processes on MacOS
126+
DEFAULT_NUM_PROCESS = 1
126127
DEFAULT_FILTER_TOKEN_ATTRIBUTES = {
127128
"is_space": "Whitespace",
128129
"is_punct": "Punctuation",

0 commit comments

Comments
 (0)