We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 30932dc commit 753a255Copy full SHA for 753a255
codeflash/code_utils/code_utils.py
@@ -10,6 +10,7 @@
10
11
from codeflash.cli_cmds.console import logger
12
13
+
14
def encoded_tokens_len(s: str) -> int:
15
'''Function for returning the approximate length of the encoded tokens
16
It's an approximation of BPE encoding (https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)'''
0 commit comments