Skip to content

Commit 275c6fb

Browse files
authored
Enforce tokens without adding very large numbers to avoid floating po… (#16)
* Enforce tokens without adding very large numbers to avoid floating point issues Signed-off-by: aerdem4 <ahmeterd4@gmail.com> * Make vllm cite prompt example better Signed-off-by: aerdem4 <ahmeterd4@gmail.com> --------- Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
1 parent 5d65e36 commit 275c6fb

19 files changed

+417
-192
lines changed

example_notebooks/transformers/force_last_phrase_logits_processor.ipynb

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,19 @@
2323
"execution_count": 2,
2424
"id": "a85f8503",
2525
"metadata": {},
26-
"outputs": [],
26+
"outputs": [
27+
{
28+
"name": "stderr",
29+
"output_type": "stream",
30+
"text": [
31+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
32+
" warnings.warn(\n",
33+
"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
34+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
35+
" warnings.warn(\n"
36+
]
37+
}
38+
],
2739
"source": [
2840
"from example_notebooks.transformers.utils import LLMRunner\n",
2941
"from logits_processor_zoo.transformers import ForceLastPhraseLogitsProcessor\n",
@@ -63,7 +75,12 @@
6375
"name": "stderr",
6476
"output_type": "stream",
6577
"text": [
66-
"The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
78+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:392: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n",
79+
" warnings.warn(\n",
80+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:397: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.\n",
81+
" warnings.warn(\n",
82+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:407: UserWarning: `do_sample` is set to `False`. However, `top_k` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_k`.\n",
83+
" warnings.warn(\n"
6784
]
6885
},
6986
{
@@ -216,7 +233,7 @@
216233
"name": "python",
217234
"nbconvert_exporter": "python",
218235
"pygments_lexer": "ipython3",
219-
"version": "3.10.13"
236+
"version": "3.10.17"
220237
}
221238
},
222239
"nbformat": 4,

example_notebooks/transformers/gen_length_logits_processor.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -288,7 +288,7 @@
288288
"name": "python",
289289
"nbconvert_exporter": "python",
290290
"pygments_lexer": "ipython3",
291-
"version": "3.10.13"
291+
"version": "3.10.17"
292292
}
293293
},
294294
"nbformat": 4,

example_notebooks/transformers/multiple_choice_logits_processor.ipynb

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,19 @@
2323
"execution_count": 2,
2424
"id": "a85f8503",
2525
"metadata": {},
26-
"outputs": [],
26+
"outputs": [
27+
{
28+
"name": "stderr",
29+
"output_type": "stream",
30+
"text": [
31+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
32+
" warnings.warn(\n",
33+
"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
34+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
35+
" warnings.warn(\n"
36+
]
37+
}
38+
],
2739
"source": [
2840
"from example_notebooks.transformers.utils import LLMRunner\n",
2941
"from logits_processor_zoo.transformers import MultipleChoiceLogitsProcessor\n",
@@ -68,7 +80,12 @@
6880
"name": "stderr",
6981
"output_type": "stream",
7082
"text": [
71-
"The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
83+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:392: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n",
84+
" warnings.warn(\n",
85+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:397: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.\n",
86+
" warnings.warn(\n",
87+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:407: UserWarning: `do_sample` is set to `False`. However, `top_k` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_k`.\n",
88+
" warnings.warn(\n"
7289
]
7390
},
7491
{
@@ -237,7 +254,7 @@
237254
"name": "python",
238255
"nbconvert_exporter": "python",
239256
"pygments_lexer": "ipython3",
240-
"version": "3.10.13"
257+
"version": "3.10.17"
241258
}
242259
},
243260
"nbformat": 4,

example_notebooks/transformers/trigger_phrase_logits_processor.ipynb

Lines changed: 39 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,19 @@
2323
"execution_count": 2,
2424
"id": "b89279fe",
2525
"metadata": {},
26-
"outputs": [],
26+
"outputs": [
27+
{
28+
"name": "stderr",
29+
"output_type": "stream",
30+
"text": [
31+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
32+
" warnings.warn(\n",
33+
"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
34+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
35+
" warnings.warn(\n"
36+
]
37+
}
38+
],
2739
"source": [
2840
"from example_notebooks.transformers.utils import LLMRunner\n",
2941
"from logits_processor_zoo.transformers import TriggerPhraseLogitsProcessor, GenLengthLogitsProcessor\n",
@@ -58,9 +70,14 @@
5870
"name": "stderr",
5971
"output_type": "stream",
6072
"text": [
73+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:392: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n",
74+
" warnings.warn(\n",
75+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:397: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.\n",
76+
" warnings.warn(\n",
77+
"/home/aerdem/projects/LLM/llmenv/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:407: UserWarning: `do_sample` is set to `False`. However, `top_k` is set to `None` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_k`.\n",
78+
" warnings.warn(\n",
6179
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
62-
"Setting `pad_token_id` to `eos_token_id`:None for open-end generation.\n",
63-
"The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
80+
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
6481
]
6582
},
6683
{
@@ -96,9 +113,9 @@
96113
"\n",
97114
"Let me test this function with some examples. For n=0, it returns 0. For n=1, returns 1. For n=2, it's F(1)+F(0) = 1+0=1. For n=3, F(2)+F(1)=1+1=2. That looks correct.\n",
98115
"\n",
99-
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which is correct.\n",
116+
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which matches the standard definition. So, the function should work regardless of the starting point as long as the base cases are correct.\n",
100117
"\n",
101-
"Another test case: n=5. Let's compute it step by step. F(0)=0, F(1)=1, F(2)=1, F(3)=2, F(4)=3, F(5)=5. So the function should return 5 for n=5.\n",
118+
"Another thing to consider is the base cases. If the function is called with n=0, it returns 0, which is correct. For n=1, returns 1. For n=2, returns 1, which is correct. So, the function should handle all non-negative integers correctly.\n",
102119
"\n",
103120
"I think this should work. So, the function is straightforward. It's a simple recursive implementation, but it's not the most efficient for large n. However, for the purpose of this problem, it's acceptable.\n",
104121
"</think>\n",
@@ -162,7 +179,7 @@
162179
"output_type": "stream",
163180
"text": [
164181
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
165-
"Setting `pad_token_id` to `eos_token_id`:None for open-end generation.\n"
182+
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
166183
]
167184
},
168185
{
@@ -198,25 +215,27 @@
198215
"\n",
199216
"Let me test this function with some examples. For n=0, it returns 0. For n=1, returns 1. For n=2, it's F(1)+F(0) = 1+0=1. For n=3, F(2)+F(1)=1+1=2. That looks correct.\n",
200217
"\n",
201-
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which is correct.\n",
218+
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which matches the standard definition. So, the function should work regardless of the starting point as long as the base cases are correct.\n",
202219
"\n",
203-
"Another test case: n=5. Let's compute it step by step. F(0)=0, F(1)=1, F(2)=1, F(3)=2, F(4)=3, F(5)=5. So the function should return 5 for n=5.\n",
220+
"Another thing to consider is the base cases. If the function is called with n=0, it returns 0, which is correct. For n=1, returns 1. For n=2, returns 1, which is correct. So, the function should handle all non-negative integers correctly.\n",
204221
"\n",
205222
"I think this should work. So, the function is straightforward. It's a simple recursive implementation, but it's not the most efficient for large n. However, for the purpose of this problem, it's acceptable.\n",
206223
",,,\n",
207224
"\n",
208-
"Wait, but the problem says to make it recursive. So, the function as written is recursive. So, I think this should be the solution.\n",
225+
"Wait, but the problem says to make it recursive. So, the function should call itself with smaller arguments. The approach I have is correct and recursive.\n",
226+
"\n",
227+
"So, the final function is as I wrote above.\n",
209228
",,,\n",
210229
"\n",
211-
"Wait, but in the function I wrote, for n=0, it returns 0, which is correct. For n=1, returns 1. For n=2, returns 1, which is correct. So, the function should work as intended.\n",
230+
"Wait, but in the function, for n=2, it's F(1)+F(0) = 1+0=1, which is correct. For n=3, F(2)+F(1)=1+1=2. So, the function works as expected.\n",
212231
"\n",
213-
"I think that's a good approach. So, the function is as I wrote above.\n",
232+
"I think this should solve the problem.\n",
214233
"</think>\n",
215234
"\n",
216235
"To solve this problem, we need to generate the nth Fibonacci number using a recursive approach. The Fibonacci sequence is a series of numbers where each number is the sum of the two preceding ones, starting from 0 and 1. \n",
217236
"\n",
218237
"### Approach\n",
219-
"The approach to solve this problem involves using recursion, which is a method where the function calls itself with a smaller input until it reaches a base case. Here's a step-by-step breakdown of the approach:\n",
238+
"The approach to solve this problem involves using recursion, which is a method where a function calls itself with a modified parameter to achieve the desired result. Here's a step-by-step breakdown of the approach:\n",
220239
"\n",
221240
"1. **Base Cases**: \n",
222241
" - If `n` is 0, return 0.\n",
@@ -236,14 +255,14 @@
236255
" elif n == 1:\n",
237256
" return 1\n",
238257
" else:\n",
239-
" return fibonacci(n - 1) + fibonacci(n - 2)\n",
258+
" return fibonacci(n-1) + fibonacci(n-2)\n",
240259
"```\n",
241260
"\n",
242261
"### Explanation\n",
243262
"- **Base Cases**: The function first checks if `n` is 0 or 1. If `n` is 0, it returns 0. If `n` is 1, it returns 1. These are the simplest cases of the Fibonacci sequence.\n",
244263
"- **Recursive Case**: For any `n` greater than 1, the function calls itself with `n-1` and `n-2`, and returns the sum of these two recursive calls. This builds up the solution by solving smaller subproblems and combining their results.\n",
245264
"\n",
246-
"This approach is straightforward and easy to understand, but it's important to note that it is not the most efficient for large values of `n` due to the exponential time complexity. However, for the purpose of this problem, the recursive approach is sufficient.\n",
265+
"This approach is straightforward and leverages the divide-and-conquer strategy inherent in recursion, making it easy to understand and implement. However, it's important to note that this approach has a time complexity of O(2^n) due to the exponential number of function calls, which is not efficient for large values of `n`. For larger values, an iterative approach or memoization would be more efficient.\n",
247266
"-----END-----\n",
248267
"\n"
249268
]
@@ -277,7 +296,7 @@
277296
"output_type": "stream",
278297
"text": [
279298
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
280-
"Setting `pad_token_id` to `eos_token_id`:None for open-end generation.\n"
299+
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
281300
]
282301
},
283302
{
@@ -313,9 +332,9 @@
313332
"\n",
314333
"Let me test this function with some examples. For n=0, it returns 0. For n=1, returns 1. For n=2, it's F(1)+F(0) = 1+0=1. For n=3, F(2)+F(1)=1+1=2. That looks correct.\n",
315334
"\n",
316-
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which is correct.\n",
335+
"Wait, but sometimes people define the Fibonacci sequence starting with F(1)=1, F(2)=1, F(3)=2, etc. So, if the function is called with n=5, it should return 5. Let me see: F(5) is 5, which matches the standard definition. So, the function should work regardless of the starting point as long as the base cases are correct.\n",
317336
"\n",
318-
"Another test case: n=5. Let's compute it step by step. F(0)=0, F(1)=1, F(2)=1, F(3)=2, F(4)=3, F(5)=5. So the function should return 5 for n=5.\n",
337+
"Another thing to consider is the base cases. If the function is called with n=0, it returns 0, which is correct. For n=1, returns 1. For n=2, returns 1, which is correct. So, the function should handle all non-negative integers correctly.\n",
319338
"\n",
320339
"I think this should work. So, the function is straightforward. It's a simple recursive implementation, but it's not the most efficient for large n. However, for the purpose of this problem, it's acceptable.\n",
321340
"</think>\n",
@@ -329,7 +348,7 @@
329348
" return fibonacci(n-1) + fibonacci(n-2)\n",
330349
"```\n",
331350
"\n",
332-
"This function calculates the nth Fibonacci number using a recursive approach. It handles the base cases where n is 0 or 1 and for other values, it recursively calculates the sum of the two preceding Fibonacci numbers. While this implementation is straightforward, it's not the most efficient for large values of n due to repeated calculations.\n",
351+
"This function calculates the nth Fibonacci number using a recursive approach. It handles the base cases where n is 0 or 1 and recursively computes the value for larger n by summing the two preceding Fibonacci numbers.\n",
333352
"-----END-----\n",
334353
"\n"
335354
]
@@ -361,7 +380,7 @@
361380
"output_type": "stream",
362381
"text": [
363382
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
364-
"Setting `pad_token_id` to `eos_token_id`:None for open-end generation.\n"
383+
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
365384
]
366385
},
367386
{
@@ -429,7 +448,7 @@
429448
"name": "python",
430449
"nbconvert_exporter": "python",
431450
"pygments_lexer": "ipython3",
432-
"version": "3.10.13"
451+
"version": "3.10.17"
433452
}
434453
},
435454
"nbformat": 4,

0 commit comments

Comments
 (0)