Skip to content

Support time-based LPs #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aerdem4 merged 6 commits into from
Jul 8, 2025
Merged

Support time-based LPs #27

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
db9d973
max-time + trigger with time
alessiodevoto Jul 3, 2025
d8743c2
flake8 fix
alessiodevoto Jul 3, 2025
dece75f
update with suggestions
alessiodevoto Jul 8, 2025
b65c49c
fix style
alessiodevoto Jul 8, 2025
cf0c005
fix notebooks
alessiodevoto Jul 8, 2025
61de0ac
fix notebooks
alessiodevoto Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
322 changes: 322 additions & 0 deletions examples/transformers/max_time_logits_processor.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,322 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "28ed6952",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/data/projects/logproc_ws/logits-processor-zoo\n"
]
}
],
"source": [
"%cd ../.."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "0ea01217",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/data/envs/logproc/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"from examples.transformers.utils import LLMRunner\n",
"from logits_processor_zoo.transformers import MaxTimeLogitProcessor\n",
"\n",
"\n",
"example_prompts = [\n",
"\"\"\"\n",
"A farmer has a rectangular field. The length of the field is 20 meters longer than its width. \n",
"If the perimeter of the field is 200 meters, find the dimensions of the field.\n",
"\"\"\",\n",
"]\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5627e226",
"metadata": {},
"outputs": [],
"source": [
"runner = LLMRunner(\"deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B\")"
]
},
{
"cell_type": "markdown",
"id": "859aef8d",
"metadata": {},
"source": [
"## Default Response"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "cbf4c2d5",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n",
"The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Prompt: \n",
"A farmer has a rectangular field. The length of the field is 20 meters longer than its width. \n",
"If the perimeter of the field is 200 meters, find the dimensions of the field.\n",
"\n",
"\n",
"LLM response:\n",
"First, I'll define the width of the field as \\( w \\) meters. Since the length is 20 meters longer than the width, the length will be \\( w + 20 \\) meters.\n",
"\n",
"Next, I'll use the formula for the perimeter of a rectangle, which is \\( 2 \\times (\\text{length} + \\text{width}) \\). Plugging in the expressions for length and width, the equation becomes:\n",
"\\[\n",
"2(w + (w + 20)) = 200\n",
"\\]\n",
"\n",
"Simplifying the equation:\n",
"\\[\n",
"2(2w + 20) = 200 \\\\\n",
"4w + 40 = 200 \\\\\n",
"4w = 160 \\\\\n",
"w = 40\n",
"\\]\n",
"\n",
"Finally, the width is 40 meters, and the length is \\( 40 + 20 = 60 \\) meters.\n",
"</think>\n",
"\n",
"Let's solve the problem step by step.\n",
"\n",
"**Given:**\n",
"- The field is rectangular.\n",
"- The length is \\( 20 \\) meters longer than the width.\n",
"- The perimeter of the field is \\( 200 \\) meters.\n",
"\n",
"**Let:**\n",
"- \\( w \\) = width of the field (in meters)\n",
"- \\( l \\) = length of the field (in meters)\n",
"\n",
"**Step 1: Express the Length in Terms of the Width**\n",
"\n",
"Since the length is \\( 20 \\) meters longer than the width:\n",
"\\[\n",
"l = w + 20\n",
"\\]\n",
"\n",
"**Step 2: Use the Perimeter Formula**\n",
"\n",
"The perimeter \\( P \\) of a rectangle is given by:\n",
"\\[\n",
"P = 2l + 2w\n",
"\\]\n",
"Given that the perimeter is \\( 200 \\) meters:\n",
"\\[\n",
"2l + 2w = 200\n",
"\\]\n",
"\n",
"**Step 3: Substitute the Expression for \\( l \\) into the Perimeter Equation**\n",
"\n",
"\\[\n",
"2(w + 20) + 2w = 200\n",
"\\]\n",
"\n",
"**Step 4: Simplify and Solve for \\( w \\)**\n",
"\n",
"\\[\n",
"2w + 40 + 2w = 200 \\\\\n",
"4w + 40 = 200 \\\\\n",
"4w = 200 - 40 \\\\\n",
"4w = 160 \\\\\n",
"w = \\frac{160}{4} \\\\\n",
"w = 40 \\text{ meters}\n",
"\\]\n",
"\n",
"**Step 5: Find the Length \\( l \\)**\n",
"\n",
"\\[\n",
"l = w + 20 = 40 + 20 = 60 \\text{ meters}\n",
"\\]\n",
"\n",
"**Final Answer:**\n",
"\\[\n",
"\\boxed{\\text{Width: } 40 \\text{ meters}, \\text{ Length: } 60 \\text{ meters}}\n",
"\\]\n",
"-----END-----\n",
"\n"
]
}
],
"source": [
"runner.generate_response(example_prompts, max_tokens=4096) "
]
},
{
"cell_type": "markdown",
"id": "88bc2f8a",
"metadata": {},
"source": [
"## Interrupt after N seconds"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "7d74eb26",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Prompt: \n",
"A farmer has a rectangular field. The length of the field is 20 meters longer than its width. \n",
"If the perimeter of the field is 200 meters, find the dimensions of the field.\n",
"\n",
"\n",
"LLM response:\n",
"First, I'll define the width of the field as \\( w \\) meters. Since the length is 20 meters longer than the width, the length will be \\( w + 20 \\) meters.\n",
"\n",
"Next, I'll use the formula for the perimeter of a rectangle, which is \\( 2 \\times (\\text{\n",
"-----END-----\n",
"\n"
]
}
],
"source": [
"runner.generate_response(example_prompts,\n",
" [MaxTimeLogitProcessor(runner.tokenizer, \n",
" boost_factor=0.1, \n",
" p=2, \n",
" complete_sentences=True, \n",
" max_time=1)],\n",
" max_tokens=4096)"
]
},
{
"cell_type": "markdown",
"id": "f3da686c",
"metadata": {},
"source": [
"### Combine with TriggerPhrase"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f1e982c3",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
"Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Prompt: \n",
"A farmer has a rectangular field. The length of the field is 20 meters longer than its width. \n",
"If the perimeter of the field is 200 meters, find the dimensions of the field.\n",
"\n",
"\n",
"LLM response:\n",
"First, I'll define the width of the field as \\( w \\) meters. Since the length is 20 meters longer than the width, the length will be \\( w + 20 \\) meters.\n",
"\n",
"Next, I'll use the formula for the perimeter of a rectangle, which is \\( 2 \\times (\\text{length} + \\text{width}) \\). Plugging in the expressions for length and width, the equation becomes:\n",
"\\[\n",
"2(w + (w + 20)) = 200\n",
"\\]\n",
"\n",
"Simplifying the equation:\n",
"\\[\n",
"2(2w + 20) = 200 \\\\\n",
"4w + 40 = 200 \\\\\n",
"4w = 160 \\\\\n",
"w = 40\n",
"\\]\n",
"\n",
"Finally<Interruption> The time is over. The final answer is: width is 40 meters and length is 60 meters.\n",
"</think>\n",
"-----END-----\n",
"\n"
]
}
],
"source": [
"from logits_processor_zoo.transformers import TriggerPhraseLogitsProcessor\n",
"\n",
"hurry_up = TriggerPhraseLogitsProcessor(runner.tokenizer,\n",
" batch_size=1, \n",
" phrase=\"<Interruption> The time is over. The final answer is:\", \n",
" trigger_time=2.5, \n",
" trigger_count=1, \n",
" trigger_after=True)\n",
"max_time = MaxTimeLogitProcessor(runner.tokenizer,\n",
" boost_factor=0.1,\n",
" p=2,\n",
" complete_sentences=True,\n",
" max_time=3)\n",
"\n",
"runner.generate_response(example_prompts,\n",
" [hurry_up, max_time],\n",
" max_tokens=4096)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "logproc",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading