This repository was archived by the owner on Jul 4, 2025. It is now read-only.
File tree Expand file tree Collapse file tree 1 file changed +1
-15
lines changed
Expand file tree Collapse file tree 1 file changed +1
-15
lines changed Original file line number Diff line number Diff line change @@ -92,21 +92,6 @@ curl http://localhost:3928/v1/chat/completions \
9292 }'
9393```
9494
95- *** OPTIONAL*** : You can constrain the sampling using GBNF grammars by providing path to a grammar file
96- ``` bash title="Nitro Inference With Grammar"
97- curl http://localhost:3928/v1/chat/completions \
98- -H " Content-Type: application/json" \
99- -d ' {
100- "messages": [
101- {
102- "role": "user",
103- "content": "Who won the world series in 2020?"
104- },
105- ],
106- "grammar_file": "/path/to/grammarfile"
107- }'
108- ```
109-
11095Table of parameters
11196
11297| Parameter | Type | Description |
@@ -128,6 +113,7 @@ Table of parameters
128113| ` grp_attn_n ` | Integer| Group attention factor in self-extend|
129114| ` grp_attn_w ` | Integer| Group attention width in self-extend|
130115| ` mlock ` | Boolean| Prevent system swapping of the model to disk in macOS|
116+ | ` grammar_file ` | String | You can constrain the sampling using GBNF grammars by providing path to a grammar file|
131117
132118*** OPTIONAL*** : You can run Nitro on a different port like 5000 instead of 3928 by running it manually in terminal
133119``` zsh
You can’t perform that action at this time.
0 commit comments