Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit e9eaf6b

Browse files
committed
add(grammar): add documentation
1 parent 5fc33d4 commit e9eaf6b

File tree

1 file changed

+29
-1
lines changed

1 file changed

+29
-1
lines changed

docs/docs/features/grammar.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,32 @@
22
title: GBNF Grammar
33
description: What Nitro supports
44
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
5-
---
5+
---
6+
7+
## GBNF Grammar
8+
9+
GBNF (GGML BNF) makes it easy to set rules for how a model talks or writes. Think of it like teaching the model to always speak correctly, whether it's in emoji or proper JSON format.
10+
11+
Bakus-Naur Form (BNF) is a way to describe the rules of computer languages, files, and how they talk to each other. GBNF builds on BNF, adding modern features similar to those found in regular expressions.
12+
13+
In GBNF, we create rules (production rules) to guide how a model forms its responses. These rules use a mix of fixed characters (like letters or emojis) and flexible parts that can change. Each rule follows a format: `nonterminal ::= sequence...`.
14+
15+
To get a clearer picture, check out [this guide](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md).
16+
17+
## Use GBNF Grammar in Nitro
18+
19+
To make your Nitro model follow specific speaking or writing rules, use this command:
20+
21+
```bash title="Nitro Inference With Grammar" {9}
22+
curl http://localhost:3928/v1/chat/completions \
23+
-H "Content-Type: application/json" \
24+
-d '{
25+
"messages": [
26+
{
27+
"role": "user",
28+
"content": "Who won the world series in 2020?"
29+
},
30+
],
31+
"grammar_file": "/path/to/grammarfile"
32+
}'
33+
```

0 commit comments

Comments
 (0)