You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: README.md
+41-43Lines changed: 41 additions & 43 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,11 +17,9 @@
17
17
- Quick Setup: Approximately 10-second initialization for swift deployment.
18
18
- Enhanced Web Framework: Incorporates drogon cpp to boost web service efficiency.
19
19
20
-
## Documentation
21
-
22
20
## About Nitro
23
21
24
-
Nitro is a light-weight integration layer (and soon to be inference engine) for cutting edge inference engine, make deployment of AI models easier than ever before!
22
+
Nitro is a high-efficiency C++ inference engine for edge computing, powering [Jan](https://jan.ai/). It is lightweight and embeddable, ideal for product integration.
25
23
26
24
The binary of nitro after zipped is only ~3mb in size with none to minimal dependencies (if you use a GPU need CUDA for example) make it desirable for any edge/server deployment 👍.
27
25
@@ -40,37 +38,57 @@ The binary of nitro after zipped is only ~3mb in size with none to minimal depen
40
38
41
39
## Quickstart
42
40
43
-
**Step 1: Download Nitro**
41
+
**Step 1: Install Nitro**
44
42
45
-
To use Nitro, download the released binaries from the release page below:
Double-click on Nitro to run it. After downloading your model, make sure it's saved to a specific path. Then, make an API call to load your model into Nitro.
64
+
```bash title="Run Nitro server"
65
+
nitro
66
+
```
60
67
68
+
**Step 4: Load model**
61
69
62
-
```zsh
63
-
curl -X POST 'http://localhost:3928/inferences/llamacpp/loadmodel' \
0 commit comments