Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit d9b0856

Browse files
committed
update
1 parent 870fb47 commit d9b0856

File tree

1 file changed

+49
-15
lines changed

1 file changed

+49
-15
lines changed

docs/docs/hub/index.mdx

Lines changed: 49 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,55 @@ The DownloadSuccess event indicates that all items in the download task have bee
240240
}
241241
```
242242

243-
### List local-available models via CLI
243+
### Importing local-models
244+
245+
When clients have models that are not inside the Cortex data folder and wish to run them inside Cortex, they can import local models using either the CLI or the HTTP API.
246+
247+
#### via CLI
248+
249+
Use the following command to import a local model using the CLI:
250+
251+
```sh
252+
cortex models import --model_id my-tinyllama --model_path /Users/jamesnguyen/cortexcpp/models/cortex.so/tinyllama
253+
/1b-gguf/model.gguf
254+
```
255+
256+
Response:
257+
258+
```sh
259+
Successfully import model from '/Users/jamesnguyen/cortexcpp/models/cortex.so/tinyllama/1b-gguf/model.gguf' for modeID 'my-tinyllama'.
260+
```
261+
262+
#### via HTTP API
263+
264+
Use the following curl command to import a local model using the HTTP API:
265+
266+
```sh
267+
curl --request POST \
268+
--url http://127.0.0.1:39281/v1/models/import \
269+
--header 'Content-Type: application/json' \
270+
--data '{
271+
"model": "model-id",
272+
"modelPath": "absolute/path/to/gguf",
273+
"name": "model display name"
274+
}'
275+
```
276+
277+
### Aborting Download Task
278+
279+
Clients can abort a downloading task using the task ID. Below is a sample `curl` command to abort a download task:
280+
281+
```sh
282+
curl --location --request DELETE 'http://127.0.0.1:3928/models/pull' \
283+
--header 'Content-Type: application/json' \
284+
--data '{
285+
"taskId": "tinyllama:1b-gguf-q2-k"
286+
}'
287+
```
288+
289+
An event with type `DownloadStopped` will be emitted when the task is successfully aborted.
290+
291+
### Listing local-available models via CLI
244292

245293
You can list your ready-to-use models via CLI using `cortex models list` command.
246294

@@ -270,20 +318,6 @@ Options:
270318
-v,--version Display version
271319
```
272320

273-
### Aborting Download Task
274-
275-
Clients can abort a downloading task using the task ID. Below is a sample `curl` command to abort a download task:
276-
277-
```sh
278-
curl --location --request DELETE 'http://127.0.0.1:3928/models/pull' \
279-
--header 'Content-Type: application/json' \
280-
--data '{
281-
"taskId": "tinyllama:1b-gguf-q2-k"
282-
}'
283-
```
284-
285-
An event with type `DownloadStopped` will be emitted when the task is successfully aborted.
286-
287321
### Listing local-available models via HTTP API
288322

289323
This section describes how to list all models that are available locally on your system using the HTTP API. By making a GET request to the specified endpoint, you can retrieve a list of models along with their details, such as model ID, name, file paths, engine type, and version. This is useful for managing and verifying the models you have downloaded and are ready to use in your local environment.

0 commit comments

Comments
 (0)