@@ -20,7 +20,7 @@ Cortex has an Local Installer that packages all required dependencies, so that n
2020 - [ Mac (Universal)] ( https://app.cortexcpp.com/download/latest/mac-universal-local )
2121 - [ Linux] ( https://app.cortexcpp.com/download/latest/linux-amd64-local )
2222
23- ## Start Cortex.cpp Processes and API Server
23+ ## Start Cortex.cpp API Server
2424This command starts the Cortex.cpp API server at ` localhost:39281 ` .
2525<Tabs >
2626 <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
@@ -35,17 +35,38 @@ This command starts the Cortex.cpp API server at `localhost:39281`.
3535 </TabItem >
3636</Tabs >
3737
38+ ## Pull a Model & Select Quantization
39+ This command allows users to download a model from these Model Hubs:
40+ - [ Cortex Built-in Models] ( https://cortex.so/models )
41+ - [ Hugging Face] ( https://huggingface.co ) (GGUF): ` cortex pull <author/ModelRepo> `
42+
43+ It displays available quantizations, recommends a default and downloads the desired quantization.
44+ <Tabs >
45+ <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
46+ ``` sh
47+ $ cortex pull llama3.2
48+ $ cortex pull bartowski/Meta-Llama-3.1-8B-Instruct-GGUF
49+ ```
50+ </TabItem >
51+ <TabItem value = " Windows" label = " Windows" >
52+ ``` sh
53+ $ cortex pull llama3.2
54+ $ cortex.exe pull bartowski/Meta-Llama-3.1-8B-Instruct-GGUF
55+ ```
56+ </TabItem >
57+ </Tabs >
58+
3859## Run a Model
3960This command downloads the default ` gguf ` model format from the [ Cortex Hub] ( https://huggingface.co/cortexso ) , starts the model, and chat with the model.
4061<Tabs >
4162 <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
4263 ``` sh
43- cortex run mistral
64+ cortex run llama3.2
4465 ```
4566 </TabItem >
4667 <TabItem value = " Windows" label = " Windows" >
4768 ``` sh
48- cortex.exe run mistral
69+ cortex.exe run llama3.2
4970 ```
5071 </TabItem >
5172</Tabs >
@@ -78,33 +99,49 @@ curl http://localhost:39281/v1/chat/completions \
7899 "top_p": 1
79100}'
80101```
102+ Refer to our [ API documentation] ( https://cortex.so/api-reference ) for more details.
103+
104+ ## Show the System State
105+ This command displays the running model and the hardware system status (RAM, Engine, VRAM, Uptime)
106+ <Tabs >
107+ <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
108+ ``` sh
109+ cortex ps
110+ ```
111+ </TabItem >
112+ <TabItem value = " Windows" label = " Windows" >
113+ ``` sh
114+ cortex.exe ps
115+ ```
116+ </TabItem >
117+ </Tabs >
81118
82119## Stop a Model
83120This command stops the running model.
84121<Tabs >
85122 <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
86123 ``` sh
87- cortex models stop mistral
124+ cortex models stop llama3.2
88125 ```
89126 </TabItem >
90127 <TabItem value = " Windows" label = " Windows" >
91128 ``` sh
92- cortex.exe models stop mistral
129+ cortex.exe models stop llama3.2
93130 ```
94131 </TabItem >
95132</Tabs >
96133
97- ## Show the System State
98- This command displays the running model and the hardware system status .
134+ ## Stop Cortex.cpp API Server
135+ This command starts the Cortex.cpp API server at ` localhost:39281 ` .
99136<Tabs >
100137 <TabItem value = " MacOs/Linux" label = " MacOs/Linux" >
101138 ``` sh
102- cortex ps
139+ cortex stop
103140 ```
104141 </TabItem >
105142 <TabItem value = " Windows" label = " Windows" >
106143 ``` sh
107- cortex.exe ps
144+ cortex.exe stop
108145 ```
109146 </TabItem >
110147</Tabs >
@@ -137,4 +174,3 @@ Now that Cortex.cpp is set up, here are the next steps to explore:
1371741 . Adjust the folder path and configuration using the [ ` .cortexrc ` ] ( /docs/basic-usage/cortexrc ) file.
1381752 . Explore the Cortex.cpp [ data folder] ( /docs/data-folder ) to understand how it stores data.
1391763 . Learn about the structure of the [ ` model.yaml ` ] ( /docs/model-yaml ) file in Cortex.cpp.
140- 4 . Integrate Cortex.cpp [ libraries] ( /docs/category/libraries ) seamlessly into your Python or JavaScript applications.
0 commit comments