Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit e61191b

Browse files
louis-janlouis-jan
committed
chore: update README
1 parent 7802290 commit e61191b

File tree

2 files changed

+251
-91
lines changed

2 files changed

+251
-91
lines changed

‎README.md‎

Lines changed: 125 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -70,64 +70,150 @@ This monorepo contains two projects: CortexJS and CortexCPP.
7070
└── ...
7171
```
7272

73-
## Installation
73+
# Install
74+
## Prerequisites
7475

75-
### NPM Install
76+
### **Dependencies**
77+
78+
Before installation, ensure that you have installed the following:
79+
80+
- **Node.js**: Required for running the installation.
81+
- **NPM**: Needed to manage packages.
82+
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.
83+
84+
<aside>
85+
💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.
86+
87+
</aside>
88+
89+
### **Hardware**
90+
91+
Ensure that your system meets the following requirements to run Cortex:
92+
93+
- **OS**:
94+
- MacOSX 13.6 or higher.
95+
- Windows 10 or higher.
96+
- Ubuntu 12.04 and later.
97+
- **RAM (CPU Mode):**
98+
- 8GB for running up to 3B models.
99+
- 16GB for running up to 7B models.
100+
- 32GB for running up to 13B models.
101+
- **VRAM (GPU Mode):**
102+
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
103+
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
104+
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
105+
106+
- **Disk**: At least 10GB for app and model download.
107+
108+
## Cortex Installation
109+
110+
To install Cortex, follow the steps below:
111+
112+
### Step 1: Install Cortex
113+
114+
Run the following command to install Cortex globally on your machine:
115+
116+
```bash
117+
# Install using NPM globally
118+
npm i -g @janhq/cortex
119+
```
120+
121+
### Step 2: Verify the Installation
122+
123+
After installation, you can verify that Cortex is installed correctly by getting help information.
124+
125+
```bash
126+
# Get the help information
127+
cortex -h
128+
```
129+
130+
### Step 3: Initialize Cortex
131+
132+
Once verified, you need to initialize the Cortex engine.
133+
134+
1. Initialize the Cortex engine:
135+
136+
```
137+
cortex init
138+
```
139+
140+
1. Select between `CPU` and `GPU` modes.
141+
142+
```bash
143+
? Select run mode (Use arrow keys)
144+
> CPU
145+
GPU
146+
```
147+
148+
2. Select between GPU types.
76149

77-
* Pre-install script:
78150
```bash
79-
npm pre-install script; platform specific (MacOS / Windows / Linux)
151+
? Select GPU types (Use arrow keys)
152+
> Nvidia
153+
Others (Vulkan)
80154
```
81-
* Tag based:
82-
```json
83-
npm install @janhq/cortex
84-
npm install @janhq/cortex#cuda
85-
npm install @janhq/cortex#cuda-avx512
86-
npm install @janhq/cortex#cuda-avx
155+
3. Select CPU instructions (will be deprecated soon).
156+
157+
```bash
158+
? Select CPU instructions (Use arrow keys)
159+
> AVX2
160+
AVX
161+
AVX-512
87162
```
88163

89-
### CLI Install Script
164+
1. Cortex will download the required CPU instruction sets if you choose `CPU` mode. If you choose `GPU` mode, Cortex will download the necessary dependencies to use your GPU.
165+
2. Once downloaded, Cortex is ready to use!
166+
167+
### Step 4: Pull a model
168+
From HuggingFace
169+
```bash
170+
cortex pull janhq/phi-3-medium-128k-instruct-GGUF
171+
```
90172

173+
From Jan Hub (TBD)
91174
```bash
92-
cortex init (AVX2 + Cuda)
175+
cortex pull llama3
176+
```
93177

94-
Enable GPU Acceleration?
95-
1. Nvidia (default) - detected
96-
2. AMD
97-
3. Mac Metal
178+
### Step 5: Chat
179+
```bash
180+
cortex run janhq/phi-3-medium-128k-instruct-GGUF
181+
```
98182

99-
Enter your choice:
183+
## Run as an API server
184+
```bash
185+
cortex serve
186+
```
100187

101-
CPU Instructions
102-
1. AVX2 (default) - Recommend based on what the user has
103-
2. AVX (old CPU)
104-
3. AVX512
188+
## Build from Source
105189

106-
Enter your choice:
190+
To install Cortex from the source, follow the steps below:
107191

108-
Downloading cortex-cuda-avx.so........................25%
192+
1. Clone the Cortex repository [here](https://github.com/janhq/cortex/tree/dev).
193+
2. Navigate to the `cortex-js` folder.
194+
3. Open the terminal and run the following command to build the Cortex project:
109195

110-
Cortex is ready!
196+
```bash
197+
npx nest build
198+
```
111199

112-
It seems like you have installed models from other applications. Do you want to import them?
113-
1. Import from /Users/HOME/jan/models
114-
2. Import from /Users/HOME/lmstudio/models
115-
3. Import everything
200+
1. Make the `command.js` executable:
116201

117-
Importing from /Users/HOME/jan/models..................17%
202+
```bash
203+
chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
118204
```
119205

120-
## Backend (jan app)
206+
1. Link the package globally:
121207

122-
```json
123-
POST /settings
124-
{
125-
"gpu_enabled": true,
126-
"gpu_family": "Nvidia",
127-
"cpu_instructions": "AVX2"
128-
}
208+
```bash
209+
npm link
129210
```
130211

131-
## Client Library Configuration
212+
## Uninstall Cortex
132213

133-
TBD
214+
Run the following command to uninstall Cortex globally on your machine:
215+
216+
```
217+
# Uninstall globally using NPM
218+
npm uninstall -g @janhq/cortex
219+
```

‎cortex-js/README.md‎

Lines changed: 126 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,147 @@
1-
<p align="center">
2-
<a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="200" alt="Nest Logo" /></a>
3-
</p>
4-
5-
[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456
6-
[circleci-url]: https://circleci.com/gh/nestjs/nest
7-
8-
<p align="center">A progressive <a href="http://nodejs.org" target="_blank">Node.js</a> framework for building efficient and scalable server-side applications.</p>
9-
<p align="center">
10-
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/v/@nestjs/core.svg" alt="NPM Version" /></a>
11-
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/l/@nestjs/core.svg" alt="Package License" /></a>
12-
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/dm/@nestjs/common.svg" alt="NPM Downloads" /></a>
13-
<a href="https://circleci.com/gh/nestjs/nest" target="_blank"><img src="https://img.shields.io/circleci/build/github/nestjs/nest/master" alt="CircleCI" /></a>
14-
<a href="https://coveralls.io/github/nestjs/nest?branch=master" target="_blank"><img src="https://coveralls.io/repos/github/nestjs/nest/badge.svg?branch=master#9" alt="Coverage" /></a>
15-
<a href="https://discord.gg/G7Qnnhy" target="_blank"><img src="https://img.shields.io/badge/discord-online-brightgreen.svg" alt="Discord"/></a>
16-
<a href="https://opencollective.com/nest#backer" target="_blank"><img src="https://opencollective.com/nest/backers/badge.svg" alt="Backers on Open Collective" /></a>
17-
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://opencollective.com/nest/sponsors/badge.svg" alt="Sponsors on Open Collective" /></a>
18-
<a href="https://paypal.me/kamilmysliwiec" target="_blank"><img src="https://img.shields.io/badge/Donate-PayPal-ff3f59.svg"/></a>
19-
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://img.shields.io/badge/Support%20us-Open%20Collective-41B883.svg" alt="Support us"></a>
20-
<a href="https://twitter.com/nestframework" target="_blank"><img src="https://img.shields.io/twitter/follow/nestframework.svg?style=social&label=Follow"></a>
21-
</p>
22-
<!--[![Backers on Open Collective](https://opencollective.com/nest/backers/badge.svg)](https://opencollective.com/nest#backer)
23-
[![Sponsors on Open Collective](https://opencollective.com/nest/sponsors/badge.svg)](https://opencollective.com/nest#sponsor)-->
24-
25-
## Description
26-
27-
[Nest](https://github.com/nestjs/nest) framework TypeScript starter repository.
28-
29-
## Installation
1+
# Install
2+
## Prerequisites
3+
4+
### **Dependencies**
5+
6+
Before installation, ensure that you have installed the following:
7+
8+
- **Node.js**: Required for running the installation.
9+
- **NPM**: Needed to manage packages.
10+
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.
11+
12+
<aside>
13+
💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.
14+
15+
</aside>
16+
17+
### **Hardware**
18+
19+
Ensure that your system meets the following requirements to run Cortex:
20+
21+
- **OS**:
22+
- MacOSX 13.6 or higher.
23+
- Windows 10 or higher.
24+
- Ubuntu 12.04 and later.
25+
- **RAM (CPU Mode):**
26+
- 8GB for running up to 3B models.
27+
- 16GB for running up to 7B models.
28+
- 32GB for running up to 13B models.
29+
- **VRAM (GPU Mode):**
30+
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
31+
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
32+
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
33+
34+
- **Disk**: At least 10GB for app and model download.
35+
36+
## Cortex Installation
37+
38+
To install Cortex, follow the steps below:
39+
40+
### Step 1: Install Cortex
41+
42+
Run the following command to install Cortex globally on your machine:
3043

3144
```bash
32-
$ yarn install
45+
# Install using NPM globally
46+
npm i -g @janhq/cortex
3347
```
3448

35-
## Running the app
49+
### Step 2: Verify the Installation
50+
51+
After installation, you can verify that Cortex is installed correctly by getting help information.
3652

3753
```bash
38-
# development
39-
$ yarn run start
54+
# Get the help information
55+
cortex -h
56+
```
57+
58+
### Step 3: Initialize Cortex
4059

41-
# watch mode
42-
$ yarn run start:dev
60+
Once verified, you need to initialize the Cortex engine.
4361

44-
# production mode
45-
$ yarn run start:prod
62+
1. Initialize the Cortex engine:
63+
64+
```
65+
cortex init
4666
```
4767

48-
## Test
68+
1. Select between `CPU` and `GPU` modes.
4969

5070
```bash
51-
# unit tests
52-
$ yarn run test
71+
? Select run mode (Use arrow keys)
72+
> CPU
73+
GPU
74+
```
5375

54-
# e2e tests
55-
$ yarn run test:e2e
76+
2. Select between GPU types.
5677

57-
# test coverage
58-
$ yarn run test:cov
78+
```bash
79+
? Select GPU types (Use arrow keys)
80+
> Nvidia
81+
Others (Vulkan)
5982
```
83+
3. Select CPU instructions (will be deprecated soon).
6084

61-
## Support
85+
```bash
86+
? Select CPU instructions (Use arrow keys)
87+
> AVX2
88+
AVX
89+
AVX-512
90+
```
6291

63-
Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support).
92+
1. Cortex will download the required CPU instruction sets if you choose `CPU` mode. If you choose `GPU` mode, Cortex will download the necessary dependencies to use your GPU.
93+
2. Once downloaded, Cortex is ready to use!
6494

65-
## Stay in touch
95+
### Step 4: Pull a model
96+
From HuggingFace
97+
```bash
98+
cortex pull janhq/phi-3-medium-128k-instruct-GGUF
99+
```
66100

67-
- Author - [Kamil Myśliwiec](https://kamilmysliwiec.com)
68-
- Website - [https://nestjs.com](https://nestjs.com/)
69-
- Twitter - [@nestframework](https://twitter.com/nestframework)
101+
From Jan Hub (TBD)
102+
```bash
103+
cortex pull llama3
104+
```
105+
106+
### Step 5: Chat
107+
```bash
108+
cortex run janhq/phi-3-medium-128k-instruct-GGUF
109+
```
110+
111+
## Run as an API server
112+
```bash
113+
cortex serve
114+
```
115+
116+
## Build from Source
117+
118+
To install Cortex from the source, follow the steps below:
119+
120+
1. Clone the Cortex repository [here](https://github.com/janhq/cortex/tree/dev).
121+
2. Navigate to the `cortex-js` folder.
122+
3. Open the terminal and run the following command to build the Cortex project:
123+
124+
```bash
125+
npx nest build
126+
```
127+
128+
1. Make the `command.js` executable:
129+
130+
```bash
131+
chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
132+
```
133+
134+
1. Link the package globally:
70135

71-
## License
136+
```bash
137+
npm link
138+
```
139+
140+
## Uninstall Cortex
72141

73-
Nest is [MIT licensed](LICENSE).
142+
Run the following command to uninstall Cortex globally on your machine:
143+
144+
```
145+
# Uninstall globally using NPM
146+
npm uninstall -g @janhq/cortex
147+
```

0 commit comments

Comments
 (0)