|
| 1 | +--- |
| 2 | +title: Hugging Face |
| 3 | +description: Cortex supports all `GGUF` and `ONNX` models available in Huggingface repositories, providing access to a wide range of models. |
| 4 | +--- |
| 5 | + |
| 6 | +import Tabs from "@theme/Tabs"; |
| 7 | +import TabItem from "@theme/TabItem"; |
| 8 | + |
| 9 | +Cortex.cpp supports all `GGUF` from the [Hugging Face Hub](https://huggingface.co). |
| 10 | + |
| 11 | +You can pull HuggingFace models via: |
| 12 | +- repository handle: eg `author/model_id` |
| 13 | +- direct url: eg `https://huggingface.co/QuantFactory/OpenMath2-Llama3.1-8B-GGUF/blob/main/OpenMath2-Llama3.1-8B.Q4_0.gguf` |
| 14 | + |
| 15 | + |
| 16 | +## GGUF |
| 17 | +To view all available `GGUF` models on HuggingFace, select the `GGUF` tag in the Libraries section. |
| 18 | + |
| 19 | + |
| 20 | +<Tabs> |
| 21 | + <TabItem value="MacOs/Linux" label="MacOs/Linux"> |
| 22 | + ```sh |
| 23 | + ## Pull the Codestral-22B-v0.1-GGUF model from the bartowski organization |
| 24 | + cortex pull bartowski/Codestral-22B-v0.1-GGUF |
| 25 | + |
| 26 | + # Pull the gemma-7b model from the google organization |
| 27 | + cortex pull https://huggingface.co/QuantFactory/OpenMath2-Llama3.1-8B-GGUF/blob/main/OpenMath2-Llama3.1-8B.Q4_0.gguf |
| 28 | + ``` |
| 29 | + </TabItem> |
| 30 | + <TabItem value="Windows" label="Windows"> |
| 31 | + ```sh |
| 32 | + ## Pull the Codestral-22B-v0.1-GGUF model from the bartowski organization |
| 33 | + cortex.exe pull bartowski/Codestral-22B-v0.1-GGUF |
| 34 | + |
| 35 | + # Pull the gemma-7b model from the google organization |
| 36 | + cortex.exe pull google/gemma-7b |
| 37 | + ``` |
| 38 | + </TabItem> |
| 39 | +</Tabs> |
| 40 | + |
| 41 | +<!-- ## ONNX |
| 42 | + |
| 43 | +To view all available `ONNX` models on HuggingFace, select the `ONNX` tag in the Libraries section. |
| 44 | +<Tabs> |
| 45 | + <TabItem value="MacOs/Linux" label="MacOs/Linux"> |
| 46 | + ```sh |
| 47 | + ## Pull the XLM-Roberta-Large-Vit-B-16Plus model from the immich-app organization |
| 48 | + cortex pull immich-app/XLM-Roberta-Large-Vit-B-16Plus |
| 49 | + |
| 50 | + # Pull the mt0-base model from the bigscience organization |
| 51 | + cortex pull bigscience/mt0-base |
| 52 | + ``` |
| 53 | + </TabItem> |
| 54 | + <TabItem value="Windows" label="Windows"> |
| 55 | + ```sh |
| 56 | + ## Pull the XLM-Roberta-Large-Vit-B-16Plus model from the immich-app organization |
| 57 | + cortex.exe pull immich-app/XLM-Roberta-Large-Vit-B-16Plus |
| 58 | + |
| 59 | + # Pull the mt0-base model from the bigscience organization |
| 60 | + cortex.exe pull bigscience/mt0-base |
| 61 | + ``` |
| 62 | + </TabItem> |
| 63 | +</Tabs> |
| 64 | + |
| 65 | +## TensorRT-LLM |
| 66 | +We are still working to support all available `TensorRT-LLM` models on HuggingFace. For now, Cortex.cpp only supports built-in `TensorRT-LLM` models, which can be downloaded from the [Cortex Model Repos](/docs/hub/cortex-hub). --> |
0 commit comments