This small scale project is intended to make it easy for complete beginners to run the latest DeepSeek R1 models on their local machines by deploying the container. While my initial idea was to toy around with the open source model, I felt it would be helpful to document and containerize my work so that it can be used for folks who just want to get hands-on with these models and gain insights. By no means is this elegant code, but it is sufficient to get started as we can keep making improvements along the way! Leave a star if you find this useful!
You'll need three things installed on your local machine:
- Docker
- Git
- Python/any other tool you'd like to use for the POST requests
Clone the repo to a desired location, change the directory.
git clone https://github.com/SudarshanHV/dockerized_deepseek.git
cd dockerized-deepseekRun these commands to build a docker image and run the container respectively.
docker build -t sample_img .
docker run -p 5000:5000 sample_imgEnter your prompt in this requestHandler python scrit and run it to request the model. (Basically a POST request using the format mentioned in the script)
python3 requestHandler.pyNOTE: Depending on the hardware you are using, you may need to meddle with the size of the model that you'd need to pull. You can change the model size in the 'entrypoint.sh' script and rebuild the images. Change the below line and replace 7b with 1.5b, 8b, 14b, 32b, 70b or 671b. Start with the lower numbers. Default is 7b.
ollama pull deepseek-r1:7bThis is a list of improvements (not exhaustive) that I'd like to work on if I get some time. Feel free to contribute if you'd like to! Would greatly appreciate it!
- Definitely need an option to input the model size to the docker command. We dont want to run the 671b model or a 7b model by default.
- If you are a good observer, you'd have noticed that the container pulls the DeepSeek model every single time. I have attempted, but not yet found an effective way to pull and store the model inside an image. The bottleneck is that docker runs one process at a time, and we need a separate ollama process running to pull that image.
- Would love to add more customizations to input hardware specs. Currently, I believe it runs on whatever default hardware it finds. It'd be great if we could somehow specify a GPU/CPU to run this.
- Improve the docker commands and storage part a little bit.
- A user friendly UI for the prompts.
Adding this section as it's useful to know why this project's being made: to explore the DeepSeek models. In specific, the DeepSeek R1 model is what we are interested in (which itself is built on top of the DeepSeek v3 open source distilled model).
The models themselves can be found here at Ollama: https://ollama.com/library/deepseek-r1
There are 7 checkpoints (1.5b, 7b, 8b, 14b, 32b, 70b, 671b)
The DeepSeek research paper that elaborates further on the techniques used, including distillation and Reinforcement Learning: https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf
The DeepSeek repository: https://github.com/deepseek-ai/DeepSeek-R1
HuggingFace: https://huggingface.co/deepseek-ai/DeepSeek-R1
| DeepSeek Model | Distilled From | Memory Needed | Comparable ChatGPT Model |
|---|---|---|---|
| deepseek-r1:1.5b | Qwen 1.5 | 1.1GB | |
| deepseek-r1:7b | Qwen 7b | 4.7GB | ChatGPT-4o |
| deepseek-r1:8b | Llama 8b | 4.9GB | ChatGPT-4o |
| deepseek-r1:14b | Qwen 14b | 9.0GB | |
| deepseek-r1:32b | Qwen 32b | 19GB | ChatGPT o1-mini |
| deepseek-r1:70b | Llama 70b | 42GB | ChatGPT o1-mini |
| deepseek-r1:671b | Main Model | 414GB | ChatGPT-o1 |