GPU Cluster Setup

1. Login your GPU compute provider.

For this, it's assumed your working directory is either / or /workspace. Edit as needed.

2. Run the below:

GPU Instance with Root user (For ex: Runpod, Verda):

mkdir -p /workspace ; cd /workspace ; git clone https://github.com/keennay/gpu-cluster-setup.git ; mv gpu-cluster-setup scripts ; cd scripts

GPU Instance with Ubuntu user (For ex: Prime Intellect):

sudo mkdir -p /workspace ; sudo chown -R ubuntu:ubuntu /workspace ; cd /workspace ; git clone https://github.com/keennay/gpu-cluster-setup.git ; mv gpu-cluster-setup scripts ; cd scripts

Remaining:

./01_init.sh
./02_install_dependencies.sh
./03_install_python.sh
source ./04_setup_env.sh
./05_install_packages.sh

3.1 Template for Running Models (Also Downloads if doesn't exist in HuggingFace PATH):

serve_models/***.sh

3.2 Install any selection of open-weights models or input your desired repo:

./install_model.sh

Change Between Python Environments

source ./launch_env.sh