How to user it

[GitHub-20200408]

Can you provide a detailed user manual? Thank you

[davidamacey]

Hey! Sorry for the delayed response - just pushed a complete README that should help!

Quick Setup:

1. Clone the repo
2. Create .env file: echo "HUGGING_TOKEN2=your_hf_token" > .env
   (grab token from https://huggingface.co/settings/tokens)
3. Edit docker-compose.yaml - uncomment only what your GPU can handle:
   • Single A6000 → GPT-OSS-20B
   • Dual A6000 → Either 120B model OR 20B + ComfyUI
4. Create data dirs (or adjust paths in compose file):

   sudo mkdir -p /mnt/nas/{hf_vllm_models,ollama_webui/webui,hf_comfyui_models}

5. Run: docker compose up -d
6. Access at http://localhost:8010

Everything's commented out by default so it won't crash if you don't have enough VRAM. Just uncomment what works for your hardware. Check the README for full details on VRAM requirements.

Let me know if you need help with your specific setup!