Ally

Ally — lightweight, private service assistant for anyone

Ally is an AI-powered CLI tool designed to assist with anything from everyday tasks to complex projects efficiently, without leaving the terminal.

Ally was built a fully local agentic system using Ollama, but it also works seamlessly with:

• OpenAI
• Anthropic
• Google GenAI
• Groq
• Cerebras
• OpenRouter
• GitHub Models
• (more integrations on the way!)

This tool is best suited for scenarios where privacy is paramount and agentic capabilities are needed in the workflow.

Key Features

Default Chat Interface

A general-purpose agent that can:

• Read, write, modify, and delete files and directories.

• Access the internet.

• Execute commands and code.

• Execute shell commands directly using !<command> syntax.

  Note: Tools always ask for your permission before executing. Multiple tool calls are processed sequentially to ensure clear approval flow. Shell commands executed with ! run directly without requiring approval.

RAG

Ally can take your files, embed them into its knowledge base, and use them to respond to your prompts with a high level of accuracy.

Currently, Ally's embedding functions can use:

• Hugging Face models (locally)
• Ollama Embedding models (locally)
• NLP Cloud (hosted)
• OpenAI (hosted)
• More on the way...

RAG Tutorial:

1. Setup the config.json as shown below with the appropriate embedding settings.

2. Provide the path to the file or folder whose contents should be embedded. As an alternative, you can launch Ally from that directory.

3. Use /embed <path> <collection_name> or /embed . <collection_name> if already at the correct path.

4. Start the RAG session with /start_rag

5. End the RAG session with /stop_rag

Note that Ally will not use any external data to answer your prompts during RAG sessions unless explicitly given permission to.

Additional commands:

• Edit indexed collections with /index <collection_name> and /unindex <collection_name>. Note: newly created collections are already indexed.

• View all collections with /list

• Reset the database with /purge or delete a specific collection with /delete <collection_name>

Full Coding Project Generation Workflow (Preview version)

• Use the --create-project flag or the /project command in the default chat interface.

Complete workflow:

• Asks for your project idea.
• Runs the Brainstormer Agent to create the context space and full project specification for the Codegen Agent (in .md format).
• Optionally lets you provide more context by chatting interactively with the Brainstormer Agent.
• Runs the Codegen Agent using the generated .md files.
• Opens an interactive chat with the Codegen Agent to refine or extend the project.

This workflow is still in its early stages.

Setup

You have 2 options: Via Docker or locally on your machine.

1. Docker (Recommended)

Create a .env file (or copy .env.example) in any location

# Inference providers (only include those you need)

OPENAI_API_KEY=...
ANTHROPIC_API_KEY=...
GOOGLE_GEN_AI_API_KEY=...
CEREBRAS_API_KEY=...

# Embedding providers APIs (As needed. Only add those you need.)

NLP_CLOUD_API_KEY=...

# Google Search API (if omitted, online search tools will be limited)

GOOGLE_SEARCH_API_KEY=...
SEARCH_ENGINE_ID=...

See steps for the Google Search API part.

Open a terminal in that directory and type

# Pull the Ally image:
docker pull yassw0rks/ally:latest


# Start the container for the first time
docker run -it --env-file .env --name ally yassw0rks/ally:latest


# You could also assign a volume:
# Replace <YOUR_LOCAL_DIR> with a path on your machine.
# Note that Docker must have permission to <YOUR_LOCAL_DIR>. It can be configured from the settings.
docker run -it \
  --env-file .env \
  -v <YOUR_LOCAL_DIR>:/data \
  --name ally \
  yassw0rks/ally:latest

Next time you want to jump back in

# Check if container already running
docker ps

# If it is running
docker exec -it ally /bin/bash

# If it's stopped
docker start -ai ally

Edit the config.json file (see below) inside /app as needed. Nano is included in the image for your convenience.

Note this image does not contain Ollama. But it can easily be setup once inside the container.

2. Local

Prerequesites:

• Python
• UV
• Git (or download the source code from this repo)
• Ollama (optional)

1. Clone the Repo

In your chosen installation folder, open a terminal window and run:

git clone https://github.com/YassWorks/Ally.git

2. Configure config.json in /Ally

This file (located at Ally/) controls Ally's main settings and integrations.

Example configuration:

• Options for "provider" are "openai", "ollama", "anthropic", "google", "groq", "cerebras", "openrouter", and "github".

• The model name depends on your chosen provider (often found within the provider's dashboard or models tab).

• provider_per_model and models with null values: These fields will be automatically filled with the default provider and model specified above. For example, if you set "brainstormer": null, it will use the main "provider" and "model" values.

• system_prompts: Leave these as null to use Ally's built-in default prompts (recommended). You can override individual prompts by providing custom prompt text.

• embedding_provider: Options include "hf" (Hugging Face - runs locally) or "ollama" (for locally running embedding models).

• embedding_model: Examples: "sentence-transformers/all-MiniLM-L6-v2" (Hugging Face) or "all-minilm" (Ollama).

• scraping_method: "simple" is the only option for now. More powerful options coming in future versions.

{
    "provider": "openai",
    "provider_per_model": {
        "general": "ollama",
        "code_gen": "anthropic",
        "brainstormer": null,
        "web_searcher": null
    },

    "model": "gpt-4o",
    "models": {
        "general": "gpt-oss:20b",
        "code_gen": "claude-sonnet-3.5",
        "brainstormer": null,
        "web_searcher": null
    },

    "temperatures": {
        "general": 1,
        "code_gen": 1,
        "brainstormer": 1,
        "web_searcher": 1
    },
    "system_prompts": {
        "general": null,
        "code_gen": null,
        "brainstormer": null,
        "web_searcher": null
    },

    "embedding_provider": null,
    "embedding_model": null,

    "scraping_method": "simple"
}

Note: You could setup a volume for the embedding models between your machine and the container so that models are persisted across sessions. See below for information where the models are stored inside the container by default.

3. Configure .env in /Ally

This file stores your API keys.

# Inference providers (only include those you need)

OPENAI_API_KEY=...
ANTHROPIC_API_KEY=...
GOOGLE_GEN_AI_API_KEY=...
GROQ_API_KEY=...
CEREBRAS_API_KEY=...
OPENROUTER_API_KEY=...
GITHUB_AI_API_KEY=...

# Embedding providers APIs (As needed. Only add those you need.)

NLP_CLOUD_API_KEY=...

# Google Search API (if omitted, online search tools will be limited)

GOOGLE_SEARCH_API_KEY=...
SEARCH_ENGINE_ID=...

Programmable search engine steps:

1. Set up a Google Programmable Search Engine
2. Copy the contents above (or from .env.example) into .env.
3. Fill in your API keys and IDs.

4. Run setup executable

Depending on your OS choose either setup.cmd (Windows) or setup.sh (Linux/Mac)

Note: Ally creates its own virtual environment to keep dependencies isolated and automatically adds itself to your PATH.

Now you’re ready to run Ally from anywhere in the terminal using ally.

Use ally -h for more help.

Technical notes

1. Logging: Ally uses a centralized logging system that records detailed error information in log files. Error messages shown to users are kept concise for better readability, while full technical details (including stack traces) are logged for debugging purposes.

   • Log files are stored in:
     • Windows: %LOCALAPPDATA%\Ally\logs\
     • Linux/MacOS: ~/.local/share/Ally/logs/
   • Logs are organized by date: ally_YYYYMMDD.log
   • When reporting issues, please include relevant log file excerpts

2. Edit the following environment variable if needed:

Environment Variable	Purpose
ALLY_HISTORY_DIR	Controls where Ally stores its history.
ALLY_DATABASE_DIR	Controls where Ally stores its database.
ALLY_EMBEDDING_MODELS_DIR	Controls where Ally stores its embedding models (Hugging Face).

Defaults are:

Windows:
%LOCALAPPDATA%\Ally\...

Linux & MacOS:
~/.local/share/Ally/...

3. RAG-related tools used by Ally are large in size and are therefore downloaded only after RAG settings are enabled in the config.json file. As a result, Ally will perform additional downloads the next time it is launched following these configuration changes.

4. To save a chat, use /id to view the conversation ID. The next time you open Ally, continue the conversation by using the -i flag followed by the ID. You can do the same inside the CLI, just do /id <your_id>

5. Embedding and scraping files that require OCR (such as PDFs and DOCX) currently use a CPU-only PyTorch installation. You can modify the configuration to utilize a GPU if desired, though this is typically only necessary for processing very large files.

License

Apache-2.0

---

Issues and PRs are always welcome 💌

Contact me via email to discuss contributions or collaborations on other projects if you liked my work!