Skip to content

configuration.md

Latest commit

 

History

History
95 lines (74 loc) · 4.15 KB

configuration.md

File metadata and controls

95 lines (74 loc) · 4.15 KB
Raw

Configuration

FerryAI is configured once via AI::config(array $config). Values are read through FerryAI\Core\AIConfig (dot-notation get()), and most also have an environment fallback.

AI::config([
    'backend'  => 'onnx',
    'device'   => 'cpu',
    'backends' => [ /* per-task model paths */ ],
]);

Top-level keys

Key Default Meaning
backend auto Default task backend: onnx, llama, cpu/cpu_native, or auto (→ ONNX).
device auto cpu, cuda, or auto. Selects GPU offload where supported.
model_cache temp dir Where the model hub caches downloads (FERRY_AI_MODEL_CACHE).
max_tokens 2048 Default generation length.
temperature 0.7 Default sampling temperature (0 = greedy).
top_p 1.0 Default nucleus threshold.
stream_timeout 30 Max seconds for a streaming response.
verify_signatures true Enforce SHA-256 / Ed25519 model verification.
log_level warning PSR-style log level: debug, info, warning, error.

backends.*

Key Used by Notes
backends.embedding.model_path AI::embed(), AI::similarity() Dir with model.onnx + tokenizer.json, or a .onnx file.
backends.embedding.tokenizer_path embedding Override tokenizer.json location.
backends.classify.model_path AI::classify() Classification ONNX model.
backends.moderate.model_path AI::moderate() Moderation ONNX model.
backends.predict.model_path AI::predict() RubixML .rbm model (see cpu backend).
backends.llama.model_path AI::chat(), AI::stream() GGUF model.

Embedding options

Key Default Meaning
embedding.model all-MiniLM-L6-v2 Fallback model name if backends.embedding.model_path is unset.
embedding.pooling mean mean, cls, eos, max.
embedding.normalize true L2-normalise output vectors.

Vector store

Key Default Meaning
vector.driver sqlite sqlite or pgsql (FERRY_AI_VECTOR_DRIVER).
vector.dimension 0 Expected vector dimension (0 = auto-detect from first insert).
vector.db_path :memory: SQLite database path.
vector.dsn pgsql:host=127.0.0.1;port=5432 PostgreSQL DSN (FERRY_AI_PG_DSN).
vector.user postgres PostgreSQL user (FERRY_AI_PG_USER).
vector.password postgres PostgreSQL password (FERRY_AI_PG_PASSWORD).
vector.metric cosine cosine, euclidean, dot.

See vector-store.

Model pool & observability

Key Default Meaning
model_pool.max_memory_bytes ~2 GB LRU eviction budget.
model_pool.shared_memory false Opt-in cross-worker weight sharing (ext-shmop).
observability.metrics false Record counters/timings (FerryAI\Metrics).
observability.profiling false Record per-op durations (FerryAI\Profiler).
observability.logging false JSON log lines (set observability.log_file).

See observability in the README and examples/22-observability.php.

Native library env vars

Variable Purpose
FERRY_AI_LLAMA_WRAPPER Path to ferry_llama.dll (or set FERRY_AI_LLAMA_LIB to llama.dll in the same dir).
FERRY_AI_VEC_EXTENSION_LIB Path to the sqlite-vec vec0 library (opt-in native KNN).
FERRY_AI_TOKENIZERS_LIB Path to the native tokenizers-cpp library (optional).
FERRY_AI_RUBIXML_AUTOLOAD Path to an isolated rubix/ml autoloader.

The directory holding native DLLs must be on PATH at runtime.

Dynamic configuration

AI::backend() and AI::device() let you switch at runtime:

AI::backend('llama');               // switch backend for next calls
AI::device('cuda');                 // enable GPU offload

AI::reset() clears all facade state. AI::warmup(['model1', 'model2']) preloads models into the shared pool for instant first inference.