UnityVecEnv

A framework for running Unity environments as Gymnasium-compatible vectorized environments for reinforcement learning. Multiple agents train in parallel inside Unity while Python drives the training loop.

Warning

Development status: Alpha. UnityVecEnv is under active development. APIs, the communication protocol, and package structure may change between minor releases. Pin exact versions and use matching Python and Unity package versions.

Overview

UnityVecEnv bridges Unity and Python over a local HTTP connection using Protocol Buffers. Unity runs an embedded HTTP server; Python acts as the client. Observations and actions are binary-serialized in batches for minimal overhead.

Python training loop
      │  HTTP + Protobuf
      ▼
UnityVectorEnv  ──────────►  Unity (CommunicatorHttpServer)
  • sends actions                 • runs physics steps
  • receives obs/rewards          • collects observations
  • Gymnasium VectorEnv API       • serializes batched results

Features

• A standard Gymnasium VectorEnv interface for Unity environments.
• Many agents batched inside one Unity process.
• Multiple Unity processes exposed as one environment with FlattenedVectorEnvThreaded.
• Continuous, discrete, and mixed action/observation spaces, plus visual observations.
• Gymnasium NEXT_STEP and SAME_STEP autoreset modes.
• Typed per-agent info values and run-level environment parameters.
• Optional ONNX export and Unity tensor-renaming utilities.

Quick Start

from unity_vecenv import UnityVectorEnv

env = UnityVectorEnv(
    executable_path="path/to/MyGame.exe",
    num_envs=16,
    no_graphics=True,
    time_scale=10,
)

obs, info = env.reset()

for _ in range(1000):
    actions = env.action_space.sample()
    obs, rewards, dones, truncates, info = env.step(actions)

env.close()

For more detailed information, see both the Python side and and Unity side documentation.

Installation

Unity Package

Install the tagged package through Package Manager → Install package from git URL, or add it to your project's Packages/manifest.json:

{
  "dependencies": {
    "com.mka.gymvecenv": "https://github.com/martkartasev/UnityVecEnv.git?path=/Unity#v0.1.9"
  }
}

For local UnityVecEnv development, use Install package from disk and select ./Unity/package.json instead.

Python Package

pip install unity-vecenv

Install the optional ONNX export and model-renaming utilities with:

pip install "unity-vecenv[onnx]"

For local development:

pip install -e "./Python/unity_vecenv[dev,onnx]"

For CUDA-enabled ONNX export, install the appropriate PyTorch build for your system before installing the onnx extra.

See docs/python-usage.md for the full Python API reference and docs/unity-usage.md for implementing agents in Unity.

Managing Protobuf

If for some reason, you need have the need to change the API definition, edit Protobuf/communication.proto, then regenerate both Python and C# bindings.

One-command generation (from repo root)

bash ./Protobuf/generate_protos.sh

Manual — Python

pip install grpcio-tools

python -m grpc_tools.protoc \
  -I ./Protobuf \
  --python_out=./Python/unity_vecenv/src/unity_vecenv/protobuf_gen \
  --pyi_out=./Python/unity_vecenv/src/unity_vecenv/protobuf_gen \
  ./Protobuf/communication.proto

Manual — C#

Install protoc or Grpc.Tools, then:

protoc -I ./Protobuf \
  --csharp_out=./Unity/Runtime/Scripts/ProtobufGenerated \
  ./Protobuf/communication.proto

License

UnityVecEnv is available under the MIT License.