You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Robot Control Stack (RCS)** is a flexible Gymnasium wrapper-based robot control interface made for robot learning and specifically Vision-Language-Action (VLA) models.
4
-
It unifies MuJoCo simulation and real world robot control with four supported robots: FR3/Panda, xArm7, UR5e and SO101.
5
-
It ships with several pre-build apps including data collection via teleoperation and remote model inference via [vlagents](https://github.com/RobotControlStack/vlagents).
4
+
### A lean, ROS-free Sim-to-Real framework for training and deploying Vision-Language-Action (VLA) models and Reinforcement Learning (RL) agents.
**Robot Control Stack (RCS)** is a flexible, native [Gymnasium](https://gymnasium.farama.org/) wrapper-based robot control interface designed specifically for modern robot learning and Vision-Language-Action (VLA) models.
16
+
17
+
It completely unifies **MuJoCo simulation** and real-world physical robot control into a single, seamless API. Currently, RCS natively supports four robots out-of-the-box: **Franka FR3/Panda, xArm7, UR5e, and SO101.**
Traditional robotics middleware (like ROS/ROS2) and complex motion planning pipelines (like MoveIt or standard `ros2_control`) are built for asynchronous, distributed systems. This often becomes a massive bottleneck when attempting to train modern, synchronous machine learning models.
24
+
25
+
**RCS is built differently:**
26
+
***Zero ROS Overhead:** No complex message-passing, middleware, or network configuration required. Run natively in Python with a lightweight C++ backend.
27
+
***Frictionless Sim-to-Real:** Train your Reinforcement Learning or VLA policies in our MuJoCo Gymnasium wrapper, and deploy the *exact same code* directly to physical hardware.
28
+
***Synchronous Execution:** Optimized specifically for the highly parallelized, synchronous data collection required by modern ML workflows.
29
+
***Ready-to-Use Apps:** Ships with pre-built applications for data collection via teleoperation and remote model inference via [vlagents](https://github.com/RobotControlStack/vlagents).
30
+
31
+
## 🧩 Wrapper-Based Architecture
32
+
33
+
RCS utilizes a highly modular, wrapper-based architecture, allowing you to easily stack capabilities (cameras, grippers, action spaces) as needed.
Flexibly compose your gym environment to your needs:
37
+
## 💻 Example: Composing your Environment
38
+
39
+
Flexibly compose your Gymnasium environment to fit your exact training needs. *For common environment compositions, factory functions such as `rcs.envs.creators.SimEnvCreator` are provided.*
obs, reward, terminated, truncated, info = env.step(act)
95
113
print(obs)
114
+
96
115
```
97
-
For common environment compositions factory functions such as `rcs.envs.creators.SimEnvCreator` are provided.
98
-
This and other example can be found in the [examples](examples/) folder.
99
116
100
-
## Installation
117
+
> **Note:** This and other examples can be found in the [`examples/`]() folder.
118
+
119
+
## 🛠️ Installation
120
+
101
121
### From Source
102
122
103
-
Make sure that common build tools i.e. `build-essential` and a C++ compiler like `gcc` or `clang` are installed on your system/conda/docker.
123
+
Make sure that common build tools (i.e., `build-essential`) and a C++ compiler like `gcc` or `clang` are installed on your system/conda/docker.
124
+
125
+
*RCS works best in Python 3.11, and all extensions have been tested to work in 3.11.*
104
126
105
-
RCS works best in python 3.11, all extensions have been tested to work in 3.11.
106
-
- For python >3.11 the `rcs_realsense` extension wont work for the pyrealsense2 version that RCS is using.
107
-
- For python >3.12 the ompl python module is not available on pypi as of now. If OMPL is not used, it's save to remove this dependency in the pyproject.toml.
127
+
**For Python >3.11: The `rcs_realsense` extension won't work due to the `pyrealsense2` version RCS utilizes.*
128
+
**For Python >3.12: The `ompl` python module is currently not available on PyPI. If OMPL is not used, it is safe to remove this dependency in `pyproject.toml`.*
RCS supports various hardware extensions (e.g., FR3, xArm7, RealSense). These are located in the `extensions` directory.
148
+
## 🦾 Hardware Extensions
128
149
129
-
To install an extension:
150
+
RCS supports various hardware extensions to seamlessly connect your policies to the real world (e.g., FR3, xArm7, RealSense). These are located in the `extensions` directory.
151
+
152
+
To install a specific robot extension (example for Franka FR3):
130
153
131
154
```shell
132
155
pip install -ve extensions/rcs_fr3
133
156
```
134
157
135
-
For a full list of extensions and detailed documentation, visit [robotcontrolstack.org/extensions](https://robotcontrolstack.org/extensions).
158
+
For a full list of extensions and detailed documentation, visit **[robotcontrolstack.org/extensions](https://robotcontrolstack.org/extensions)**.
159
+
160
+
## ⚠️ Troubleshooting & FAQ
161
+
***License error or group argument not found during installation?** Make sure you are using a pip version `>=25.1` and setuptools version `>=45`.
162
+
***Dependency error during installation?** Make sure you are using Python 3.11. RCS extensions currently do not support 3.12+ due to OMPL and RealSense dependencies.
163
+
***Simulation is running too slow?** Check that you have enable on-demand rendering: `SimCameraSet(..., render_on_demand=True)` to render camera frames only once per step. Resolution and number of cameras in the scene has a large impact on simulation speed. Make sure to use a decent GPU when rendering is enabled.
164
+
136
165
137
-
## Documentation
138
-
For full documentation, including installation, usage, and API reference, please visit:
If you find RCS useful for your academic work, please consider citing it:
171
+
## 🤝 Contribution
172
+
173
+
We welcome contributions from the robotics and ML community! For contribution guidelines, please check out **[robotcontrolstack.org/contributing](https://robotcontrolstack.org/contributing)**.
174
+
175
+
## 📝 Citation
176
+
177
+
If you find RCS useful for your academic work please consider citing it:
146
178
147
179
```bibtex
148
180
@inproceedings{juelg2026robotcontrolstack,
@@ -153,4 +185,6 @@ If you find RCS useful for your academic work, please consider citing it:
153
185
note={Accepted for publication.}
154
186
}
155
187
```
156
-
For more scientific info, visit the [paper website](https://robotcontrolstack.github.io/).
188
+
189
+
For more scientific information and supplementary videos, visit the **[paper website](https://robotcontrolstack.github.io/)**.
0 commit comments