Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/source/getting_started/build_from_source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,9 @@ based on the Python version. Note that ``pip`` and ``uv pip`` has slightly diff
cd /tmp/nlopt-python
git submodule update --init --recursive

# Use the same Python version as your isaacteleop install (3.10, 3.11, 3.12, or 3.13)
# The Python version must match your isaacteleop install exactly (3.10, 3.11, 3.12, or 3.13).
# nlopt builds a CPython ABI-specific extension (.cpython-311-*.so), so a wheel built
# with Python 3.11 will not load under Python 3.12 or vice versa.
uv venv --python=3.11 /tmp/nlopt-wheel-venv
VIRTUAL_ENV=/tmp/nlopt-wheel-venv uv pip install numpy setuptools wheel
/tmp/nlopt-wheel-venv/bin/python setup.py bdist_wheel -d /tmp/nlopt-wheels/
Expand Down
151 changes: 62 additions & 89 deletions docs/source/getting_started/quick_start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,12 @@ commands to the terminal.
.. contents:: Steps
:local:
:depth: 1
:backlinks: none

.. _check-out-code-base:

1. Check out code base (Optional)
---------------------------------
----------------------------------

Clone the repository and enter the project directory:

Expand All @@ -25,8 +28,10 @@ Clone the repository and enter the project directory:
As a quick start guide, we don't need to build the code base from source. However, we still need
to clone the repository for a couple quick samples to run.

.. _install-isaacteleop-pip-package:

2. Install the ``isaacteleop`` pip package
------------------------------------------
-------------------------------------------

In a new terminal, activate your preferred virtual or conda environment, then install the package
from PyPI (or from a local wheel if you built from source):
Expand All @@ -39,9 +44,10 @@ from PyPI (or from a local wheel if you built from source):
Instead of installing the package from PyPI, you can build from source and install the local wheel.
See :doc:`build_from_source/index` for more details.

.. note::
**ARM64 / aarch64 systems only** (e.g. NVIDIA DGX Spark): PyPI does not publish pre-built
``nlopt`` wheels for ARM64, so the ``retargeters`` extra cannot be installed directly from PyPI
.. dropdown:: ARM64 / aarch64 only (e.g. NVIDIA DGX Spark)

PyPI does not publish pre-built ``nlopt`` wheels for ARM64, so the ``retargeters`` extra cannot
be installed directly from PyPI
(see `issue #452 <https://github.com/NVIDIA/IsaacTeleop/issues/452>`_). Follow the
:ref:`aarch64 nlopt wheel build steps <aarch64-nlopt-wheel>` from the build-from-source guide
first, then install ``isaacteleop`` with an additional ``--find-links``:
Expand Down Expand Up @@ -116,73 +122,49 @@ You should see output similar to:
Also take note of the ``source /home/dev/.cloudxr/run/cloudxr.env`` path it mentioned in the
output. You will need to source it in step :ref:`load-cloudxr-environment-variables`.

CloudXR Configurations (Optional)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. dropdown:: CloudXR configurations (optional)

You can also inspect the CloudXR environment variables by running:
The CloudXR runtime uses the ``auto-webrtc`` device profile by default (Pico & Quest). For
Apple Vision Pro it defaults to ``auto-native``.

.. code-block:: bash

cat ~/.cloudxr/run/cloudxr.env
To inspect the active settings after startup:

You should see:

.. code-block:: text

export NV_DEVICE_PROFILE=auto-webrtc
...

By default, the CloudXR runtime is configured to use the ``auto-webrtc`` device profile for Pico &
Quest headsets. For Apple Vision Pro, the runtime is configured to use the ``auto-native`` device
profile.

To do that, you can override CloudXR configurations by creating an ``env`` file and pass it to the
CloudXR runtime via the ``--cloudxr-env-config`` argument.
.. code-block:: bash

.. code-block:: bash
cat ~/.cloudxr/run/cloudxr.env

echo 'NV_DEVICE_PROFILE=auto-native' > custom.env
python -m isaacteleop.cloudxr --cloudxr-env-config=./custom.env
To override settings, create an env file and pass it at startup:

Again, you can also inspect the CloudXR environment variables by looking at the
``~/.cloudxr/run/cloudxr.env`` file:
.. code-block:: bash

.. dropdown:: Custom CloudXR configurations
echo 'NV_DEVICE_PROFILE=auto-native' > custom.env
python -m isaacteleop.cloudxr --cloudxr-env-config=./custom.env

.. list-table:: Custom CloudXR configurations
.. list-table:: Environment variables
:header-rows: 1
:widths: 20 15 30 20
:width: 100%
:widths: 25 15 35 25

* - Variable
- Default
- Description
- Possible Values
* - NV_DEVICE_PROFILE
- Values
* - ``NV_DEVICE_PROFILE``
- ``auto-webrtc``
- Custom device profile to use
- | ``auto-webrtc``
| ``auto-native``
| ``Quest3``
| ``AppleVisionPro``
* - NV_CXR_ENABLE_PUSH_DEVICES
- Device profile
- ``auto-webrtc``, ``auto-native``, ``Quest3``, ``AppleVisionPro``
* - ``NV_CXR_ENABLE_PUSH_DEVICES``
- ``true``
- Enable or disable push device overseer for hand tracking
- | ``true``
| ``false``
* - NV_CXR_FILE_LOGGING
- Push device overseer for hand tracking
- ``true``, ``false``
* - ``NV_CXR_FILE_LOGGING``
- ``true``
- Enable or disable file-based logging, when disabled logs are printed to the console
- | ``true``
| ``false``
- File-based logging (disable to print to console)
- ``true``, ``false``

.. code-block:: bash

export NV_DEVICE_PROFILE=auto-native
...
.. _whitelist-firewall-ports:

4. Whitelist ports for Firewall
-------------------------------
---------------------------------

CloudXR requires certain network ports to be open. Depending on your firewall configuration, you
might need to whitelist them manually.
Expand Down Expand Up @@ -224,24 +206,28 @@ running the CloudXR runtime and wss proxy in containerized environment; or using

.. _connect-quest-pico:

.. dropdown:: Meta Quest and Pico headsets
.. dropdown:: Meta Quest, PICO headset, or desktop browser
:open:

To stream from a Meta Quest or PICO headset, you will need a CloudXR web client. For your
convenience, we host a prebuilt CloudXR web client at `nvidia.github.io/IsaacTeleop/client`_.
You can just open this URL in your headset's browser. No need to build or install a separate client.
No physical headset required for a quick test: open
`https://nvidia.github.io/IsaacTeleop/client <https://nvidia.github.io/IsaacTeleop/client>`__
in a **desktop browser** — IWER (Immersive Web Emulator Runtime) loads automatically and
emulates a Meta Quest 3 headset.

For a real headset, open the same URL in your **Meta Quest or PICO browser**.

.. important::

Make sure your headset is updated to the latest firmware (system software) version before
connecting. Older firmware may ship an outdated browser/WebXR runtime that fails to connect or
streams with reduced functionality.
If using a physical headset, make sure it is updated to the latest firmware before connecting.
Older firmware may ship an outdated WebXR runtime that fails to connect or streams with reduced
functionality.

.. tip::
.. note::

For quick validation, you can also open the `nvidia.github.io/IsaacTeleop/client`_ URL in a
desktop browser. IWER (Immersive Web Emulator Runtime) will automatically load to emulate a Meta
Quest 3 headset.
If GitHub Pages is unreachable (corporate network, air-gapped machine), start the server with
``--host-client`` in step :ref:`run-cloudxr-server` and open
``https://<your-ip>:48322/client/`` instead of the GitHub Pages URL. Port 48322 is already
whitelisted in step :ref:`whitelist-firewall-ports`.

.. tab-set::
.. tab-item:: CloudXR web client
Expand Down Expand Up @@ -280,33 +266,18 @@ running the CloudXR runtime and wss proxy in containerized environment; or using
For advanced usage and troubleshooting of CloudXR, see the `CloudXR documentation`_ for more
details.

Alternatively, pass ``--host-client`` when starting the launcher to serve
the web client directly from the streaming host (useful when GitHub Pages is
unreachable). The client is served at ``/client/`` on the WSS proxy port
(no extra port needed). The launcher prints the local URL on startup:
.. dropdown:: Offline / air-gapped use

.. code-block:: text
On **first run**, the launcher fetches ``index.html`` and ``bundle.js`` from GitHub Pages and
caches them in ``~/.cloudxr/static-client/`` (override with
``TELEOP_WEB_CLIENT_STATIC_DIR``). Subsequent runs are fully offline.

web client: https://10.0.1.5:48322/client/ (hosted locally — open on your headset or browser)
For a **true air-gapped machine**, pre-stage the two files before the first run — copy them
from ``https://nvidia.github.io/IsaacTeleop/client/`` on a networked host, then transfer the
``~/.cloudxr/static-client/`` directory to the air-gapped machine.

Open that URL instead of the GitHub Pages link above. The rest of the
flow — certificate acceptance, entering the server IP, clicking Connect —
is the same. Port 48322 is already open from step 4.

.. note::

On **first run**, the launcher fetches ``index.html`` and ``bundle.js`` from
GitHub Pages and caches them in ``~/.cloudxr/static-client/`` (override with
``TELEOP_WEB_CLIENT_STATIC_DIR``). Subsequent runs are fully offline.

For a **true air-gapped machine**, pre-stage the two files before the first
run — copy or download them from
``https://nvidia.github.io/IsaacTeleop/client/`` into
``~/.cloudxr/static-client/`` on a networked host, then transfer the
directory to the air-gapped machine.

The source code for the web client is in the :code-dir:`deps/cloudxr/webxr_client/` directory. To build the web
client from source, see :doc:`build_from_source/webxr`.
The source code for the web client is in the :code-dir:`deps/cloudxr/webxr_client/` directory.
To build the web client from source, see :doc:`build_from_source/webxr`.

.. _connect-apple-vision-pro:

Expand Down Expand Up @@ -341,8 +312,10 @@ Source the setup script so that the OpenXR runtime points to CloudXR:
Make sure to run the rest of the commands in the same terminal. Or if have to open a new
terminal, source the CloudXR environment variables again.

.. _run-teleop-example:

7. Run a teleop example
-----------------------
------------------------

Run the simplified gripper retargeting example. This demonstrates the full
pipeline: reading XR controller input via CloudXR, retargeting it through the
Expand Down
2 changes: 1 addition & 1 deletion docs/source/references/egocentric_hand_reconstruction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Video Capture
---------------------------

To capture egocentric video with an OAK camera, see the
`OAK camera plugin <https://nvidia.github.io/IsaacTeleop/main/device/oak.html>`_ documentation.
:doc:`/device/oak` documentation.

Setup
-----
Expand Down
Loading