🤖 mjlab Training Studio

Professional web-based GUI for humanoid robot training with motion imitation

⚠️ Note: This repository extends mjlab with a user-friendly training studio. The GUI and wrapper scripts were AI-generated on October 4, 2025.

---

📥 New to mjlab? Installation Guides

• 🚀 Quick Start - Get running in 5 minutes
• 🔧 Installation Guide - Complete step-by-step setup
• 💻 System Requirements - Hardware specs & compatibility

Already installed? Continue below ↓

---

🚀 Quick Start

Launch the Training Studio

cd /path/to/mjlab
uv run --with gradio python app/gui/mjlab_gui.py

Access at: http://localhost:7860

Your First Motion Imitation in 3 Steps:

1. 📊 Process Motion (Tab: Motion Imitation → Step 1)

   • Select from 496 pre-made LAFAN1 motions
   • Click "Process Motion"
   • Watch video render in real-time (~30 seconds)

2. 🎭 Train Robot (Tab: Motion Imitation → Step 2)

   • Select your processed motion
   • Click "Start Imitation Training"
   • Wait ~4 hours (or use Quick preset for testing)

3. 🎮 Visualize (Tab: Test & Visualize)

   • Select your checkpoint
   • Click "Start Visualization"
   • See your robot perform the motion in 3D!

---

✨ Features

🎯 Complete Training Workflows

Workflow	What You Get	Time
Velocity Tracking	Robot that walks, runs, turns, strafes	2-3 hours
Motion Imitation	Robot that performs custom dance/sports moves	4-5 hours
Checkpoint Visualization	Interactive 3D viewer of trained behaviors	Instant

🎨 World-Class User Experience

• Live Progress Dashboards - Real-time metrics, progress bars, ETA
• Auto-Refresh - No manual refresh needed, updates every 2 seconds
• Video Playback - Rendered motion previews with autoplay
• Embedded 3D Viewer - Viser iframe loads directly in dashboard
• One-Click Workflows - Preset configurations for beginners
• 496 Motion Library - LAFAN1 dataset browser with search

🔧 Local-First Design

• ✅ No Cloud Required - Fully functional offline
• ✅ No WandB Account - Bypasses cloud dependencies gracefully
• ✅ Organized Outputs - Timestamped folders with all artifacts
• ✅ Non-Invasive - Zero modifications to core mjlab code

---

📖 Detailed Workflows

Velocity Tracking Training

Train a robot to follow movement commands like a video game character:

1. Go to "Velocity Tracking" tab
2. Choose preset:
   • ⚡ Quick (30 min) - Basic locomotion test
   • 📊 Standard (2 hours) - Good walking behavior
   • 🔬 Full (8 hours) - Publication-quality results
3. Click "Start Training"
4. Monitor progress:
   • Iteration counter (e.g., 5000/30000)
   • Mean reward (target: ~100+)
   • Episode length (target: ~500 steps)
   • Training speed (~200-400 it/s)
   • ETA to completion

What the robot learns:

• Walk forward/backward (-1.0 to +1.0 m/s)
• Strafe left/right (-0.5 to +0.5 m/s)
• Turn in place (-1.0 to +1.0 rad/s)
• Smooth transitions between gaits
• Stable standing

---

Motion Imitation (Complete Pipeline)

Step 1: Process Motion Data

From LAFAN1 Library:

1. Navigate to "Motion Imitation" → "Step 1: Dataset Browser"
2. Select robot type (G1, GO1, etc.)
3. Browse 496 motions (dance, sports, locomotion, interactions)
4. Click a motion to see preview chart
5. Click "Process Motion"
6. Watch live progress:
   • Frame interpolation (30 FPS → 50 FPS)
   • MuJoCo simulation
   • Video rendering with moviepy
   • NPZ file saving
7. Output: app/datasets/processed/<motion>_<timestamp>/
   • motion.npz - Training data
   • motion.mp4 - Preview video (auto-plays)
   • processing.log - Complete log

From Custom CSV:

1. Click "Upload Custom CSV"
2. Select your motion file (columns = joint names, rows = frames)
3. Motion name auto-fills from filename
4. Continue with processing...

CSV Format:

joint_1,joint_2,joint_3,...,joint_29
0.1,0.2,0.3,...,0.0
0.12,0.21,0.31,...,0.01
...

Step 2: Train Robot on Motion

1. Select processed NPZ from dropdown
2. Configure training (or use defaults):
   • Parallel Environments: 1024 (fewer for motion imitation)
   • Max Iterations: 30,000 (~4 hours)
   • Save Interval: 500 (checkpoint frequency)
   • Learning Rate: 0.0005
3. Click "Start Imitation Training"
4. Monitor dashboard:
   • Iteration progress
   • Mean reward (track improvement)
   • Episode length
   • Time elapsed / ETA
   • Training speed
5. Output: logs/rsl_rl/g1_tracking/<timestamp>_<motion_name>/
   • model_0.pt, model_500.pt, ... - Checkpoints
   • params/env.yaml - Contains motion file path
   • summaries/ - TensorBoard logs

Step 3: Visualize Results

1. Go to "Test & Visualize" tab
2. Checkpoint auto-selects (most recent)
3. Task auto-detects (Tracking vs Velocity)
4. Click "Start Visualization"
5. Wait ~15 seconds watching live log:
   • ✅ Loading checkpoint
   • 🎭 Found motion file from training config
   • 🏗️ Creating environment
   • 🧠 Loading policy
   • 🌐 Starting Viser viewer
6. 3D viewer auto-loads in iframe!
7. Interact with robot:
   • Rotate camera with mouse
   • Watch motion loop continuously
   • See learned behavior quality

---

🎮 Interface Guide

Dashboard Layout

┌─────────────────────────────────────────────────────────────┐
│  🤖 mjlab Training Studio                                    │
│  Professional robotics training with guided workflows        │
├─────────────────────────────────────────────────────────────┤
│  ⚙️ Global Settings  │  Tabs: → → → → →                     │
│  Device: cuda:0 ▼    │                                       │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  Tab Content Area:                                           │
│  ┌───────────────────────┬─────────────────────────────┐    │
│  │   Controls &          │   Live Dashboard /          │    │
│  │   Configuration       │   Output Display            │    │
│  │                       │                             │    │
│  │   - Dropdowns         │   - Progress Metrics        │    │
│  │   - Sliders           │   - Status Messages         │    │
│  │   - Buttons           │   - Video/3D Viewer         │    │
│  │   - Presets           │   - Collapsible Logs        │    │
│  └───────────────────────┴─────────────────────────────┘    │
│                                                               │
│  Documentation & Tips (below each tab)                       │
└─────────────────────────────────────────────────────────────┘

Key UI Elements

• 📊 Progress Bars - Visual completion status
• ⏱️ Live Timers - Real-time ETA calculations
• 📋 Collapsible Logs - Detailed output when needed
• 📋 Copy Buttons - One-click log copying
• 🔄 Auto-Refresh - No manual refresh needed
• 🎬 Video Players - Auto-play with loop
• 🌐 Embedded iframes - 3D viewers in dashboard

---

🏗️ Architecture

Application Structure

app/
├── gui/mjlab_gui.py              # Main Gradio dashboard (2,316 lines)
├── scripts/
│   ├── process_motion_local.py   # Motion processing wrapper
│   ├── train_motion_local.py     # Training wrapper  
│   └── visualize_checkpoint.py   # Visualization script
└── datasets/
    ├── LAFAN1_Retargeting_Dataset/  # 496 pre-made motions
    └── processed/                    # Your processed outputs
        └── <motion>_<timestamp>/
            ├── motion.npz
            ├── motion.mp4
            └── processing.log

Design Philosophy

Non-Invasive Wrapper Pattern:

• Core mjlab library remains unchanged
• Wrapper scripts handle WandB bypass via monkey-patching
• All application code in separate app/ directory

Local-First Development:

• No cloud accounts required
• All data stored locally in organized folders
• WandB integration preserved for users who want it

World-Class UX:

• Live progress tracking everywhere
• Auto-detection and smart defaults
• Clear error messages with solutions
• Professional visual design

---

🔧 Technical Details

WandB Bypass Strategy

The core mjlab scripts expect WandB for artifact management. Our wrappers provide local alternatives:

Motion Processing:

# Patches wandb.init to return mock run object
# Allows video rendering to complete
# Copies files to permanent storage

Training:

# Patches wandb.Api.artifact to return local paths  
# Uses local NPZ files instead of downloads
# Adds motion names to checkpoint folders

Process Management

• ProcessManager class - Streams subprocess output in real-time
• Unbuffered I/O - PYTHONUNBUFFERED=1 for immediate logging
• Thread-safe queues - Background streaming without blocking
• Graceful termination - Clean shutdown on Stop button

Motion File Resolution

For checkpoint visualization, motion files are found via:

1. Primary: Read from params/env.yaml (saved during training)
2. Fallback: Parse motion name from folder name
3. Validation: Check file exists before loading

---

📁 Output Organization

Processed Motions

app/datasets/processed/
└── dance2_subject5_2025-10-04_01-17-13/
    ├── motion.npz           # Training data (joint pos/vel, body states)
    ├── motion.mp4           # Rendered preview video
    └── processing.log       # Complete processing log

Training Checkpoints

logs/rsl_rl/
├── g1_velocity/
│   └── 2025-10-03_23-55-36/
│       ├── model_0.pt, model_500.pt, ...
│       ├── params/
│       │   ├── env.yaml     # Environment config
│       │   └── agent.yaml   # Training hyperparameters
│       └── summaries/       # TensorBoard logs
│
└── g1_tracking/
    └── 2025-10-04_02-45-09_dance2_subject5_2025-10-04_01-17-13/
        ├── model_0.pt, model_500.pt, ...
        ├── params/
        │   └── env.yaml     # ← Contains motion_file path!
        └── summaries/

Note: Tracking checkpoints include motion name in folder for easy identification.

---

🎓 Learning Resources

For Beginners:

Start Here: Tab 1 → "Getting Started"

• Overview of the workflow
• Quick start guide
• Recommended first project

First Project: Velocity Tracking (30 minutes)

1. Click "Velocity Tracking" tab
2. Click "⚡ Quick" preset
3. Click "Start Training"
4. Wait 30 minutes
5. Visualize your walking robot!

For Advanced Users:

Motion Imitation:

• Requires understanding of:
  • Motion retargeting concepts
  • Joint angle representations
  • Imitation learning theory

Custom Motions:

• CSV format requirements
• Joint naming conventions
• FPS and interpolation settings

Hyperparameter Tuning:

• Learning rate effects
• Batch size vs convergence
• Reward shaping strategies

Additional Documentation:

• mjlab Core: See README.mjlab.md for original framework docs
• Application Layer: See app/README.md for technical details
• Session History: See .cursor/gui-development-session.md for development context
• API Reference: See docs/api/ for configuration details

---

🐛 Troubleshooting

Dashboard won't start

# Kill any existing instances
pkill -f mjlab_gui.py

# Restart
cd /home/mitch/Repositories/mjlab
uv run --with gradio python app/gui/mjlab_gui.py

Visualization stuck on "Loading..."

# Kill zombie processes
pkill -9 -f visualize_checkpoint.py

# Restart from GUI
# Or hard-refresh browser: Ctrl+Shift+R

"Connection refused" in iframe

Solution: Wait 15 seconds for Viser server to start, then click "🔄 Load Viewer"

Alternative: Open http://localhost:8080 in a new browser tab

Training won't start

Check:

1. NPZ file exists in app/datasets/processed/
2. GPU is available: nvidia-smi
3. No other training running: ps aux | grep train

Common Fix:

• Restart dashboard (kills existing processes)
• Check logs in collapsible accordion

Video won't render

Check:

1. "Render video" toggle is ON
2. Disk space available: df -h /tmp
3. moviepy installed: uv run python -c "import moviepy"

Note: Video rendering adds ~10-20 seconds to processing

Wrong task selected for checkpoint

Don't worry! Server-side validation auto-corrects the task based on checkpoint path.

If still broken: Hard-refresh browser (Ctrl+Shift+R)

---

📊 Performance Guide

Hardware Recommendations

Component	Minimum	Recommended	Our Setup
GPU	NVIDIA GTX 1080 (8GB)	RTX 3090 (24GB)	RTX 6000 Ada (48GB)
RAM	16 GB	32 GB	64+ GB
CPU	4 cores	8+ cores	16+ cores
Storage	50 GB	100 GB	500 GB+

Training Times (1024 environments)

Task	Iterations	Time	GPU Usage
Velocity (Quick)	5,000	30 min	80-95%
Velocity (Standard)	15,000	2 hours	80-95%
Motion Imitation	30,000	4 hours	80-95%

Processing Times

Motion Length	Frames	Process Time	Video Render
Short	1,000	3 sec	5 sec
Medium	3,000	10 sec	10 sec
Long	6,000+	20 sec	20 sec

GPU Utilization

Current (Single GPU):

• Training uses cuda:0 only
• Second GPU (cuda:1) is idle
• GPU-bound during simulation (80-95%)
• CPU-bound during policy updates (20-40% GPU)

To use both GPUs: Run two separate training jobs manually:

# Terminal 1
uv run train <task> --device cuda:0 ...

# Terminal 2  
uv run train <task> --device cuda:1 ...

---

🎨 Screenshots & Demo

Dashboard Overview

[Placeholder for screenshot of main dashboard]

Motion Processing

[Placeholder for screenshot of processing with live progress]

3D Visualization

[Placeholder for screenshot of embedded Viser viewer]

Training Metrics

[Placeholder for screenshot of training dashboard]

---

🔍 Advanced Features

TensorBoard Integration

Launch TensorBoard for detailed training metrics:

# From GUI: Click "Launch TensorBoard" button
# Or manually:
uv run tensorboard --logdir logs/rsl_rl --port 6006

Access at: http://localhost:6006

View:

• Reward curves over time
• Policy loss and value loss
• Learning rate schedules
• Episode length trends

Custom Motion CSV Format

Your CSV should have:

• Header row: Joint names (29 joints for Unitree G1)
• Data rows: Joint angles in radians
• Frame rate: Any (will interpolate to 50 FPS)

Example:

left_hip_pitch,left_hip_roll,left_hip_yaw,...
0.1,0.05,0.0,...
0.12,0.06,0.01,...

Joint names: See src/mjlab/asset_zoo/robots/unitree_g1/g1_constants.py

Checkpoint Management

Automatic Naming:

• Velocity: logs/rsl_rl/g1_velocity/<timestamp>/
• Tracking: logs/rsl_rl/g1_tracking/<timestamp>_<motion_name>/

Motion name in folder makes it easy to identify which motion was trained!

View checkpoints:

ls -lR logs/rsl_rl/

---

🤝 Contributing

For Core mjlab Development:

See original repository: https://github.com/mujocolab/mjlab

For GUI Improvements:

This repository (mjlab-gui) focuses on the application layer:

Areas for contribution:

• Additional GUI tabs (terrain generation, hyperparameter tuning)
• New motion processing features
• Improved visualization tools
• Better error handling and recovery
• Performance optimizations
• Documentation improvements

Development setup:

git clone https://github.com/experientialtech/mjlab-gui.git
cd mjlab-gui
# Follow mjlab installation instructions
# Then launch GUI as shown above

---

📄 License

This repository contains:

1. mjlab core library - See LICENSE (original mjlab license)
2. Training Studio GUI - AI-generated code (October 4, 2025)
   • See individual file headers for disclaimers
   • Review before production use

---

🙏 Acknowledgments

• mjlab Framework: https://github.com/mujocolab/mjlab
• LAFAN1 Dataset: Retargeted for Unitree robots
• Viser: Interactive 3D visualization
• RSL-RL: RL training algorithms
• MuJoCo: Physics simulation
• Warp: GPU acceleration

GUI Development:

• AI Assistant: Claude Sonnet 4.5 (via Cursor)
• Developer: Mitch (experientialtech)
• Date: October 4, 2025
• Session: ~3 hours of iterative development

---

📞 Support

Common Issues:

See Troubleshooting section above, or check:

• .cursor/gui-development-session.md - Full technical context
• app/README.md - Application architecture
• Tab 6 in GUI: "Help & Examples"

For Help:

1. Check the collapsible logs in the GUI
2. Review the session development notes
3. Open an issue on GitHub
4. Check original mjlab documentation

---

🗺️ Roadmap

Completed ✅

• Web-based training studio
• Motion imitation workflow
• Local WandB bypass
• Embedded 3D visualization
• Live progress dashboards
• LAFAN1 dataset integration
• Comprehensive documentation

Planned 🎯

• Multi-GPU training support
• Checkpoint comparison tool
• Motion blending/sequencing
• ONNX export integration
• Custom terrain generation UI
• Real robot deployment wizard
• Training hyperparameter auto-tuning

Future Ideas 💡

• Cloud deployment option
• Multi-user support
• Motion sharing marketplace
• Benchmark leaderboards
• Mobile app for monitoring
• Integration with physical Unitree robots

---

📚 Related Links

• Original mjlab: https://github.com/mujocolab/mjlab
• MuJoCo: https://mujoco.org/
• Gradio: https://gradio.app/
• Viser: https://viser.studio/
• LAFAN1: https://github.com/ubisoft/ubisoft-laforge-animation-dataset

---

Built with ❤️ and 🤖 AI
Last Updated: October 4, 2025
Version: 1.0.0

---

For the original mjlab README, see README.mjlab.md