🚘 Self-Supervised Multi-Modal Perception + Reinforcement Learning for Autonomous Driving

This project presents a robust autonomous driving pipeline that combines Self-Supervised Learning (SSL) for perception and Proximal Policy Optimization (PPO) for decision-making in the CARLA simulator. We leverage RGB, Depth, and Semantic Segmentation inputs fused into a unified representation that drives a reinforcement learning agent to navigate complex environments.

---

📌 Key Features

• 🔀 Multi-modal Perception: Fusion of RGB, depth, and segmentation modalities.
• 🧠 Self-Supervised Learning: Contrastive loss and reconstruction loss for learning high-level visual embeddings.
• 🕹️ Deep Reinforcement Learning: PPO agent trained with perception embeddings for control in CARLA.
• 🔁 Temporal Consistency: GRU-based temporal modeling in the perception system.
• 🧪 Robust Reward Function: Penalizes collisions, lane deviation, off-road behavior; rewards smooth turns and optimal speed.

---

🧱 Architecture Overview

Inputs → Encoders (ResNet/DeepLab) → Fusion Transformer → Temporal GRU
→ SSL heads (contrastive, reconstruction) → 256D Perception Embedding → PPO Policy

Additional modules:

• Perception decoder for SSL supervision
• Reinforcement Learning agent with continuous control output

---

🧪 Experimental Setup

• Simulator: CARLA
• RL Algorithm: PPO (Stable-Baselines3)
• SSL Tasks: Contrastive learning, pixel reconstruction
• Evaluation Metrics:

Metric	Baseline RL	SSL + RL
Total Reward	1500	4200
Collision Rate	4.5	1.2
Lane Deviation	0.78 m	0.32 m
Off-road %	22%	4%

---

🧠 Training Details

• Pretraining: SSL on multi-modal CARLA scenes
• Reinforcement Learning: PPO using 256D SSL embeddings
• Hardware: NVIDIA A100 (64GB), training time ~6 days
• Losses Used:
  • 🔹 InfoNCE contrastive loss
  • 🔹 L1 reconstruction loss
  • 🔹 PPO loss with entropy regularization

---

📂 Project Structure

├── src/
│   ├── models/               # ResNet/DeepLab, Transformer, GRU
│   ├── ssl_trainer.py        # Self-supervised pretraining
│   ├── rl_training_with_ssl.py  # Trained SSL + RL
├── rl_agent/
│   ├── ppo_policy.py         # PPO policy and learning loop
│   ├── reward_function.py    # Custom reward function
├── evaluation/
│   ├── metrics.py            # Evaluation metrics
│   ├── visualize.py          # Trajectory and reward plots
├── carla_utils/
│   ├── environment_wrapper.py
├── configs/
│   ├── perception.yaml
│   ├── rl.yaml
├── README.md

---

🛠️ How to Run

1. Clone the repo

git clone https://github.com/your-username/self-driving-ssl-rl.git
cd self-driving-ssl-rl

2. Setup environment

conda create -n sslrl python=3.6
conda activate sslrl
pip install -r requirements.txt

3. Run SSL pretraining

python perception/train_ssl.py --config configs/perception.yaml

4. Train RL

python rl_agent/ppo_policy.py --config configs/rl.yaml

4. Train SSL+RL

python rl_agent/ppo_policy.py --config configs/rl.yaml

---

📊 Results

• 🚗 Smoother and safer driving behavior
• 📉 3.3x reduction in collision rate
• 🚦 Improved handling at junctions and turns

---

🔮 Future Work

• 🤖 Human-in-the-loop reward tuning
• 🔁 Online self-supervised learning with exploration
• 🧩 Integration with template-based planning modules

---

📜 Citation

If you use or reference this work in academic or industrial projects:

@misc{SushmaMareddy2025sslrl,
  title={Self-Supervised Multi-Modal Perception and Reinforcement Learning for Autonomous Driving},
  author={Sushma Mareddy, Ravan Ranveer Budda},
  year={2025},
  note={Project Report, NYU Courant}
}