🎙️ Oncydra — Voice-First AI Cancer Care Platform

Empowering cancer patients through conversational AI — track symptoms, understand side effects, prepare for appointments, and stay connected with care teams through natural voice interactions.

"Every year, 20 million people are diagnosed with cancer worldwide. Oncydra ensures they're never alone in their journey." — WHO Global Cancer Statistics 2024

🎯 Project Overview

Oncydra is a voice-first oncology care platform that uses multi-agent AI to support cancer patients throughout their treatment journey. The system features 26 ADK agents and 5 ElevenLabs voice agents across 8 backend services, enabling patients to:

Report symptoms through natural conversation
Get immediate "Is This Normal?" triage guidance
Prepare questions and rehearse doctor conversations
Receive personalized education content
Stay on track with appointments and reminders
Call their care team with AI advocacy

For clinicians, Oncydra provides real-time patient monitoring with Human-in-the-Loop (HITL) safety controls ensuring AI-generated content is reviewed before reaching patients.

🏗️ Architecture

Architecture Diagram 1 — Patient Dashboard (Symptom, Education, Advocacy, Reminder Services)

Architecture Diagram 2 — Patient Dashboard (Timeline & Communication Services)

Architecture Diagram 3 — Clinician Dashboard (Consultation & Analytics Services)

🔧 Core Services

Oncydra consists of 7 Cloud Run microservices + 1 Cloud Function:

Service	Type	Description	ADK Agents
🩺 Symptom Service	Cloud Run	Extracts and analyzes symptoms from voice check-ins	7
📅 Timeline Service	Cloud Run	Extracts appointments from conversations, syncs to Google Calendar	1
📚 Education Service	Cloud Run	Generates personalized oncology education with Google Search grounding	5
🎯 Advocacy Service	Cloud Run	Prepares questions and rehearsal dialogues for appointments	4
🏥 Consultation Service	Cloud Run	Processes clinical consultations with HITL safety gating	5
⏰ Reminder Service	Cloud Run	Monitors check-ins and sends personalized email reminders (LoopAgent)	4
📞 Communication Service	Cloud Run	AI-powered phone calls via ElevenLabs + Twilio	0 (ElevenLabs)
📊 Analytics Function	Cloud Function	BigQuery queries for population-level insights	0

Total: 26 ADK Agents + 5 ElevenLabs Conversational Agents

🤖 Agent Workflow

graph TD
    subgraph "Patient Voice Interaction"
        A[🎙️ ElevenLabs Patient Companion] --> B[Symptom Service]
        C[🎙️ ElevenLabs Triage Assistant] --> D[TriageAgent]
        E[🎙️ ElevenLabs Timeline Assistant] --> F[Timeline Service]
        G[🎙️ ElevenLabs Mock Rehearsal] --> H[Advocacy Service]
        I[🎙️ ElevenLabs Care Advocate] --> J[Communication Service]
    end
    
    subgraph "Symptom Intelligence Pipeline — 7 Agents"
        B --> K[SymptomRootAgent - Orchestrator]
        K --> L[SymptomExtractionAgent - Parallel]
        K --> M[TrendAnalyzerAgent - Parallel]
        K --> N[RiskScorerAgent - Parallel]
        L & M & N --> O[SynthesisAgent - Gatherer]
        O --> P[QualityReviewerAgent - Critique]
        D --> Q[TriageAgent - Standalone]
    end
    
    subgraph "Education Pipeline — 5 Agents"
        R[Education Service] --> S[EducationOrchestratorAgent]
        S --> T[TopicGeneratorAgent]
        T --> U[ContentResearcherAgent + Google Search]
        U --> V[ContentSynthesizerAgent]
        W[QuickEducatorAgent - Standalone]
    end
    
    subgraph "Advocacy Pipeline — 4 Agents"
        H --> X[AdvocacyOrchestrator]
        X --> Y[QuestionCoachAgent]
        Y --> Z[RehearsalDialogueAgent]
        Z --> AA[AppointmentPrepAgent]
    end
    
    subgraph "Reminder Pipeline — 4 Agents (Loop Pattern)"
        AB[Reminder Service] --> AC[FollowUpGuardianAgent - Loop Orchestrator]
        AC --> AD[CheckInMonitorAgent - Iteration 1]
        AD --> AE[AppointmentMonitorAgent - Iteration 2]
        AE --> AF[ReminderComposerAgent - Iteration 3]
        AF --> AG[📧 Gmail API]
    end
    
    subgraph "Consultation Pipeline — 5 Agents (HITL Pattern)"
        AH[Consultation Service] --> AI[ConsultationOrchestratorAgent]
        AI --> AJ[ClinicalEntityExtractorAgent]
        AJ --> AK[SummaryGeneratorAgent]
        AK --> AL[ConcernDetectorAgent]
        AL --> AM[HITLGatekeeperAgent]
        AM -->|🔴 BLOCKED| AN[Clinician Review Required]
        AN -->|✅ Approved| AO[Actions Execute]
    end
    
    subgraph "Communication Service — ElevenLabs + Twilio"
        J --> AP[Outbound AI Phone Call]
        AP --> AQ[get_patient_context webhook]
        AP --> AR[book_appointment webhook]
        AP --> AS[report_outcome webhook]
    end
    
    P --> AT[(Firestore)]
    AT --> AB
    AT --> AH
    F --> AU[📅 Google Calendar]

✨ Key Features

For Patients

Feature	Description
🎙️ Voice Check-in	Talk naturally about how you're feeling; AI extracts symptoms automatically
❓ "Is This Normal?" Triage	Ask about worrying symptoms and get immediate, personalized guidance
📞 AI Care Team Calls	Oncydra calls your care team on your behalf via Twilio
🎯 Doctor Rehearsal	Practice conversations and questions before appointments
📚 Personalized Education	Get relevant information about your specific treatment and symptoms
🌀 Orbital Timeline	Unique radial visualization of your care journey
📅 Voice Calendar Sync	Say "I have a CT scan Tuesday" and it syncs to Google Calendar
📊 Weekly Trends	Track symptom patterns over time

For Clinicians

Feature	Description
👥 Patient Directory	View all patients with risk stratification
🚨 Real-time Alerts	Get notified of concerning symptom patterns
📝 SOAP Notes	AI-generated clinical documentation
✅ HITL Safety Controls	Review and approve AI content before it reaches patients
📈 Population Analytics	BigQuery-powered insights across patient cohorts
🔄 Consultation Processing	Extract entities, detect concerns, generate summaries
🎙️ Voice Timeline Entry	Add appointments to patient timelines via voice

ElevenLabs Voice Agents

Agent	Purpose	Webhook Tools
Patient Companion	Daily symptom check-ins	—
Triage Assistant	"Is This Normal?" assessment	—
Timeline Assistant	Extract appointments from conversation	—
Mock Rehearsal	Practice appointment conversations	`get_rehearsal_context`
Care Advocate	AI-powered phone calls to care team	`get_patient_context`, `book_appointment`, `report_outcome`

🛠️ Technology Stack

Frontend

Framework: React 18 + TypeScript + Vite
Styling: Tailwind CSS
Animations: Framer Motion
Charts: Recharts
Voice: ElevenLabs React SDK
State: Firebase Realtime Listeners

Backend

Runtime: Python 3.11 + FastAPI
AI Framework: Google Agent Development Kit (ADK)
LLM: Gemini 2.5 Flash, Gemini 3.0 Flash, Gemini 3.0 Pro
Voice: ElevenLabs Conversational AI + Twilio
Database: Firestore
Events: Google Cloud Pub/Sub
Scheduling: Cloud Scheduler
Analytics: BigQuery
Hosting: Cloud Run + Cloud Functions
Secrets: Google Secret Manager

ADK Patterns Implemented

✅ Sequential Agent Orchestration
✅ Parallel Fan-out/Gather
✅ Review/Critique Pattern
✅ LoopAgent (Iterative Monitoring)
✅ Human-in-the-Loop (HITL) Gating
✅ Google Search Grounding
✅ Custom FunctionTools (10 tools)

📋 Prerequisites

Node.js 18+
Python 3.11+
Google Cloud Project with:
- Firestore enabled
- Cloud Run enabled
- Cloud Functions enabled
- Pub/Sub enabled
- BigQuery enabled
- Secret Manager enabled
- Gmail API enabled
- Google Calendar API enabled
ElevenLabs account with Conversational AI agents
Twilio account (for phone calls)
Firebase project for authentication

🚀 Quick Start

1. Clone and Setup

git clone https://github.com/faithogun/oncydra.git
cd oncydra

2. Frontend Setup

# Install dependencies
npm install

# Create environment file
cp .env.example .env.local

# Add your values to .env.local (see Environment Variables section)

# Run development server
npm run dev

3. Backend Setup

Each service can be run locally for development:

cd backend/symptom-service

# Create virtual environment
python -m venv venv
source venv/bin/activate  # or `venv\Scripts\activate` on Windows

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export GOOGLE_CLOUD_PROJECT=oncydra
export GOOGLE_API_KEY=your_google_api_key

# Run locally
uvicorn main:app --port 8080

4. Deploy to Cloud Run

See Deployment Guide below for full deployment commands.

🔧 Environment Variables

Frontend (.env.local)

Create a .env.local file in the project root:

# Google API (for Gemini)
GOOGLE_API_KEY=your_google_api_key

# ElevenLabs Voice Agent IDs
VITE_ELEVENLABS_AGENT_ID=agent_xxx              # Patient Companion (Check-in)
VITE_ELEVENLABS_TRIAGE_AGENT_ID=agent_xxx       # Triage Assistant
VITE_ELEVENLABS_TIMELINE_AGENT_ID=agent_xxx     # Timeline Assistant
VITE_ELEVENLABS_REHEARSAL_AGENT_ID=agent_xxx    # Mock Rehearsal
VITE_ELEVENLABS_CARE_ADVOCATE_ID=agent_xxx      # Care Advocate (Phone Calls)
VITE_ELEVENLABS_SUMMARY_AGENT_ID=agent_xxx      # Summary Reader (TTS)

# Google Calendar OAuth
VITE_GOOGLE_CALENDAR_CLIENT_ID=your_client_id.apps.googleusercontent.com

Note: Firebase configuration is in firebase.ts. ElevenLabs API keys are stored in Google Secret Manager and accessed by backend services only.

Backend Services (Secret Manager)

All sensitive values are stored in Google Secret Manager and injected at deployment:

Secret Name	Used By	Description
`GOOGLE_API_KEY`	All services	Gemini API key
`ELEVENLABS_API_KEY`	symptom, timeline, communication, reminder	ElevenLabs API key
`GMAIL_TOKEN_JSON`	timeline, consultation, reminder	Gmail OAuth token (JSON)
`TWILIO_ACCOUNT_SID`	communication	Twilio account SID
`TWILIO_AUTH_TOKEN`	communication	Twilio auth token
`TWILIO_PHONE_NUMBER`	communication	Twilio phone number
`ELEVENLABS_PHONE_NUMBER_ID`	communication	ElevenLabs phone number ID
`DEMO_PHONE_NUMBER`	communication	Demo recipient phone number

🚢 Deployment

Deploy All Services

Each service is deployed to Cloud Run with its required environment variables and secrets:

Symptom Service

cd backend/symptom-service
gcloud run deploy symptom-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra,GOOGLE_CLOUD_LOCATION=us-central1" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest,ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest"

Timeline Service

cd backend/timeline-service
gcloud run deploy timeline-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest,ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest,GMAIL_TOKEN_JSON=GMAIL_TOKEN_JSON:latest"

Education Service

cd backend/education-service
gcloud run deploy education-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra,GOOGLE_CLOUD_LOCATION=europe-west1" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest"

Advocacy Service

cd backend/advocacy-service
gcloud run deploy advocacy-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest,ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest"

Consultation Service

cd backend/consultation-service
gcloud run deploy consultation-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra,GOOGLE_CLOUD_LOCATION=us-central1,GOOGLE_GENAI_USE_VERTEXAI=true" \
  --set-secrets "ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest,GMAIL_TOKEN_JSON=GMAIL_TOKEN_JSON:latest"

Reminder Service

cd backend/reminder-service
gcloud run deploy reminder-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra,GOOGLE_CLOUD_LOCATION=us-central1" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest,ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest,GMAIL_TOKEN_JSON=GMAIL_TOKEN_JSON:latest"

Communication Service

cd backend/communication-service
gcloud run deploy communication-service \
  --source . \
  --region europe-west1 \
  --allow-unauthenticated \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra" \
  --set-secrets "GOOGLE_API_KEY=GOOGLE_API_KEY:latest,ELEVENLABS_API_KEY=ELEVENLABS_API_KEY:latest,ELEVENLABS_PHONE_NUMBER_ID=ELEVENLABS_PHONE_NUMBER_ID:latest,TWILIO_PHONE_NUMBER=TWILIO_PHONE_NUMBER:latest,TWILIO_AUTH_TOKEN=TWILIO_AUTH_TOKEN:latest,TWILIO_ACCOUNT_SID=TWILIO_ACCOUNT_SID:latest,DEMO_PHONE_NUMBER=DEMO_PHONE_NUMBER:latest"

Analytics Cloud Function

cd backend/analytics-function
gcloud functions deploy analytics \
  --runtime python311 \
  --trigger-http \
  --allow-unauthenticated \
  --region europe-west1 \
  --set-env-vars "GOOGLE_CLOUD_PROJECT=oncydra"

📁 Project Structure

oncydra/
│   ├── components/
│   │   ├── PatientDashboard.tsx      # Main patient interface
│   │   ├── ClinicianDashboard.tsx    # Clinician interface
│   │   ├── Home.tsx                  # Landing page
│   │   └── AuthFlow.tsx              # Authentication
│   ├── contexts/                     # React contexts
│   └── firebase.ts                   # Firebase config
│
├── backend/
│   ├── symptom-service/              # 7 ADK agents
│   │   ├── agents.py
│   │   ├── tools.py
│   │   ├── main.py
│   │   └── requirements.txt
│   │
│   ├── timeline-service/             # 1 ADK agent
│   │   ├── agents.py
│   │   ├── tools.py
│   │   └── main.py
│   │
│   ├── education-service/            # 5 ADK agents
│   │   ├── agents.py
│   │   └── main.py
│   │
│   ├── advocacy-service/             # 4 ADK agents
│   │   ├── agents.py
│   │   └── main.py
│   │
│   ├── consultation-service/         # 5 ADK agents + HITL
│   │   ├── agents.py
│   │   └── main.py
│   │
│   ├── reminder-service/             # 4 ADK agents 
│   │   ├── agents.py
│   │   ├── email_templates.py
│   │   └── main.py
│   │
│   ├── communication-service/        # ElevenLabs + Twilio
│   │   └── main.py
│   │
│   └── analytics-function/           # BigQuery Cloud Function
│       └── main.py
│
├── public/
│   ├── architecture-1.png            # Patient Dashboard diagram
│   ├── architecture-2.png            # Timeline & Communication diagram
│   └── architecture-3.png            # Clinician Dashboard diagram
│
├── .env.example                      # Environment template
├── package.json
└── README.md

📊 Data Flow

Firestore Collections

Collection	Purpose
`users`	Patient/clinician profiles
`users/{id}/checkIns`	Voice check-in transcripts and extracted symptoms
`users/{id}/timeline`	Appointments and treatment events
`users/{id}/education`	Generated education content
`consultations`	Clinical consultation records with HITL status

BigQuery Tables

Table	Purpose
`oncydra_analytics.symptom_trends`	Aggregated population-level symptom data

Pub/Sub Topics

Topic	Trigger	Subscriber
`checkin-completed`	After symptom check-in saved	Reminder Service
`timeline-updated`	After timeline event created	Reminder Service

📊 Platform Statistics

Metric	Count
ADK Agents	26
ElevenLabs Voice Agents	5
Backend Services	8 (7 Cloud Run + 1 Cloud Function)
Custom FunctionTools	10
ElevenLabs Webhook Tools	4
Architectural Patterns	7

🔐 Security & Privacy

Authentication: Firebase Auth with role-based access (patient/clinician)
Secrets: All API keys stored in Google Secret Manager
HITL Controls: AI-generated clinical content requires clinician approval before reaching patients
Data Encryption: All data encrypted in transit (TLS) and at rest (Firestore)
HIPAA Considerations: Designed with healthcare privacy principles in mind
No Frontend Secrets: ElevenLabs API keys never exposed to frontend

🎬 Demo

Live Demo: oncydra.web.app

Video Walkthrough: YouTube

🏆 Built For

AI Partner Catalyst: Accelerate Innovation Hackathon — ElevenLabs Challenge Track

👩‍💻 Author

Faith Ogundimu
PhD Student in Cancer Genomics | AI & Health Tech

LinkedIn: faith-ogundimu
GitHub: faith-ogun

📄 License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

🎙️ Built with ❤️ for cancer patients and their care teams

26 agents. 8 services. 5 voice personas.
But at its core, Oncydra is just one thing: a companion that listens when patients need it most.

Name		Name	Last commit message	Last commit date
Latest commit History 31 Commits
backend		backend
components		components
contexts		contexts
public		public
services		services
utils		utils
.firebaserc		.firebaserc
.gitignore		.gitignore
App.tsx		App.tsx
LICENSE		LICENSE
README.md		README.md
env.example		env.example
firebase.json		firebase.json
index.html		index.html
index.tsx		index.tsx
metadata.json		metadata.json
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
vite.config.ts		vite.config.ts

License

faith-ogun/Oncydra

Folders and files

Latest commit

History

Repository files navigation

🎙️ Oncydra — Voice-First AI Cancer Care Platform

🎯 Project Overview

🏗️ Architecture

Architecture Diagram 1 — Patient Dashboard (Symptom, Education, Advocacy, Reminder Services)

Architecture Diagram 2 — Patient Dashboard (Timeline & Communication Services)

Architecture Diagram 3 — Clinician Dashboard (Consultation & Analytics Services)

🔧 Core Services

🤖 Agent Workflow

✨ Key Features

For Patients

For Clinicians

ElevenLabs Voice Agents

🛠️ Technology Stack

Frontend

Backend

ADK Patterns Implemented

📋 Prerequisites

🚀 Quick Start

1. Clone and Setup

2. Frontend Setup

3. Backend Setup

4. Deploy to Cloud Run

🔧 Environment Variables

Frontend (.env.local)

Backend Services (Secret Manager)

🚢 Deployment

Deploy All Services

Symptom Service

Timeline Service

Education Service

Advocacy Service

Consultation Service

Reminder Service

Communication Service

Analytics Cloud Function

📁 Project Structure

📊 Data Flow

Firestore Collections

BigQuery Tables

Pub/Sub Topics

📊 Platform Statistics

🔐 Security & Privacy

🎬 Demo

🏆 Built For

👩‍💻 Author

📄 License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages