Skip to content
/ StepGuide Public

AI Assistant with Google ADK, Pinecone RAG, and Next.js - Real-time voice/text chat with context-aware responses

License

MIT license
0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

ashwinkrishna979/StepGuide

BranchesTags

Repository files navigation

StepGuide - AI Assistant with Google ADK & Pinecone

A full-stack AI assistant application built with Google's Agent Development Kit (ADK), Pinecone vector database, and Next.js. This project demonstrates real-time voice/text interaction with AI agents, context-aware responses using RAG (Retrieval-Augmented Generation), and modern authentication.

🌟 Features

  • Real-time AI Voice & Text Chat: Bidirectional streaming with Google's Gemini Live API
  • Context-Aware Responses: RAG implementation using Pinecone vector database
  • Google Search Integration: Agent can search the web for real-time information
  • Multi-modal Support: Handle both text and images in documents
  • User Authentication: Firebase authentication with secure session management
  • Modern UI: Built with Next.js 15, React 19, and Tailwind CSS
  • WebSocket Communication: Real-time bidirectional streaming between client and server

🏗️ Architecture

┌─────────────────┐
│   Next.js       │
│   Frontend      │ ← User Interface (React 19 + Tailwind)
└────────┬────────┘
         │
         ├─ Firebase Auth (Authentication)
         ├─ WebSocket (Real-time ADK streaming)
         │
┌────────▼────────┐
│  FastAPI        │
│  ADK Server     │ ← Google ADK Agent + WebSocket
└────────┬────────┘
         │
         ├─ Google Gemini API (AI Model)
         ├─ Google Search (Web search)
         │
┌────────▼────────┐
│  FastAPI        │
│  Pinecone API   │ ← Vector DB + RAG
└─────────────────┘

📋 Prerequisites

  • Python 3.10+ (for backend services)
  • Node.js 18+ (for frontend)
  • Google Cloud Account with Gemini API access
  • Pinecone Account (free tier available)
  • Firebase Project (for authentication)

🚀 Quick Start

1. Clone the Repository

git clone https://github.com/ashwinkrishna979/pipadk.git
cd pipadk

2. Set Up Environment Variables

Root Directory (.env)

cp .env.example .env
# Edit .env and add your configuration

Required variables:

  • DEMO_AGENT_MODEL: Gemini model name (default provided)
  • PINECONE_API_URL: URL to your Pinecone API service

Pinecone API (pinecone-api/.env)

cd pinecone-api
cp .env.example .env
# Edit .env and add your Pinecone API key

Get your Pinecone API key from: https://app.pinecone.io/

Frontend (frontend/.env.local)

cd frontend
cp .env.example .env.local
# Edit .env.local and add your Firebase and API configurations

Firebase configuration can be found in your Firebase Console:

  • Project Settings → General → Your apps → SDK setup and configuration

3. Install Dependencies

Backend (ADK Agent)

# From root directory
pip install -e .
# or use uv for faster installation
uv pip install -e .

Pinecone API Service

cd pinecone-api
pip install -e .
# or
uv pip install -e .

Frontend

cd frontend
npm install
# or
yarn install

4. Run the Services

You'll need 3 terminal windows:

Terminal 1: Pinecone API Service

cd pinecone-api
python -m app.app
# Runs on http://localhost:8000

Terminal 2: ADK Agent Service

cd app
python main.py
# Runs on http://localhost:8001 (WebSocket)

Terminal 3: Frontend

cd frontend
npm run dev
# Runs on http://localhost:3000

5. Access the Application

Open your browser and navigate to:

📚 Project Structure

pipadk/
├── app/                          # ADK Agent Service (FastAPI + WebSocket)
│   ├── main.py                   # FastAPI application entry point
│   ├── package.json              # Node.js dependencies (if any)
│   └── root_agent/               # Root agent implementation
│       ├── __init__.py
│       └── agent.py              # Agent definition with tools
│
├── pinecone-api/                 # Pinecone Vector DB Service
│   ├── pyproject.toml            # Python dependencies
│   ├── README.md                 # Service documentation
│   ├── app/
│   │   ├── __init__.py
│   │   ├── app.py                # FastAPI app for RAG operations
│   │   └── schema.py             # Pydantic schemas
│   └── auth/                     # Authentication utilities
│
├── frontend/                     # Next.js 15 Frontend
│   ├── package.json              # Node.js dependencies
│   ├── next.config.ts            # Next.js configuration
│   ├── tsconfig.json             # TypeScript configuration
│   ├── tailwind.config.ts        # Tailwind CSS configuration
│   ├── app/                      # App router pages
│   │   ├── layout.tsx            # Root layout
│   │   ├── page.tsx              # Home page
│   │   ├── globals.css           # Global styles
│   │   ├── dashboard/            # Document upload & management
│   │   │   └── page.tsx
│   │   ├── live-assistant/       # Enhanced voice assistant with ADK
│   │   │   ├── page.tsx
│   │   │   ├── ADKStreamingClient.ts
│   │   │   ├── AudioPlayer.ts
│   │   │   ├── types.ts
│   │   │   ├── components.tsx
│   │   │   └── LiveAssistantLayout.tsx
│   │   ├── live-test-interface/  # Full-featured voice assistant test UI
│   │   │   ├── page.tsx
│   │   │   ├── ADKStreamingClient.ts
│   │   │   ├── AudioPlayer.ts
│   │   │   └── types.ts
│   │   ├── example/              # Example page template
│   │   │   └── page.tsx
│   │   ├── login/                # Login page
│   │   │   └── page.tsx
│   │   └── signup/               # Sign up page
│   │       └── page.tsx
│   ├── components/               # Reusable React components
│   │   ├── Navbar.tsx            # Navigation bar
│   │   ├── auth/                 # Authentication components
│   │   │   ├── auth-card.tsx
│   │   │   ├── login-form.tsx
│   │   │   ├── logout.tsx
│   │   │   └── signup-form.tsx
│   │   └── ui/                   # shadcn/ui components
│   │       ├── button.tsx
│   │       ├── card.tsx
│   │       ├── form.tsx
│   │       ├── input.tsx
│   │       ├── label.tsx
│   │       └── toast.tsx
│   ├── hooks/                    # Custom React hooks
│   │   └── use-toast.ts
│   ├── lib/                      # Utility libraries
│   │   ├── firebase.ts           # Firebase configuration
│   │   ├── liveapi-client.ts     # ADK WebSocket client
│   │   └── utils.ts              # Utility functions
│   ├── public/                   # Static assets
│   ├── config.ts                 # Application configuration
│   └── proxy.ts                  # Authentication middleware
│
├── .env.example                  # Example environment variables
├── pyproject.toml                # Root Python project configuration
├── setup-env.sh                  # Environment setup helper script
├── install_optimizations.sh      # Installation optimization script
├── README.md                     # This file
├── CONTRIBUTING.md               # Contribution guidelines
├── SECURITY.md                   # Security best practices
├── LICENSE                       # MIT License
├── PRE_PUBLICATION_CHECKLIST.md  # Pre-publication checklist
├── PERFORMANCE_OPTIMIZATIONS.md  # Performance optimization notes
└── EVENT_CALLBACK_FLOW.md        # Event callback flow documentation

🔧 Configuration Details

Environment Variables

ADK Agent Service

  • DEMO_AGENT_MODEL: Gemini model to use (e.g., gemini-2.5-flash-native-audio-preview-12-2025)
  • PINECONE_API_URL: URL to Pinecone API service (default: http://localhost:8000)
  • GOOGLE_APPLICATION_CREDENTIALS: Path to Google Cloud service account JSON (if using Vertex AI)

Pinecone Service

Frontend

  • Firebase Config (from Firebase Console):

    • NEXT_PUBLIC_FIREBASE_API_KEY
    • NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN
    • NEXT_PUBLIC_FIREBASE_PROJECT_ID
    • NEXT_PUBLIC_FIREBASE_APP_ID
    • NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID

  • Firebase Admin SDK (for server-side):

    • FIREBASE_ADMIN_CLIENT_EMAIL
    • FIREBASE_ADMIN_PRIVATE_KEY

  • Cookie Authentication:

    • AUTH_COOKIE_NAME: Cookie name for auth token
    • AUTH_COOKIE_SIGNATURE_KEY_CURRENT: Secret key for signing cookies (min 32 chars)
    • AUTH_COOKIE_SIGNATURE_KEY_PREVIOUS: Previous secret key for rotation
    • USE_SECURE_COOKIES: Set to true in production

  • API URLs:

    • NEXT_PUBLIC_PINECONE_API_URL: Pinecone API URL (default: http://localhost:8000)
    • NEXT_PUBLIC_ADK_WS_URL: ADK WebSocket URL (default: ws://localhost:8001)

Generating Cookie Signature Keys

# Generate random 32+ character strings
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"

🎯 Key Features Explained

1. Live Voice Assistant

  • Real-time voice interaction with AI
  • Audio streaming using WebSocket
  • Native audio support with Gemini
  • Located at /live-assistant

2. RAG with Pinecone

  • Upload documents (PDF, text, images)
  • Vector embeddings for semantic search
  • Context-aware responses using retrieved information
  • Multi-modal document support

3. Google Search Integration

  • Agent can search the web in real-time
  • Provides up-to-date information
  • Integrated seamlessly with RAG

4. Firebase Authentication

  • Secure user registration and login
  • Session management with HTTP-only cookies
  • Protected routes and API endpoints

🛠️ Development

Running Tests

# Backend tests
pytest

# Frontend tests (if configured)
cd frontend
npm test

Code Formatting

# Python
black .
pylint app/

# TypeScript/JavaScript
cd frontend
npm run lint

Building for Production

Frontend

cd frontend
npm run build
npm start

📝 API Documentation

ADK WebSocket Endpoint

  • URL: ws://localhost:8001/ws/{user_id}/{session_id}
  • Protocol: WebSocket bidirectional streaming
  • Events: See ADK documentation for event types

Pinecone API Endpoints

  • POST /chat_response/: Query the RAG system
  • POST /upload/: Upload files to vector database
  • GET /list_files/: List uploaded files
  • DELETE /delete_file/{file_id}: Delete a file

🤝 Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • Google ADK: Agent Development Kit for building AI agents
  • Pinecone: Vector database for semantic search
  • Firebase: Authentication and backend services
  • Next.js: React framework for production
  • shadcn/ui: Beautiful and accessible components

📧 Contact

Ashwin Mariyathu Krishnakumar

🔗 Useful Links

⚠️ Important Notes

  • API Keys: Never commit your .env files to version control
  • Production: Set USE_SECURE_COOKIES=true and use HTTPS in production
  • Costs: Be aware of API usage costs for Google Cloud, Pinecone, and Firebase
  • Rate Limits: Implement rate limiting for production deployments

🚧 Roadmap

  • Add authentication layer in backend and pinecone API
  • Add per-user segregation for document storage
  • Add user dashboard with usage statistics
  • Implement conversation history
  • Add support for more document types
  • Deploy to cloud platforms (Vercel, Google Cloud)
  • Add comprehensive test coverage
  • Improve caching layer for improved performance

About

AI Assistant with Google ADK, Pinecone RAG, and Next.js - Real-time voice/text chat with context-aware responses

Topics

python firebase typescript ai mcp nextjs gemini voice-assistant pinecone wesocket rag fast-api llm agentic-ai

Resources

Readme

License

MIT license

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages