<<<<<<< HEAD

ClickHouse Data Ingestion Tool

A web-based application for bidirectional data ingestion between ClickHouse and Flat Files.

Prerequisites

• Python 3.8 or higher
• Node.js 14 or higher
• ClickHouse server running locally or accessible

Installation

Backend Setup

1. Navigate to the backend directory:

cd backend

2. Run the setup script (Windows PowerShell):

.\setup.ps1

3. Start the backend server:

.\venv\Scripts\Activate.ps1
uvicorn app.main:app --reload

Frontend Setup

1. Navigate to the frontend directory:

cd frontend

2. Install dependencies:

npm install

3. Start the development server:

npm start

Configuration

Backend Configuration

The backend configuration is managed through the .env file in the backend directory. Key settings include:

• ClickHouse connection details
• JWT settings
• File upload settings
• API settings

Frontend Configuration

The frontend configuration is managed through environment variables in .env file:

• REACT_APP_API_URL: Backend API URL (default: http://localhost:8000/api/v1)

Usage

1. Access the application at http://localhost:3000
2. Login with your ClickHouse credentials
3. Use the dashboard to:
   • Import data from flat files
   • Export data to flat files
   • Join tables
   • Preview data

Troubleshooting

Common Issues

1. Permission Denied Errors

   • Run PowerShell as Administrator
   • Ensure no Python processes are running
   • Delete and recreate the virtual environment

2. ClickHouse Connection Issues

   • Verify ClickHouse server is running
   • Check connection details in .env file
   • Ensure proper credentials are provided

3. Dependency Installation Issues

   • Update pip: python -m pip install --upgrade pip
   • Clear pip cache: pip cache purge
   • Try installing packages individually

Support

For additional support, please contact the development team.

bidirectional-ingestion-tool

abccf7d02738cfc10fa53b519d76f99b13588c27