Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
68a9034
feat: Add GitHub Actions workflow for automated EC2 deployment
moarshy Feb 22, 2025
512185b
docs: Add comprehensive deployment documentation for GitHub Actions w…
moarshy Feb 22, 2025
c3f68e1
fix: Improve EC2 deployment workflow with dynamic host and branch sel…
moarshy Feb 22, 2025
a728e9d
fix: Remove unnecessary git stash pop in deployment workflow
moarshy Feb 22, 2025
1908975
fix: Add sudo to git stash in deployment workflow
moarshy Feb 22, 2025
49c1f58
fix: Update health check response message
moarshy Feb 22, 2025
e4cd532
fix status for ws and wss
moarshy Feb 22, 2025
5e05334
docs: Update deployment documentation and workflow for main branch de…
moarshy Feb 22, 2025
0f497c8
docs: Update deployment documentation for main branch trigger
moarshy Feb 22, 2025
954d62d
chore: Update deployment workflow to trigger on main branch
moarshy Feb 22, 2025
b9529a4
feat: Add GitHub Actions workflow for A100 instance deployment
moarshy Feb 22, 2025
b17569e
fix: Improve SSH configuration and deployment workflow for A100 instance
moarshy Feb 22, 2025
9a16486
chore: Add debug logging to A100 deployment workflow
moarshy Feb 22, 2025
231bb41
refactor: Improve A100 deployment workflow SSH configuration
moarshy Feb 22, 2025
3438dfd
refactor: Streamline A100 deployment workflow with direct git and doc…
moarshy Feb 22, 2025
e9e5992
feat: Enhance A100 deployment workflow with environment and Python setup
moarshy Feb 22, 2025
8a2a04a
feat: Optimize A100 deployment script with refined environment and ex…
moarshy Feb 22, 2025
31173e0
feat: Refine A100 deployment workflow with improved environment handl…
moarshy Feb 22, 2025
4df1126
feat: Update A100 deployment workflow to fetch from feat/deploy-actio…
moarshy Feb 22, 2025
9786026
feat: Add EC2 deployment workflow with multi-instance support
moarshy Feb 23, 2025
f26f33b
chore: Update A100 deployment workflow to target main branch
moarshy Feb 23, 2025
0d45a22
docs: Comprehensive update to deployment documentation
moarshy Feb 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 161 additions & 0 deletions .github/deploy-readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# GitHub Actions Deployment Documentation

This repository contains deployment workflows for both EC2 and A100 instances. The deployment process is automated through GitHub Actions and triggers on pushes or merged pull requests to the main branch.

## Prerequisites

### Deployment Path
All instances must use the standardized deployment path:
```
/home/ubuntu/naptha/naptha-node
```

### Server Requirements
- Ubuntu OS (recommended: Ubuntu 22.04 LTS)
- Docker and Docker Compose installed
- For A100 instances: Python with Miniforge environment
- Proper permissions set up for deployment path

## Initial Server Setup

### Create deployment directory:
```bash
mkdir -p /home/ubuntu/naptha/naptha-node
chown -R ubuntu:ubuntu /home/ubuntu/naptha
```

### Install dependencies:
```bash
# Docker and Docker Compose
sudo apt update
sudo apt install docker.io docker-compose
```

## SSH Authentication Setup

### For EC2 Instances (.pem method)
- Use your existing EC2 .pem key
- Add the content to GitHub Secrets as `EC2_SSH_KEY`

### For A100 Instances (SSH key method)

1. Generate deployment SSH key pair:
```bash
ssh-keygen -t ed25519 -C "github-actions-deploy"
# Save as 'github-actions-deploy'
```

2. Add public key to authorized_keys on each A100 instance:
```bash
cat github-actions-deploy.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
```

## GitHub Secrets Configuration

### Required Secrets for EC2
- `EC2_SSH_KEY`: The .pem file content
- `EC2_HOST_1`: First EC2 instance hostname/IP
- `EC2_HOST_2`: Second EC2 instance hostname/IP
- `DEPLOY_PATH`: `/home/ubuntu/naptha/naptha-node`

### Required Secrets for A100
- `SSH_PRIVATE_KEY`: Content of github-actions-deploy private key
- `SSH_KNOWN_HOSTS`: Content of generated known_hosts entries
- `A100_HOST_1`: First A100 instance hostname/IP
- `A100_HOST_2`: Second A100 instance hostname/IP
- `A100_USER`: SSH username for A100 instances
- `A100_DEPLOY_PATH`: `/home/ubuntu/naptha/naptha-node`

## Repository Requirements

### Required Files
- `docker-compose.yml`: Docker services configuration
- `docker-ctl.sh`: Docker service management script
- `launch.sh`: Main deployment script

### File Permissions
```bash
chmod +x docker-ctl.sh launch.sh
```

## Workflow Operation

### Trigger Conditions
- Push to main branch
- Merged pull request to main branch

### Deployment Process

#### EC2 Deployment
- SSH connection using .pem key
- Code checkout and deployment
- Service restart

#### A100 Deployment
- SSH setup with generated keys
- Python environment activation
- Docker-based deployment:
- Build containers
- Service management
- Launch application

## Monitoring and Troubleshooting

### Deployment Status
- Monitor in GitHub Actions tab
- Each instance shows separate status
- Success/failure indicators for each step

### Common Issues and Solutions

#### SSH Connection Failed
```bash
# Check SSH connection manually
ssh -i your-key.pem ubuntu@EC2_HOST
# or
ssh -i github-actions-deploy A100_USER@A100_HOST
```

#### Python Not Found (A100)
```bash
# Verify Python installation
which python
python --version
```

#### Docker Issues
```bash
# Check Docker status
sudo systemctl status docker
# Check Docker Compose
docker-compose --version
```

### Logging
- GitHub Actions provides detailed logs
- Instance-specific logs in `/var/log`
- Docker logs via `docker-ctl.sh logs`

## Security Considerations

### SSH Key Management
- Keys stored as GitHub Secrets
- Proper file permissions (600)
- Regular key rotation recommended

### Access Control
- Minimal required permissions
- Separate keys for different environments
- No sensitive data in repository

## Additional Notes

### Deployment Path Structure
```
/home/ubuntu/naptha/naptha-node/
├── docker-compose.yml
├── docker-ctl.sh
├── launch.sh
└── [other application files]
```
91 changes: 91 additions & 0 deletions .github/workflows/deploy-a100.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Deploy Application

on:
push:
branches:
- main
pull_request:
types:
- closed
branches:
- main

jobs:
deploy:
if: github.event.pull_request.merged == true || github.event_name == 'push'
runs-on: ubuntu-latest
strategy:
matrix:
instance: [1, 2]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup SSH Config
run: |
mkdir -p ~/.ssh
echo "${{ secrets.SSH_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts

- name: Setup SSH Key
uses: webfactory/ssh-agent@v0.8.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}

- name: Deploy to A100 Instance ${{ matrix.instance }}
env:
HOST: ${{ matrix.instance == 1 && secrets.A100_HOST_1 || secrets.A100_HOST_2 }}
DEPLOY_USER: ${{ secrets.A100_USER }}
DEPLOY_PATH: ${{ secrets.A100_DEPLOY_PATH }}
run: |
ssh -o StrictHostKeyChecking=no $DEPLOY_USER@$HOST "DEPLOY_PATH='${DEPLOY_PATH}' bash -s" << 'EOF'
# Load environment and Python
export PATH="/home/$USER/miniforge3/bin:$PATH"
source ~/.bashrc

# Explicitly activate conda environment
source /home/$USER/miniforge3/bin/activate

# Check Python and pip
echo "Python version:"
python --version
echo "Python location:"
which python
echo "Pip location:"
which pip

# Navigate and setup deployment
sudo chown -R $DEPLOY_USER:$DEPLOY_USER $DEPLOY_PATH
cd $DEPLOY_PATH

# Git operations
sudo git stash
git fetch origin feat/deploy-actions
git reset --hard origin/feat/deploy-actions

# Install required Python packages if needed
if [ -f requirements.txt ]; then
pip install -r requirements.txt
fi

# Make scripts executable
chmod +x ./docker-ctl.sh

# Your deployment commands
docker compose -f docker-compose.yml build --no-cache
./docker-ctl.sh down

# Run launch.sh with Python environment preserved
sudo -E PATH=$PATH bash launch.sh
echo "Deployment completed at $(date)"
EOF

- name: Deployment Status
if: always()
run: |
if [ "${{ job.status }}" == "success" ]; then
echo "✅ Deployment successful to A100 Instance ${{ matrix.instance }}"
else
echo "❌ Deployment failed for A100 Instance ${{ matrix.instance }}"
fi
59 changes: 59 additions & 0 deletions .github/workflows/deploy-ec2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Deploy Application

on:
push:
branches:
- main
pull_request:
types:
- closed
branches:
- main

jobs:
deploy:
if: github.event.pull_request.merged == true || github.event_name == 'push'
runs-on: ubuntu-latest
strategy:
matrix:
instance: [1, 2]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install SSH key
run: |
mkdir -p ~/.ssh
echo "${{ secrets.EC2_SSH_KEY }}" > ~/.ssh/daimon.pem
chmod 600 ~/.ssh/daimon.pem
if [ "${{ matrix.instance }}" == "1" ]; then
ssh-keyscan -H "${{ secrets.EC2_HOST_1 }}" >> ~/.ssh/known_hosts
else
ssh-keyscan -H "${{ secrets.EC2_HOST_2 }}" >> ~/.ssh/known_hosts
fi

- name: Deploy to EC2 Instance ${{ matrix.instance }}
env:
EC2_HOST: ${{ matrix.instance == 1 && secrets.EC2_HOST_1 || secrets.EC2_HOST_2 }}
DEPLOY_PATH: ${{ secrets.DEPLOY_PATH }}
run: |
ssh -i ~/.ssh/daimon.pem ubuntu@$EC2_HOST << EOF
sudo chown -R ubuntu:ubuntu $DEPLOY_PATH
cd $DEPLOY_PATH
sudo git stash
git fetch origin main
git reset --hard origin/main
sudo bash stop_service.sh
sudo bash launch.sh
echo "Deployment completed to Instance ${{ matrix.instance }} at \$(date)"
EOF

- name: Deployment Status
if: always()
run: |
if [ "${{ job.status }}" == "success" ]; then
echo "✅ Deployment successful to Instance ${{ matrix.instance }}"
else
echo "❌ Deployment failed for Instance ${{ matrix.instance }}"
fi
2 changes: 1 addition & 1 deletion launch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1754,7 +1754,7 @@ startup_summary() {
services+=("$(echo $NODE_COMMUNICATION_PROTOCOL | tr '[:lower:]' '[:upper:]')_Server_${port}")

# Health check based on server type
if [ "${NODE_COMMUNICATION_PROTOCOL}" = "ws" ]; then
if [ "${NODE_COMMUNICATION_PROTOCOL}" = "ws" -o "${NODE_COMMUNICATION_PROTOCOL}" = "wss" ]; then
# WebSocket health check using /health endpoint
if curl -s http://localhost:$port/health > /dev/null; then
statuses+=("✅")
Expand Down
2 changes: 1 addition & 1 deletion node/server/http_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ async def shutdown_event():

@self.app.get("/health")
async def health_check():
return {"status": "ok", "communication_protocol": "http"}
return {"status": "ok!!!", "communication_protocol": "http"}

# Handle validation errors when request data doesn't match the expected Pydantic models
# Logs the validation error details and request body, then returns a 422 response with the error info
Expand Down
Loading