Add SFTP data plane microservice with API key authentication and scoped permissions

[Copilot]

Implements SFTP file transfer service with API key authentication, scoped read/write permissions, and full audit logging. Files organized by org/user for isolation.

Core Components

• apps/shared/pkg/sftp/ - SFTP server using Go SSH/SFTP libraries

  • API key authentication via SSH password callback
  • Per-user handlers enforcing read/write scopes
  • Path traversal protection, no symlinks
  • Directory structure: /var/lib/sftp/{org_id}/{user_id}/

• apps/sftp-service/ - Microservice following existing patterns

  • SFTP server (port 2222) + HTTP health endpoint (port 3020)
  • API key validator queries database, checks scopes, updates last_used_at
  • Audit logger writes all operations to TimescaleDB

• apps/shared/pkg/database/api_keys.go - APIKey model

  • SHA-256 hashed keys, scopes stored as CSV
  • Indexes on key_hash, user_id, organization_id

API Key Scopes

// Scopes control file operations
"sftp:read"   // download, list, stat
"sftp:write"  // upload, delete, mkdir, rename  
"sftp:*"      // both read and write
"sftp"        // both read and write

Usage

# Create API key
./apps/sftp-service/scripts/create-api-key.sh "My Key" user-123 org-456 "sftp:*"

# Connect via SFTP
sftp -P 2222 user@hostname
# Password: <api-key>

Docker Configuration

• Exposed port 2222 for SFTP access
• Persistent volume sftp-data for files and host key
• Single replica in Swarm (avoid connection conflicts)
• Traefik labels for HTTP health checks only

Database Migration

Run apps/sftp-service/migrations/001_create_api_keys_table.sql to create the api_keys table with proper indexes and foreign keys.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• connectrpc.com
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• go.googlesource.com
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• go.opentelemetry.io
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• go.yaml.in
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• google.golang.org
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• gopkg.in
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• gorm.io
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)
• nhooyr.io
  • Triggering command: /update-job-proxy /update-job-proxy c mpile 03.o 5907�� tp-service/go.mod mpile pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.lin-importcfg --gdwarf-5 (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-45b5622b4a20 -j DOCKER-ISOLATION-STAGE-2 e359ba6.0 noop /home/dependabot/bin/test --root /var/run/docker/cat-file /usr/bin/readlinblob test -e /etc/ssl/certs/6b99d060.0 readlink e/git-remote-https ECC_P256_Certifi/usr/sbin/iptables /tmp/runc-proces--wait (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-00bbb1616124 -j DOCKER-ISOLATION-STAGE-2 ot_CA_2021.pem /usr/lib/git-cor/usr/share/ca-certificates/mozilla/Go_Daddy_Class_2_CA.crt /usr/bin/test 994f93e71e4039ddgit 42304cf5:go.mod /bin/git test -e 860aaa6.0 git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

look at the other microservices and use the same architecture and structure

can you make a SFTP data plane microservice?

We want proper auditing, authentication via a api key, key traces back to user for audit logs.
❤️
Click to react
🔥
Click to react
✅
Click to react
Add Reaction
Reply
Forward
More
[1:52 PM]Saturday, January 17, 2026 at 1:52 PM
User needs the scoped permission for operations, read and write are seperate scopes

use pkg/sftp

Created from VS Code via the GitHub Pull Request extension.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[veryCrunchy]

Use gorm automigrate.
Ensure the table name doesn't collide with existing tables.

[veryCrunchy]

Users need to be able to manage their SFTP keys from the dashboard.
These keys should be bound to a game server or deployment.

[veryCrunchy]

Use our existing permissions scope system.