Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -316,11 +316,26 @@ jobs:
chmod +x src/scripts/build_linux.sh
./src/scripts/build_linux.sh

- name: Install deb packaging tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
debhelper devscripts fakeroot dpkg-dev

- name: Build .deb package
run: |
chmod +x src/scripts/build_deb.sh
./src/scripts/build_deb.sh

- name: Upload Linux Archive as Artifact
uses: actions/upload-artifact@v7
with:
name: linux-assets
path: release/linux/*.tar.gz*
path: |
release/linux/*.tar.gz
release/linux/*.tar.gz.sha256
release/linux/*.deb
release/linux/*.deb.sha256
retention-days: 90

- name: Build Summary
Expand All @@ -333,6 +348,10 @@ jobs:
echo "" >> $GITHUB_STEP_SUMMARY
ls -lh release/linux/*.tar.gz 2>/dev/null | awk '{print "- " $9 " (" $5 ")"}' >> $GITHUB_STEP_SUMMARY || echo "No archive files found" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Deb Packages:**" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
ls -lh release/linux/*.deb 2>/dev/null | awk '{print "- " $9 " (" $5 ")"}' >> $GITHUB_STEP_SUMMARY || echo "No deb packages found" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**SHA256 Checksums:**" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
cat release/linux/*.sha256 2>/dev/null | awk '{print "- `" $1 "`"}' >> $GITHUB_STEP_SUMMARY || echo "No checksums found" >> $GITHUB_STEP_SUMMARY
Expand Down Expand Up @@ -463,6 +482,7 @@ jobs:
### Downloads
- **macOS**: DMG installer (Apple Silicon and Intel) - Desktop app with UI
- **Linux**: Standalone binary archive (amd64) - API server only (no UI)
- **Linux (.deb)**: Debian/Ubuntu package (amd64) with systemd unit
- **Chrome Extension**: Browser extension for PII detection in web forms

### macOS Installation
Expand Down Expand Up @@ -530,6 +550,18 @@ jobs:

See included `README.txt` for systemd service setup and advanced configuration.

### Debian / Ubuntu (.deb)

```bash
sudo dpkg -i kiji-privacy-proxy_${{ steps.version.outputs.version }}_amd64.deb
# The systemd unit is installed but not enabled by default.
# Configure /etc/default or env vars first, then:
sudo systemctl enable --now kiji-privacy-proxy
```

Files land under `/opt/kiji-privacy-proxy/` and a `kiji-proxy`
wrapper is placed on `PATH` at `/usr/bin/kiji-proxy`.

### Chrome Extension

**Install from zip:**
Expand Down Expand Up @@ -585,6 +617,8 @@ jobs:
cp release-assets/dmg-assets/latest-mac.yml assets/ 2>/dev/null || true
cp release-assets/linux-assets/*.tar.gz assets/ 2>/dev/null || true
cp release-assets/linux-assets/*.tar.gz.sha256 assets/ 2>/dev/null || true
cp release-assets/linux-assets/*.deb assets/ 2>/dev/null || true
cp release-assets/linux-assets/*.deb.sha256 assets/ 2>/dev/null || true
cp release-assets/chrome-assets/*.zip assets/ 2>/dev/null || true
cp release-assets/chrome-assets/*.zip.sha256 assets/ 2>/dev/null || true

Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ kiji-proxy
release/
build/

# Transient debian/ tree staged at the project root by build_deb.sh
# (canonical packaging files live under packaging/debian/).
/debian/

# Logs
*.log

Expand Down
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,24 @@ open Kiji-Privacy-Proxy-*.dmg
```

**Linux (Standalone Server):**

Debian / Ubuntu (.deb):
```bash
wget https://github.com/dataiku/kiji-proxy/releases/download/vX.Y.Z/kiji-privacy-proxy_X.Y.Z_amd64.deb
sudo dpkg -i kiji-privacy-proxy_X.Y.Z_amd64.deb

# The systemd unit is installed but not enabled by default.
sudo systemctl enable --now kiji-privacy-proxy
# Or run in the foreground:
kiji-proxy
```

Other distros (tarball):
```bash
# Download and extract
wget https://github.com/dataiku/kiji-proxy/releases/download/vX.Y.Z/kiji-privacy-proxy-X.Y.Z-linux-amd64.tar.gz
tar -xzf kiji-privacy-proxy-X.Y.Z-linux-amd64.tar.gz
cd kiji-privacy-proxy-X.Y.Z-linux-amd64

# Run
./run.sh
```

Expand Down Expand Up @@ -326,7 +337,8 @@ See [docs/02-development-guide.md](docs/02-development-guide.md) for detailed de
Download the latest release from [GitHub Releases](https://github.com/dataiku/kiji-proxy/releases):

- **macOS:** `Kiji-Privacy-Proxy-{version}.dmg` (~400MB)
- **Linux:** `kiji-privacy-proxy-{version}-linux-amd64.tar.gz` (~150MB)
- **Linux (tarball):** `kiji-privacy-proxy-{version}-linux-amd64.tar.gz` (~150MB)
- **Linux (Debian/Ubuntu):** `kiji-privacy-proxy_{version}_amd64.deb`

**Automated Builds:** CI/CD builds both platforms in parallel on every release tag.

Expand Down
31 changes: 30 additions & 1 deletion docs/01-getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,34 @@ See [transparent-proxy-setup.md](transparent-proxy-setup.md) for complete detail
- x86_64 architecture
- GCC runtime libraries (usually pre-installed)

**Installation Steps:**
**Quick Install (Debian / Ubuntu):**

```bash
wget https://github.com/dataiku/kiji-proxy/releases/download/v{version}/kiji-privacy-proxy_{version}_amd64.deb

# Verify checksum (optional)
wget https://github.com/dataiku/kiji-proxy/releases/download/v{version}/kiji-privacy-proxy_{version}_amd64.deb.sha256
sha256sum -c kiji-privacy-proxy_{version}_amd64.deb.sha256

sudo dpkg -i kiji-privacy-proxy_{version}_amd64.deb
```

The package installs the binary under `/opt/kiji-privacy-proxy/`, places a
`kiji-proxy` wrapper on `PATH` at `/usr/bin/kiji-proxy`, and ships the
systemd unit `kiji-privacy-proxy.service` (installed but **not** enabled).

```bash
# Configure (env vars are picked up by the unit via DynamicUser/StateDirectory)
sudo systemctl edit kiji-privacy-proxy
# Add an [Service] override with Environment="OPENAI_API_KEY=..." etc.

sudo systemctl enable --now kiji-privacy-proxy
sudo systemctl status kiji-privacy-proxy
```

To uninstall: `sudo dpkg -r kiji-privacy-proxy`.

**Manual Install (other distros — tarball):**

1. **Download and Extract:**
```bash
Expand Down Expand Up @@ -511,6 +538,8 @@ You don't need to run `git tag` or `git push` — the workflow does it for you.
- `Kiji-Privacy-Proxy-1.0.1.dmg`
- `kiji-privacy-proxy-1.0.1-linux-amd64.tar.gz`
- `kiji-privacy-proxy-1.0.1-linux-amd64.tar.gz.sha256`
- `kiji-privacy-proxy_1.0.1_amd64.deb`
- `kiji-privacy-proxy_1.0.1_amd64.deb.sha256`

**9. Test the Release:**

Expand Down
42 changes: 41 additions & 1 deletion docs/03-building-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ Kiji Privacy Proxy can be built for two platforms with different deployment mode

### Linux (Standalone Binary)
- **Format:** API server binary (no UI)
- **Package:** Tarball with binary and libraries (~150-200MB)
- **Package:** Tarball (`*.tar.gz`, ~150-200MB) and Debian package
(`*.deb`, Debian/Ubuntu) with binary and libraries
- **Components:** Go backend + ML model + libraries
- **User Interface:** HTTP API only (no web UI)

Expand Down Expand Up @@ -394,6 +395,45 @@ cd ../..

**Build Time:** 8-12 minutes (first run), 3-5 minutes (cached)

### Building the Debian / Ubuntu Package (`.deb`)

The `.deb` is built by `src/scripts/build_deb.sh` using `debhelper` /
`dpkg-buildpackage`. It consumes the staging tree produced by
`build_linux.sh` (no Go/Rust toolchain runs in this step), so the Linux
tarball build must run first.

**Prerequisites (Debian/Ubuntu host):**

```bash
sudo apt-get install -y debhelper devscripts fakeroot dpkg-dev
```

**Build:**

```bash
./src/scripts/build_linux.sh # produces release/linux/<pkg>-<ver>-linux-amd64/ + tarball
./src/scripts/build_deb.sh # produces release/linux/kiji-privacy-proxy_<ver>_amd64.deb
```

`build_deb.sh` copies `packaging/debian/` to `./debian` at the project root,
regenerates `debian/changelog` from the project version, runs
`dpkg-buildpackage -b -us -uc` (binary-only, unsigned), moves the resulting
`.deb` + `.buildinfo` + `.changes` into `release/linux/`, generates a
`.sha256`, and removes the staged `debian/` on exit.

**Layout when installed (`dpkg -i`):**

| Path | Source |
|------|--------|
| `/opt/kiji-privacy-proxy/bin/kiji-proxy` | Go binary |
| `/opt/kiji-privacy-proxy/lib/libonnxruntime.so*` | Bundled ONNX Runtime |
| `/usr/bin/kiji-proxy` | Wrapper that sets `LD_LIBRARY_PATH` |
| `/lib/systemd/system/kiji-privacy-proxy.service` | systemd unit (not enabled by default) |

The canonical packaging files live under `packaging/debian/`. Edit them
there; `build_deb.sh` stages a working copy at the project root and cleans
it up on exit.

### Verification

The `verify_linux_build.sh` script checks:
Expand Down
10 changes: 7 additions & 3 deletions docs/04-release-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ The entire release process runs in a single GitHub Actions workflow:
| Job | Runner | Output |
|-----|--------|--------|
| `build-dmg` | `macos-latest` | `Kiji-Privacy-Proxy-{version}.dmg`, `*-mac.zip`, `latest-mac.yml` |
| `build-linux` | `ubuntu-latest` | `kiji-privacy-proxy-{version}-linux-amd64.tar.gz` (+ `.sha256`) |
| `build-linux` | `ubuntu-latest` | `kiji-privacy-proxy-{version}-linux-amd64.tar.gz` (+ `.sha256`), `kiji-privacy-proxy_{version}_amd64.deb` (+ `.sha256`) |
| `build-chrome` | `ubuntu-latest` | `kiji-privacy-proxy-extension-{version}.zip` (+ `.sha256`) |
| `create-release` | `ubuntu-latest` | Tags `v{version}` (PR-merge path only), publishes GitHub Release |

Expand Down Expand Up @@ -307,8 +307,12 @@ then atomically creates the release with every asset attached
1. Setup Go, Rust
2. Cache LFS, Go modules, tokenizers library, ONNX Runtime
3. Verify Git LFS model files
4. Build via `src/scripts/build_linux.sh`
5. Upload `linux-assets` artifact
4. Build via `src/scripts/build_linux.sh` (tarball + staging tree)
5. Install `debhelper devscripts fakeroot dpkg-dev`
6. Build the `.deb` via `src/scripts/build_deb.sh`, which consumes the
staging tree from step 4 and runs `dpkg-buildpackage -b -us -uc`
against `packaging/debian/`
7. Upload `linux-assets` artifact (tarball, `.deb`, and both `.sha256`s)

**Build Time:** 4–6 minutes (cached), 12–15 minutes (cold)

Expand Down
6 changes: 6 additions & 0 deletions packaging/debian/changelog
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kiji-privacy-proxy (0.0.0) unstable; urgency=medium

* Placeholder. src/scripts/build_deb.sh regenerates this file from the
project version before invoking dpkg-buildpackage; do not hand-edit.

-- Dataiku Open Source Lab <opensource@dataiku.com> Thu, 01 Jan 1970 00:00:00 +0000
23 changes: 23 additions & 0 deletions packaging/debian/control
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Source: kiji-privacy-proxy
Section: net
Priority: optional
Maintainer: Dataiku Open Source Lab <opensource@dataiku.com>
Build-Depends: debhelper-compat (= 13)
Standards-Version: 4.6.0
Homepage: https://github.com/dataiku/kiji-proxy
Vcs-Browser: https://github.com/dataiku/kiji-proxy
Vcs-Git: https://github.com/dataiku/kiji-proxy.git
Rules-Requires-Root: no

Package: kiji-privacy-proxy
Architecture: amd64
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: Privacy-preserving proxy for LLM API calls
Kiji Privacy Proxy detects and redacts personally identifiable
information in requests and responses to LLM APIs such as OpenAI.
The package ships the Go API server with an embedded ML entity
detection model and the bundled ONNX Runtime shared library.
.
This is the standalone server build (no desktop UI). Configuration
is via environment variables or a config.json file; see the README
for details.
46 changes: 46 additions & 0 deletions packaging/debian/copyright
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: kiji-privacy-proxy
Upstream-Contact: Dataiku Open Source Lab <opensource@dataiku.com>
Source: https://github.com/dataiku/kiji-proxy

Files: *
Copyright: 2024-2026 Dataiku
License: Apache-2.0

Files: build/libonnxruntime.so*
Copyright: Microsoft Corporation
License: MIT
Comment: Bundled ONNX Runtime shared library, redistributed unmodified
from https://github.com/microsoft/onnxruntime.

License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian systems, the complete text of the Apache 2.0 license can
be found in /usr/share/common-licenses/Apache-2.0.

License: MIT
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
.
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
16 changes: 16 additions & 0 deletions packaging/debian/kiji-privacy-proxy.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[Unit]
Description=Kiji Privacy Proxy
After=network.target

[Service]
Type=simple
DynamicUser=yes
StateDirectory=kiji-proxy
Environment="LD_LIBRARY_PATH=/opt/kiji-privacy-proxy/lib"
Environment="KIJI_DATA_PATH=/var/lib/kiji-proxy"
ExecStart=/opt/kiji-privacy-proxy/bin/kiji-proxy
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
6 changes: 6 additions & 0 deletions packaging/debian/kiji-proxy.wrapper
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh
# Wrapper installed at /usr/bin/kiji-proxy so users can launch the binary
# from PATH without manually setting LD_LIBRARY_PATH for the bundled
# libonnxruntime.so.
exec env LD_LIBRARY_PATH="/opt/kiji-privacy-proxy/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
/opt/kiji-privacy-proxy/bin/kiji-proxy "$@"
35 changes: 35 additions & 0 deletions packaging/debian/rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/make -f
# Build the kiji-privacy-proxy .deb from the staging tree produced by
# src/scripts/build_linux.sh — no Go/Rust toolchains are run here.

export DH_VERBOSE = 1

PKG := kiji-privacy-proxy
DESTDIR := debian/$(PKG)
PREFIX := opt/kiji-privacy-proxy

STAGED := $(firstword $(wildcard release/linux/$(PKG)-*-linux-amd64))

%:
dh $@

override_dh_auto_configure:
override_dh_auto_build:
override_dh_auto_test:

override_dh_auto_install:
@if [ -z "$(STAGED)" ]; then \
echo "ERROR: no staging tree under release/linux/." >&2; \
echo "Run src/scripts/build_linux.sh before building the deb." >&2; \
exit 1; \
fi
install -d $(DESTDIR)/$(PREFIX)/bin $(DESTDIR)/$(PREFIX)/lib $(DESTDIR)/usr/bin
install -m 0755 $(STAGED)/bin/kiji-proxy $(DESTDIR)/$(PREFIX)/bin/kiji-proxy
cp -a $(STAGED)/lib/. $(DESTDIR)/$(PREFIX)/lib/
install -m 0755 debian/kiji-proxy.wrapper $(DESTDIR)/usr/bin/kiji-proxy

override_dh_shlibdeps:
dh_shlibdeps -l$(CURDIR)/$(DESTDIR)/$(PREFIX)/lib -- --ignore-missing-info

override_dh_installsystemd:
dh_installsystemd --no-enable --no-start
1 change: 1 addition & 0 deletions packaging/debian/source/format
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.0 (native)
Loading
Loading