Add RustFS as an alternative in-cluster storage backend#1054
Open
cypres wants to merge 2 commits into
Open
Conversation
Adds `--storage-backend rustfs`: an in-cluster, S3-compatible object store (rustfs.com) as a drop-in alternative to MinIO, installed via the official Helm chart (https://charts.rustfs.com). - install-rustfs.sh: standalone-mode helm install. Always sets the perf-critical RUSTFS_OBS_ENVIRONMENT=production and RUSTFS_OBS_LOGGER_LEVEL=warn (chart's config.rustfs.obs_environment / log_level), and runs with no resource limits (resources: {}). Generates non-default credentials (the chart rejects the well-known rustfsadmin default). - storage/rustfs.sh: credential discovery + bucket bootstrap + workflow-cred Secrets + Helm values fragment, mirroring storage/minio.sh. Bucket creation uses the vendor-neutral AWS CLI (Apache-2.0) hitting the S3 API directly, not MinIO's mc (AGPLv3); path-style addressing, idempotent via head-bucket. - MinIO exclusivity: RustFS and MinIO are mutually exclusive. RustFS never installs or adds MinIO (microk8s/install.sh gates the minio addon on the minio/auto backends only); an already-installed MinIO is left untouched. - configure-storage.sh / deploy-osmo-minimal.sh: backend wiring, help text, and workload-identity rejection for rustfs. - README: storage matrix, install-helper table, phases, and flags. Also lands the storage/ Bazel test scaffolding (BUILD + minio/rustfs addressing-style sh_tests), which was previously untracked. Validated: chart renders via `helm template`; bucket create/idempotency and object round-trip verified against a live RustFS container using aws-cli. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Hans Arnholm <harnholm@nvidia.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThis PR adds RustFS as a first-class in-cluster S3-compatible storage backend alternative to MinIO in OSMO minimal deployments. Changes include docs and flag updates, a Helm-based RustFS installer, rustfs storage wiring, deployment orchestration to conditionally enable MinIO, and tests for addressing-style handling. ChangesRustFS Storage Backend Integration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
deployments/scripts/deploy-osmo-minimal.sh (1)
139-147:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate auth help text to include
rustfsas unsupported for workload identity.Current help says workload identity is not valid for
minioonly, but runtime behavior rejectsrustfstoo.Proposed fix
- Not valid for --storage-backend minio. + Not valid for --storage-backend minio|rustfs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@deployments/scripts/deploy-osmo-minimal.sh` around lines 139 - 147, Update the help text for the --auth-method flag to state that workload-identity is not valid for both minio and rustfs (in addition to existing note about minio); modify the description near the --auth-method and --storage-backend help block so it explicitly lists "not valid for --storage-backend minio or rustfs" (referencing the --auth-method flag and --storage-backend option in deploy-osmo-minimal.sh) to match runtime behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@deployments/scripts/configure-storage.sh`:
- Line 33: Update the usage/help string that documents the --backend options so
it includes "s3" again; specifically modify the line that currently reads "#
--backend {auto|minio|rustfs|azure-blob|byo|none} Backend (default: auto)" to
include "s3" in the brace list (e.g.,
"{auto|s3|minio|rustfs|azure-blob|byo|none}") so the usage matches the script
behavior and other help output.
In `@deployments/scripts/storage/rustfs.sh`:
- Line 88: The AWS_CLI_IMAGE default uses the floating tag
amazon/aws-cli:latest; update the AWS_CLI_IMAGE variable in the rustfs.sh script
to a pinned image (preferably an immutable digest) instead of :latest, and
ensure the kubectl run/invocation that references AWS_CLI_IMAGE uses that pinned
value; locate the AWS_CLI_IMAGE assignment and the kubectl run command in
rustfs.sh and replace the default value with a specific tag or digest (e.g.,
amazon/aws-cli@sha256:...) so the bootstrap uses a fixed, non-floating image.
- Around line 91-106: The bucket-setup pod currently interpolates sensitive
values directly into the /bin/sh -c string (export
AWS_ACCESS_KEY_ID='$RUSTFS_USER' ... and unquoted
$RUSTFS_ENDPOINT_URL/$RUSTFS_BUCKET) and leaves $KUBECTL unquoted; instead, stop
inline shell interpolation by passing credentials and values via kubectl run's
environment flags or mounting the secret—add --env
AWS_ACCESS_KEY_ID="$RUSTFS_USER" --env AWS_SECRET_ACCESS_KEY="$RUSTFS_PASS"
--env AWS_DEFAULT_REGION=us-east-1 --env
RUSTFS_ENDPOINT_URL="$RUSTFS_ENDPOINT_URL" --env RUSTFS_BUCKET="$RUSTFS_BUCKET"
(or mount the secret) and update the pod command to reference those env vars
without injecting values into the command string, and ensure the $KUBECTL
variable is quoted in the timeout invocation so the timeout
"$BUCKET_SETUP_TIMEOUT" "$KUBECTL" run ... call is safe.
In `@deployments/scripts/storage/tests/minio_addressing_style_test.sh`:
- Around line 65-73: The test's env block can leak a runner-level
MINIO_ADDRESSING_STYLE into the default-case run; update the environment passed
to bash "$MINIO_SCRIPT" in the test so MINIO_ADDRESSING_STYLE is explicitly
isolated (e.g., set MINIO_ADDRESSING_STYLE= or unset it in the env list) before
invoking MINIO_SCRIPT to ensure the default-path assertion is deterministic;
modify the env invocation that currently sets PATH, KUBECTL, KUBECTL_LOG,
NAMESPACE, OUTPUT_VALUES, MINIO_ROOT_USER and MINIO_ROOT_PASSWORD to also clear
or unset MINIO_ADDRESSING_STYLE.
In `@deployments/scripts/storage/tests/rustfs_addressing_style_test.sh`:
- Around line 65-73: The test can inherit RUSTFS_ADDRESSING_STYLE from the
environment making assertions flaky; modify the env block that calls bash
"$RUSTFS_SCRIPT" to explicitly unset or set RUSTFS_ADDRESSING_STYLE='' (or
RUSTFS_ADDRESSING_STYLE=default) in the invocation so the default path is
deterministic—update the env lines where PATH, KUBECTL, NAMESPACE,
OUTPUT_VALUES, RUSTFS_ACCESS_KEY, RUSTFS_SECRET_KEY are exported alongside
RUSTFS_ADDRESSING_STYLE to ensure the script run via bash "$RUSTFS_SCRIPT" sees
the intended addressing style.
---
Outside diff comments:
In `@deployments/scripts/deploy-osmo-minimal.sh`:
- Around line 139-147: Update the help text for the --auth-method flag to state
that workload-identity is not valid for both minio and rustfs (in addition to
existing note about minio); modify the description near the --auth-method and
--storage-backend help block so it explicitly lists "not valid for
--storage-backend minio or rustfs" (referencing the --auth-method flag and
--storage-backend option in deploy-osmo-minimal.sh) to match runtime behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8cc92bf9-dac1-414a-9484-62ce3046b5e7
📒 Files selected for processing (9)
deployments/scripts/README.mddeployments/scripts/configure-storage.shdeployments/scripts/deploy-osmo-minimal.shdeployments/scripts/install-rustfs.shdeployments/scripts/microk8s/install.shdeployments/scripts/storage/BUILDdeployments/scripts/storage/rustfs.shdeployments/scripts/storage/tests/minio_addressing_style_test.shdeployments/scripts/storage/tests/rustfs_addressing_style_test.sh
- configure-storage.sh: restore s3 in the --backend usage brace list. - storage/rustfs.sh: pin the bucket-bootstrap image to an immutable, multi-arch manifest-list digest (amazon/aws-cli:2.31.10@sha256:c354...) instead of the floating :latest; pass credentials/endpoint/bucket via kubectl run --env so no secrets are interpolated into the /bin/sh command string, and quote $KUBECTL in the timeout invocation. - storage tests: clear inherited *_ADDRESSING_STYLE (and the STORAGE_ADDRESSING_STYLE fallback) via 'env -u' in the default-path run so the assertion is deterministic regardless of the runner environment. - deploy-osmo-minimal.sh: note workload-identity is invalid for rustfs as well as minio in the --auth-method help. Validated: bash -n clean; both addressing-style tests pass and stay deterministic under a leaked *_ADDRESSING_STYLE env; env-based bucket command re-verified end-to-end (create + idempotent re-run) against a live RustFS container. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Hans Arnholm <harnholm@nvidia.com>
vvnpn-nv
approved these changes
May 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
--storage-backend rustfs: an in-cluster, S3-compatible object store (rustfs.com) as a drop-in alternative to MinIO, installed via the official Helm chart (https://charts.rustfs.com).Also lands the storage/ Bazel test scaffolding (BUILD + minio/rustfs addressing-style sh_tests), which was previously untracked.
Validated: chart renders via
helm template; bucket create/idempotency and object round-trip verified against a live RustFS container using aws-cli.Issue #None
Checklist
Summary by CodeRabbit
New Features
Documentation
Bug Fixes / Validation
Tests