Presigned-URL support (url_for) + honor path prefix in standard flow (#7)#8
Merged
Conversation
… flow (#7) - Add `url_for(key, *, expires_in=3600, client_method='get_object', **params)` to the S3 bucket stores (BaseS3BucketReader + prefix-aware S3BucketReader override). Returns a presigned URL via the boto3 client — lets a caller 302-redirect so the object store serves bytes + HTTP Range directly, off the app process. Works for any S3-compatible endpoint (AWS, R2, MinIO, Supabase). First customer: lacing.ArtifactStore.blob_location (reelee storage migration). - Fix: S3Store's `path=` (documented as the key prefix) was silently DROPPED in the standard AWS flow — only the Supabase / skip-check branches honored it. Now applied there too (normalized like S3BucketReader.__post_init__). - Add moto-based tests (test_url_for.py) — run anywhere, no real S3. An autouse fixture clears AWS_ENDPOINT_URL because test_base sets it (localstack) via os.environ without cleanup, which otherwise leaks across the suite. Note: the localstack-parametrized tests in test_base/test_store fail without a running localstack server (pre-existing/environmental); making them skip-when- unreachable + use monkeypatch for env is a separate test-hygiene follow-up. Claude-Session: https://claude.ai/code/session_01NLg6gmcHzRiDviqdgyooZk
thorwhalen
added a commit
to thorwhalen/lacing
that referenced
this pull request
Jun 20, 2026
…elee#134) ArtifactStore.from_s3(bucket, *, catalog=None, prefix=None, **s3_kwargs) wires an S3-compatible object store (AWS / R2 / MinIO / Supabase) as the `blobs` backend via s3dol — pure DI, the facade + all callers unchanged. Content- addressed blobs are a natural fit (flat, immutable, dedup, forever-cacheable). blob_location (Phase 0) returns a presigned GET URL via s3dol's new url_for, so the serving layer can 302-redirect and let the store deliver bytes + Range directly. s3dol imported lazily; the catalog defaults to in-memory (Phase 2 adds the Postgres catalog). Tests: tests/test_artifact_store_s3.py — moto-mocked (no real S3, runs in CI): content-addressed roundtrip, presigned-URL blob_location (+ blob_path None for object stores), missing-blob None, dual-write save, key-prefix. An autouse fixture clears AWS endpoint env so @mock_aws always intercepts. pyproject: dev extra gains `s3dol` + `moto[s3]`; new `s3` extra (s3dol). Needs s3dol's url_for (i2mint/s3dol#8) on master before this ships. Claude-Session: https://claude.ai/code/session_01NLg6gmcHzRiDviqdgyooZk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7.
What
url_for(key, *, expires_in=3600, client_method='get_object', **params)on the S3 bucket stores — returns a presigned URL via the boto3 client (prefix-aware override onS3BucketReader). Lets a caller 302-redirect so the object store serves bytes + HTTPRangedirectly, off the app process. Works for any S3-compatible endpoint (AWS / R2 / MinIO / Supabase) — pure client call, no object fetch.S3Store(path=...)(documented as the key prefix) was silently dropped in the standard AWS flow — only the Supabase / skip-check branches honored it. Now applied there too.test_url_for.py, moto-based so they run anywhere (no real S3). An autouse fixture clearsAWS_ENDPOINT_URLbecausetest_basesets it (localstack) viaos.environwithout cleanup, leaking across the suite.First customer
lacing's
ArtifactStore.blob_location(reelee storage migration → S3, reelee#134): it probes the blob backend forurl_forand 302-redirects when present.Note
The
localstack-parametrized tests intest_base/test_storefail without a running localstack server (pre-existing/environmental). Making them skip-when-unreachable + usemonkeypatchfor env is a separate test-hygiene follow-up.https://claude.ai/code/session_01NLg6gmcHzRiDviqdgyooZk