Skip to content

feat(storage)!: 3.0 — immutable coroutine-safe devices, enums, max static analysis#78

Merged
loks0n merged 14 commits into
mainfrom
feat/storage-2.x
Jul 20, 2026
Merged

feat(storage)!: 3.0 — immutable coroutine-safe devices, enums, max static analysis#78
loks0n merged 14 commits into
mainfrom
feat/storage-2.x

Conversation

@loks0n

@loks0n loks0n commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Prepares the next major release of utopia-php/storage (3.0.0 — latest tag is storage/2.0.7), modernising the interface in line with newer utopia libraries such as dns.

No global state

  • The static device registry (Storage::setDevice/getDevice/exists/$devices) is removed. Devices are constructed directly and passed around; Storage shrinks to the pure human() helper.
  • The static S3::setRetryAttempts()/setRetryDelay() are gone (see below).

Immutability

  • All configuration moves to readonly constructor promotion. setTelemetry(), setHttpVersion(), and setTransferChunkSize() are replaced by telemetry, httpVersion, retryAttempts, retryDelay constructor arguments; transfer chunk size becomes a per-call argument: transfer($path, $dest, $device, $chunkSize).
  • Secret keys are marked #[\SensitiveParameter].

Coroutine safety

  • The S3 adapter previously mutated shared $this->headers/$this->amzHeaders on every operation — two concurrent requests on one device instance (e.g. Swoole coroutines) would race and corrupt each other's SigV4 signatures. Headers are now built locally per call and passed through call()/getSignatureV4(); a device instance holds no request state.
  • Side fixes: the old code signed empty headers it never sent (now it signs exactly what it sends), and retries appended to the previous attempt's response buffer (now reset per attempt).

Enums over string constants

  • Storage::DEVICE_*Utopia\Storage\DeviceType (string-backed), returned by Device::getType().
  • S3::ACL_*Utopia\Storage\Acl, taken as a typed constructor parameter.
  • Region constants stay strings: providers add clusters faster than releases, and custom values are valid.

Maximum static analysis

  • packages/storage/phpstan.neon pins level max (root baseline is level 5); packages/storage/rector.php extends the root baseline with the remaining stable prepared sets (typeDeclarationDocblocks, privatization, instanceOf, rectorPreset).
  • src and tests fully conform: the S3 response is a typed readonly value object instead of stdClass, decoded XML is honestly narrowed, the chunked-upload &$metadata contract is a shared UploadMetadata array shape, and the Telemetry decorator uses a generic closure-based measure() instead of magic dispatch.

Docs

  • README rewritten around direct construction, with an "Upgrading from 2.x" migration section; also corrects methods that never existed (listFiles/deleteDirectorygetFiles/deletePath).

PSR-18 HTTP client

  • The S3 adapter no longer talks to cURL directly: requests are built as PSR-7 messages and sent through any PSR-18 client, injected via the new client constructor argument. The default is utopia-php/client with the cURL adapter and no request timeout (matching the old behavior for large chunk uploads); the Swoole coroutine adapter or any other PSR-18 client drops in.
  • The httpVersion option and HTTP_VERSION_* constants are gone — transport concerns (timeouts, TLS, HTTP version, connection reuse) belong to the injected client.
  • Retries use the utopia-php/client Retry decorator with a new S3\RetryStrategy that encodes the S3-specific transient-error policy (SlowDown/ServiceUnavailable/Throttling/RequestThrottled XML codes, 429/503 fallback). The default client wraps the cURL adapter in it; injecting a client controls the whole retry policy, so the retryAttempts/retryDelay constructor arguments are gone.
  • New S3ClientTest unit-tests signing, retries, retry exhaustion, NoSuchKey mapping, and XML decoding against a scripted PSR-18 stub.

Simplification pass (net −656 lines)

  • Storage class deleted entirely (registry was already gone; human() removed too). The stub File validator, unused getPath() $prefix parameter, S3::METHOD_* constants (PSR-7 Method used instead), and the unused utopia-php/system + ext-zlib dependencies are gone.
  • One telemetry mechanism: the Telemetry decorator. S3 no longer has an internal histogram or telemetry constructor argument, so wrapped devices are not double-counted.
  • One upload flow: upload($sourcePath) removed, uploadChunk() takes chunk contents, and uploadData() is now a concrete base-class method composing prepareUploaduploadChunkfinalizeUpload (previously duplicated in each adapter).
  • Filesystem-only methods (createDirectory, getDirectorySize, getPartitionFreeSpace/TotalSpace, getFiles) left the abstract Device contract — Local keeps them, S3 keeps its own getFiles, the -1/no-op S3 stubs are deleted. getName()/getDescription() removed; getType() identifies adapters. abort($extra) renamed abort($uploadId).

Consumer PoC feedback (Appwrite + Appwrite Cloud)

  • Telemetry::getDevice() added: both consumers wrap every injected device in the decorator, which made adapter-specific methods (Local partition metrics, S3 object listing) unreachable after the contract shrink. Consumers should inject the concrete adapter type where adapter-specific methods are needed (e.g. an S3-typed resource); the accessor is the escape hatch for call sites that receive an already-wrapped device.
  • listFiles(prefix, max, cursor) replaces the inconsistent getFiles() (path strings on Local vs raw ListObjectsV2 array on S3): typed FileList/FileInfo readonly value objects, identical on every adapter, back on the base Device contract and forwarded by the decorator. The S3 single-object Contents quirk is normalized inside the adapter instead of by every consumer.

Typed exceptions

  • Every runtime failure extends Utopia\Storage\Exception\StorageException: NotFoundException (missing files, now consistent across adapters and methods — S3 HEAD 404s previously threw a generic exception with an empty message), TransportException (PSR-18 delivery failures, previous chained), RemoteException (service errors; HTTP status as code, service error identifier such as SlowDown as a readonly errorCode), and UploadException (invalid chunked-upload state). Programmer errors throw SPL \InvalidArgumentException. Existing catch (\Exception) blocks keep working.

Test plan

  • bin/monorepo check storage — pint, PHPStan (level max), rector all pass
  • bin/monorepo test storage — 57 unit + 31 e2e tests pass; the e2e suite runs against a real MinIO container and exercises the rewritten per-request SigV4 signing, multipart, retry, and transfer paths end to end
  • vale README.md docs — no errors
  • bin/monorepo validate — package conventions hold

🤖 Generated with Claude Code

…nalysis

Prepare the 3.0 major release of utopia-php/storage:

- Remove the global device registry (Storage::setDevice/getDevice);
  Storage now only holds the human() formatting helper
- Replace every setter (setTelemetry, setHttpVersion, setTransferChunkSize,
  static S3::setRetryAttempts/setRetryDelay) with readonly constructor
  arguments; transfer() takes the chunk size per call instead
- Make the S3 adapter coroutine-safe: request headers are built per call
  and passed through call()/getSignatureV4() instead of mutating shared
  instance state; retries now reset the response buffer instead of
  appending to it
- Replace string constants with enums: Storage::DEVICE_* becomes the
  DeviceType enum returned by getType(), S3::ACL_* becomes the Acl enum
- Type the S3 response as a readonly value object with honest narrowing
  of decoded XML instead of stdClass juggling
- Raise static analysis to the maximum: package-level phpstan.neon at
  level max and rector.php with all stable prepared sets, with src and
  tests fully conforming
- Mark secret keys #[\SensitiveParameter] and add declare(strict_types=1)
  across the package

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR modernizes the storage package for the next major release. The main changes are:

  • Removes the static storage registry and mutable device configuration.
  • Moves S3 requests to PSR-18 clients with per-request signing state.
  • Replaces string constants with typed enums for device types and ACLs.
  • Adds typed file listing and shared upload flow abstractions.
  • Updates docs, tests, dependency metadata, and static analysis settings.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • Non-positive transfer chunk sizes now fail before chunk math or writes.
  • The telemetry wrapper passes the value through to the guarded device implementation.

Important Files Changed

Filename Overview
packages/storage/src/Storage/Device/Local.php Adds a runtime guard before Local transfer chunk arithmetic.
packages/storage/src/Storage/Device/S3.php Adds a runtime guard before S3 transfer chunk arithmetic and ranged reads.
packages/storage/src/Storage/Device/Telemetry.php Forwards the transfer chunk size to the wrapped device.
packages/storage/tests/Storage/Device/LocalTest.php Adds test coverage for rejecting an invalid Local transfer chunk size.

Reviews (13): Last reviewed commit: "feat(storage): typed exception hierarchy..." | Re-trigger Greptile

Comment thread packages/storage/src/Storage/Device/Local.php
Comment thread packages/storage/src/Storage/Device/S3.php
loks0n and others added 2 commits July 20, 2026 11:57
Replace the raw cURL transport in the S3 adapter with any PSR-18 client,
injected via the new `client` constructor argument and defaulting to
utopia-php/client with the cURL adapter and no request timeout
(matching the previous behavior for large chunk uploads).

- Requests are built as PSR-7 messages (utopia-php/psr7) and signed
  exactly as sent; the S3-specific transient-error retry loop stays in
  call() since generic retry decorators cannot parse S3 XML error codes
- Drop the `httpVersion` option and `HTTP_VERSION_*` constants:
  transport concerns (timeouts, TLS, HTTP version, connection reuse)
  now belong to the injected client
- Add S3ClientTest covering signing, retry, exhaustion, NoSuchKey
  mapping, and XML decoding against a scripted PSR-18 stub

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the hand-rolled retry loop in S3::call() with the
utopia-php/client Retry decorator driven by a new S3\RetryStrategy that
encodes the S3-specific transient-error policy (SlowDown,
ServiceUnavailable, Throttling, RequestThrottled XML codes, with plain
429/503 as fallback).

- Drop the retryAttempts/retryDelay constructor arguments from S3 and
  its subclasses; the default client wraps the cURL adapter in
  Retry(new RetryStrategy()), and injecting a client now controls the
  whole retry policy (strategy, backoff, or none)
- Remove S3::isTransientError() and the usleep loop; call() sends once
- Rename S3SlowDownTest to S3MultipartTest (transient-error coverage
  moved to RetryStrategyTest); S3ClientTest's stub now implements the
  full client Adapter so retries are exercised through the real
  decorator

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/storage/src/Storage/Device/Local.php
Comment thread packages/storage/src/Storage/Device/S3.php
loks0n and others added 11 commits July 20, 2026 12:20
Simplification pass over the 3.0 surface, net -656 lines:

- Delete the Storage class (human() included), the stub File validator,
  the unused getPath() $prefix parameter, the S3 METHOD_* constants
  (utopia-php/psr7 Method is used instead), and the unused
  utopia-php/system and ext-zlib dependencies
- One telemetry mechanism: the Telemetry decorator. S3's internal
  storage.operation histogram and the telemetry constructor argument
  are removed, so wrapped devices are no longer double-counted
- One upload flow: uploads are data-based. upload($sourcePath) is
  removed, uploadChunk() now takes chunk contents instead of a source
  file path, and uploadData() is a concrete Device method composing
  prepareUpload + uploadChunk + finalizeUpload (previously duplicated
  per adapter)
- Filesystem-only methods (createDirectory, getDirectorySize,
  getPartitionFreeSpace, getPartitionTotalSpace, getFiles) leave the
  abstract Device contract; Local keeps them, S3 keeps its own
  getFiles, and the -1/no-op S3 stubs are deleted
- getName()/getDescription() are removed everywhere; getType()
  identifies an adapter. abort()'s cryptic $extra is renamed $uploadId

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address review feedback: transfer() with a chunk size of zero crashed
with a division by zero on large files, and a negative value silently
returned true without writing the destination. Both Local and S3 now
throw, and the runtime guard replaces the positive-int annotation as
the contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI runs rector without a cache and flagged validator classes and tests
that the locally cached run skipped: typed test properties, privatized
final-class properties, and removed useless var tags.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Working checklist for creating or modernizing a package: immutable
constructor-only state and coroutine safety, honest abstract surfaces,
composition via decorators and PSR seams, enums for closed sets,
max-level static analysis with runtime guards over annotations, the
rector cache pitfall, test-tier contracts, and major-release hygiene.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spell out parameter/dependencies shorthand in the skill and add ACLs,
composable, docblock, and redeclaring to the shared vocabulary in
their existing section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Found migrating Appwrite and Appwrite Cloud: every device they inject
is Telemetry-wrapped, so adapter-specific methods that left the base
Device contract (Local partition metrics, S3 object listing) became
unreachable through standard wiring. getUnderlying() gives consumers a
typed path to the concrete adapter without widening the base contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getFiles() was never one abstraction: Local returned path strings while
S3 returned its decoded ListObjectsV2 array, leaking AWS quirks (the
single-object Contents shape) to every consumer. listFiles(prefix, max,
cursor) returns FileList/FileInfo readonly value objects consistently
on both adapters and rejoins the base Device contract, so the Telemetry
decorator forwards it and Device-typed consumers can paginate portably.
The cursor is the S3 continuation token, or a numeric offset for Local.

Also rename Telemetry::getUnderlying()/$underlying to getDevice()/
$device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unit tests now map one-to-one onto src/Storage: the two S3 unit test
files merge into Device/S3Test.php, and the e2e suite (S3Base plus the
MinIO-backed S3Test) moves to tests/E2E under its own namespace. The
phpunit suites become directory-based, so new tests are picked up
without editing phpunit.xml.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every runtime failure now extends Utopia\Storage\Exception\StorageException
so consumers can catch storage problems as one category, or branch:

- NotFoundException: missing files, now consistent on every adapter and
  method — S3 HEAD errors (empty body) previously surfaced as a generic
  exception with an empty message and Local stat/transfer failures were
  untyped
- TransportException: PSR-18 delivery failures, previous chained
- RemoteException: service error responses, with the HTTP status as the
  code and the service's own error identifier (e.g. SlowDown) as a
  readonly errorCode property; the message is the parsed S3 Message
  rather than the raw XML body
- UploadException: invalid chunked-upload state (missing chunk, never
  prepared, finalize failure)

Invalid arguments (non-positive chunk size, page size above the adapter
limit, unknown validator mime type) throw SPL InvalidArgumentException:
programmer errors, not storage failures. Existing catch (\Exception)
blocks keep working.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@loks0n
loks0n merged commit 8de51ce into main Jul 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant