fix: bucket URL is incorrect for local S3 emulators

[peter-smith-phd]

The AWS SDK addresses S3 buckets using virtual-hosted-style URLs by default (https://<bucket>.s3.amazonaws.com). When targeting custom or local S3-compatible endpoints (for example LocalStack or MinIO), path-style addressing (https://<endpoint>/<bucket>) is often required instead.

There are two ways to enable path-style addressing on the S3 clients, used for asset publishing (cdk-assets-lib) and by the toolkit (toolkit-lib).

1. Setting AWS_ENDPOINT_URL (or AWS_ENDPOINT_URL_S3) to use a local loopback host, such as localhost, 127.0.0.1 or ::1.
2. Setting the CDK_S3_FORCE_PATH_STYLE environment variable to explicitly enable the feature. This name follows the existing convention of CDK-specific environment variables such as CDK_DOCKER and CDK_HOME.

Checklist

• Unit tests added/updated
• Integration tests added/updated (if deploying new resource types or cross-service interactions)

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[rix0rrr]

Is there a way to autoderive the correct behavior here? Zero-conf is better than an option fleet...

[peter-smith-phd]

Is there a way to autoderive the correct behavior here? Zero-conf is better than an option fleet...

Great question. This option is solely necessary when the user sets a custom AWS_ENDPOINT_URL_S3 . If the URL they provide doesn't support subdomains properly (which is a problem for 127.0.0.1 or localhost), then S3 path-style addressing is required.

So, we could automatically set forcePathStyle if there's a change to AWS_ENDPOINT_URL_S3 (or AWS_ENDPOINT_URL that implicitly sets AWS_ENDPOINT_URL_S3). This is fine if the new URL refers to a non-AWS location (like an emulator), but could be problematic if the new URL points so something like a FIPS endpoint (prefers virtual host for performance reasons).

The most accurate solution would be to detect 127.0.0.1 and localhost explicitly, and automatically set the flag only in those cases. Would that be more acceptable?

[rix0rrr]

I think I would like that better, yes. I'm debating whether it would be beneficial to check the URL and see if it ends in one of the AWS domains (*.amazonaws.com etc), and disable the feature if not. Should work for FIPS endpoints, VPC endpoints, and if our domain list is exhaustive enough also for ADC regions.

But to be frank, I don't know enough about the various network configurations people could run to know if that's exhaustive enough or not, or people would want virtual hosts with custom domains as well... so probably doing the following will do the most good for the most people:

• Keep the environment variable for explicit control
• Autodetect to path-based routing for special hosts like localhost and 127.0.0.1

[peter-smith-phd]

Thanks, that sounds like a plan. I suspect that localhost and 127.0.0.1 are the far most common hostnames that would be used, whereas trying to enumerate all those ADC regions would end up with yearly maintenance 😄

I've updated the PR, and have created a shared package (based off how zip is shared). Hopefully this is the correct approach.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.73%. Comparing base (425ccb0) to head (6acde5a).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1625   +/-   ##
=======================================
  Coverage   88.73%   88.73%           
=======================================
  Files          77       77           
  Lines       11359    11359           
  Branches     1584     1585    +1     
=======================================
  Hits        10079    10079           
  Misses       1250     1250           
  Partials       30       30           

Flag	Coverage Δ
suite.unit	88.73% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.