Skip to content

[Feature Request] barman-cloud plugin should support virtual-hosted style addressing for S3-compatible storage #712

@buddyGH

Description

@buddyGH

Description:

I'm using CloudNativePG with the barman-cloud plugin (version 0.9.0) for PostgreSQL backups to Alibaba Cloud OSS. However, I've encountered a critical issue: the plugin only supports path-style addressing, while Alibaba Cloud OSS (and many other S3-compatible services) now require virtual-hosted style addressing.

Problem Details:

AWS has deprecated path-style addressing since September 30, 2023
Many cloud providers (Alibaba Cloud OSS, Google Cloud Storage, etc.) require or strongly recommend virtual-hosted style
The barman-cloud plugin doesn't provide any configuration parameter for addressing_style
When endpointURL is specified, it defaults to path-style addressing

Current Workaround:

Currently, users have to:

Use custom backup solutions instead of the official plugin
Or use unsupported path-style endpoints (if available)

Expected Behavior:

The barman-cloud plugin should support both addressing styles with a configuration parameter like:

configuration:
  destinationPath: "s3://bucket-name/"
  endpointURL: "https://bucket-name.oss-cn-region.aliyuncs.com"
  s3AddressingStyle: "virtual"  # or "path"

Or automatically detect the appropriate style based on the endpoint URL format.

Technical Background:

  • AWS SDK (boto3) already supports addressing_style parameter in boto3.session.Config

  • Virtual-hosted style is now the standard for S3-compatible services

  • Path-style addressing has been deprecated by AWS

Impact:

This limitation prevents users from using the official plugin with:

  • Alibaba Cloud OSS

  • Google Cloud Storage

  • DigitalOcean Spaces

  • And other S3-compatible services that enforce virtual-hosted style

Proposed Solution:

Add a new parameter s3AddressingStyle to the ObjectStore configuration that gets passed to the underlying boto3 client configuration.

Code Reference:

In barman-cloud implementation, it should configure boto3 like:

config = boto3.session.Config(
    s3={'addressing_style': addressing_style}
)
s3_client = boto3.client(
    's3',
    endpoint_url=endpoint,
    config=config
)

Environment:

  • CloudNativePG: 1.28.0

  • barman-cloud plugin: 0.9.0

  • Kubernetes: 1.32.7

  • Storage: Alibaba Cloud OSS

Additional Context:

  1. Attempted Solutions:

[ ] I have tried setting the environment variable AWS_S3_ADDRESSING_STYLE=virtual in the pod configuration, but it has no effect

  1. Current Status:
  • Alibaba Cloud OSS strictly enforces virtual-hosted style addressing

  • AWS has completely phased out path-style support

  • This issue is becoming critical as more S3-compatible services adopt virtual-hosted style

  1. Reproduction Steps:
  • Configure barman-cloud plugin with Alibaba Cloud OSS endpoint

  • Set endpointURL to virtual-hosted style format

  • Observe that the plugin still attempts path-style requests

  • Receive errors from OSS indicating path-style is not supported

  1. Workaround Limitations:
  • Custom backup scripts lack the integration and reliability of the official plugin

  • Alternative S3-compatible endpoints may not be available in all regions

  • This forces users to choose between functionality and using the official tooling

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions