diff --git a/samples/durable-functions/dotnet/LargePayload/README.md b/samples/durable-functions/dotnet/LargePayload/README.md index e6bc9683..3caac949 100644 --- a/samples/durable-functions/dotnet/LargePayload/README.md +++ b/samples/durable-functions/dotnet/LargePayload/README.md @@ -41,7 +41,8 @@ The sample enables these settings in `host.json`: "type": "azureManaged", "connectionStringName": "DTS_CONNECTION_STRING", "payloadStorageEnabled": true, - "payloadStorageThresholdBytes": 262144 + "payloadStorageThresholdBytes": 262144, + "payloadStorageMaxPayloadBytes": 15728640 }, "hubName": "%TASKHUB_NAME%" } @@ -56,6 +57,30 @@ When a payload exceeds `payloadStorageThresholdBytes`, the Durable Functions ext The sample uses a deterministic, low-compressibility **1.5 MiB** payload by default and a **262,144-byte (256 KiB)** threshold so externalization happens before the payload approaches the DTS 1 MiB message boundary. +### Maximum payload size + +`payloadStorageThresholdBytes` controls *when* a payload is externalized. A separate setting, `payloadStorageMaxPayloadBytes`, controls the **largest single payload** the runtime will externalize: + +- If you don't set `payloadStorageMaxPayloadBytes`, the default maximum is **10 MB (10,485,760 bytes, shown as 10,240 KB in the error message)**. +- A request whose payload exceeds the maximum **fails fast** — before the orchestration is scheduled — with an error like: + + ```text + Payload size 10742 KB exceeds the configured maximum of 10240 KB. Reduce the payload size or increase the max payload size limit. + ``` + +- This is a **configurable limit, not a hard product limit**. To allow larger payloads, raise `payloadStorageMaxPayloadBytes` in `host.json`. This sample sets it to `15728640` (15 MiB), so payloads up to 15 MiB are accepted. + +#### `PAYLOAD_SIZE_BYTES` is not `payloadStorageMaxPayloadBytes` + +These two settings are easy to confuse but do completely different things: + +| Setting | Where it lives | What it controls | +|---|---|---| +| `PAYLOAD_SIZE_BYTES` | app setting / `local.settings.json` | How many bytes of test data **this sample generates** for its demo payload. | +| `payloadStorageMaxPayloadBytes` | `host.json` | The **maximum** size the runtime will externalize. Payloads larger than this fail fast. | + +If you raise `PAYLOAD_SIZE_BYTES` above `payloadStorageMaxPayloadBytes` (for example `PAYLOAD_SIZE_BYTES=11000000` against the default 10 MB cap), the run fails **by design** with the error above. Raise `payloadStorageMaxPayloadBytes` to at least the payload size you want to support. + ## Prerequisites - [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0) @@ -132,7 +157,7 @@ The extension stores payload blobs with gzip content encoding, so Azure shows th | `DTS_CONNECTION_STRING` | DTS emulator or Azure connection string | `Endpoint=http://localhost:8080;Authentication=None` | | `TASKHUB_NAME` | Task hub name | `default` | | `AzureWebJobsStorage` | Storage for Functions host state and payload blobs | `UseDevelopmentStorage=true` locally | -| `PAYLOAD_SIZE_BYTES` | Payload size used by the HTTP starter | `1572864` | +| `PAYLOAD_SIZE_BYTES` | Size of the test payload this sample generates. Keep it at or below the configured `payloadStorageMaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` | ## Deploy to Azure with AZD diff --git a/samples/durable-functions/dotnet/LargePayload/host.json b/samples/durable-functions/dotnet/LargePayload/host.json index 18080dd2..480e57f6 100644 --- a/samples/durable-functions/dotnet/LargePayload/host.json +++ b/samples/durable-functions/dotnet/LargePayload/host.json @@ -21,7 +21,8 @@ "type": "azureManaged", "connectionStringName": "DTS_CONNECTION_STRING", "payloadStorageEnabled": true, - "payloadStorageThresholdBytes": 262144 + "payloadStorageThresholdBytes": 262144, + "payloadStorageMaxPayloadBytes": 15728640 }, "hubName": "%TASKHUB_NAME%" } diff --git a/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/README.md b/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/README.md index a27eac98..6e2377ae 100644 --- a/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/README.md +++ b/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/README.md @@ -41,7 +41,8 @@ The sample enables these settings in `host.json`: "type": "azureManaged", "connectionStringName": "DTS_CONNECTION_STRING", "payloadStorageEnabled": true, - "payloadStorageThresholdBytes": 262144 + "payloadStorageThresholdBytes": 262144, + "payloadStorageMaxPayloadBytes": 15728640 }, "hubName": "%TASKHUB_NAME%" } @@ -56,6 +57,31 @@ When a payload exceeds `payloadStorageThresholdBytes`, the Durable Functions ext The sample uses **three deterministic, low-compressibility 1.5 MiB activity payloads** by default and a **262,144-byte (256 KiB)** threshold so externalization happens before the payload approaches the DTS 1 MiB message boundary. +### Maximum payload size + +`payloadStorageThresholdBytes` controls *when* a payload is externalized. A separate setting, `payloadStorageMaxPayloadBytes`, controls the **largest single payload** the runtime will externalize: + +- If you don't set `payloadStorageMaxPayloadBytes`, the default maximum is **10 MB (10,485,760 bytes, shown as 10,240 KB in the error message)**. +- The cap applies to each **single** externalized payload (here, each activity's payload), not the combined total across all activities. +- A request whose payload exceeds the maximum **fails fast** — before the orchestration is scheduled — with an error like: + + ```text + Payload size 10742 KB exceeds the configured maximum of 10240 KB. Reduce the payload size or increase the max payload size limit. + ``` + +- This is a **configurable limit, not a hard product limit**. To allow larger payloads, raise `payloadStorageMaxPayloadBytes` in `host.json`. This sample sets it to `15728640` (15 MiB), so each payload up to 15 MiB is accepted. + +#### `PAYLOAD_SIZE_BYTES` is not `payloadStorageMaxPayloadBytes` + +These two settings are easy to confuse but do completely different things: + +| Setting | Where it lives | What it controls | +|---|---|---| +| `PAYLOAD_SIZE_BYTES` | app setting / `local.settings.json` | How many bytes of test data **each activity generates** for its demo payload. | +| `payloadStorageMaxPayloadBytes` | `host.json` | The **maximum** size of any single payload the runtime will externalize. Larger payloads fail fast. | + +If you raise `PAYLOAD_SIZE_BYTES` above `payloadStorageMaxPayloadBytes` (for example `PAYLOAD_SIZE_BYTES=11000000` against the default 10 MB cap), the run fails **by design** with the error above. Raise `payloadStorageMaxPayloadBytes` to at least the per-activity payload size you want to support. + ## Prerequisites - [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0) @@ -134,7 +160,7 @@ The extension stores payload blobs with gzip content encoding, so Azure shows th | `DTS_CONNECTION_STRING` | DTS emulator or Azure connection string | `Endpoint=http://localhost:8080;Authentication=None` | | `TASKHUB_NAME` | Task hub name | `default` | | `AzureWebJobsStorage` | Storage for Functions host state and payload blobs | `UseDevelopmentStorage=true` locally | -| `PAYLOAD_SIZE_BYTES` | Payload size generated by each activity | `1572864` | +| `PAYLOAD_SIZE_BYTES` | Size of the test payload each activity generates. Keep it at or below the configured `payloadStorageMaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` | | `ACTIVITY_COUNT` | Number of parallel activity invocations | `3` | ## Deploy to Azure with AZD diff --git a/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/host.json b/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/host.json index 18080dd2..480e57f6 100644 --- a/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/host.json +++ b/samples/durable-functions/dotnet/LargePayloadFanOutFanIn/host.json @@ -21,7 +21,8 @@ "type": "azureManaged", "connectionStringName": "DTS_CONNECTION_STRING", "payloadStorageEnabled": true, - "payloadStorageThresholdBytes": 262144 + "payloadStorageThresholdBytes": 262144, + "payloadStorageMaxPayloadBytes": 15728640 }, "hubName": "%TASKHUB_NAME%" } diff --git a/samples/durable-task-sdks/dotnet/LargePayload/README.md b/samples/durable-task-sdks/dotnet/LargePayload/README.md index aef0b372..2cf603b5 100644 --- a/samples/durable-task-sdks/dotnet/LargePayload/README.md +++ b/samples/durable-task-sdks/dotnet/LargePayload/README.md @@ -24,6 +24,8 @@ The sample uses a deterministic, low-compressibility **1.5 MiB** payload by defa > `THRESHOLD_BYTES` is a sample-only environment variable. The sample reads it and assigns it to the SDK's `LargePayloadStorageOptions.ThresholdBytes`, which must stay at or below `1,048,576` bytes (1 MiB). +> The same options object also caps the largest **single** externalized payload with `LargePayloadStorageOptions.MaxPayloadBytes`, which defaults to **10 MB (10,485,760 bytes)**. A larger payload fails fast with `Payload size KB exceeds the configured maximum of KB. Reduce the payload size or increase the max payload size limit.`. `PAYLOAD_SIZE_BYTES` only sets how much test data this sample generates — it is *not* the cap. To externalize payloads larger than 10 MB, set `options.MaxPayloadBytes` higher in `Program.cs` where `AddExternalizedPayloadStore` is configured. + ## Prerequisites 1. [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0) @@ -93,7 +95,7 @@ The sample works out of the box locally, but you can override the defaults with | `PAYLOAD_STORAGE_CONNECTION_STRING` | Storage connection string for payload blobs | `UseDevelopmentStorage=true` | | `PAYLOAD_STORAGE_ACCOUNT_URI` | Blob account URI for identity-based storage access | unset | | `PAYLOAD_CONTAINER_NAME` | Blob container used for externalized payloads | `durabletask-payloads` | -| `PAYLOAD_SIZE_BYTES` | Default payload size used by the run endpoint | `1572864` | +| `PAYLOAD_SIZE_BYTES` | Default size of the test payload the run endpoint generates. Keep it at or below `LargePayloadStorageOptions.MaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` | | `THRESHOLD_BYTES` | Blob offload threshold | `262144` | | `PAYLOAD_STORAGE_MANAGED_IDENTITY_CLIENT_ID` | Optional user-assigned managed identity client ID for storage | unset | | `ASPNETCORE_URLS` | Listen URLs for the web host | framework default |