Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions samples/durable-functions/dotnet/LargePayload/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ The sample enables these settings in `host.json`:
"type": "azureManaged",
"connectionStringName": "DTS_CONNECTION_STRING",
"payloadStorageEnabled": true,
"payloadStorageThresholdBytes": 262144
"payloadStorageThresholdBytes": 262144,
"payloadStorageMaxPayloadBytes": 15728640
},
"hubName": "%TASKHUB_NAME%"
}
Expand All @@ -56,6 +57,30 @@ When a payload exceeds `payloadStorageThresholdBytes`, the Durable Functions ext

The sample uses a deterministic, low-compressibility **1.5 MiB** payload by default and a **262,144-byte (256 KiB)** threshold so externalization happens before the payload approaches the DTS 1 MiB message boundary.

### Maximum payload size

`payloadStorageThresholdBytes` controls *when* a payload is externalized. A separate setting, `payloadStorageMaxPayloadBytes`, controls the **largest single payload** the runtime will externalize:

- If you don't set `payloadStorageMaxPayloadBytes`, the default maximum is **10 MB (10,485,760 bytes, shown as 10,240 KB in the error message)**.
- A request whose payload exceeds the maximum **fails fast** — before the orchestration is scheduled — with an error like:

```text
Payload size 10742 KB exceeds the configured maximum of 10240 KB. Reduce the payload size or increase the max payload size limit.
```

- This is a **configurable limit, not a hard product limit**. To allow larger payloads, raise `payloadStorageMaxPayloadBytes` in `host.json`. This sample sets it to `15728640` (15 MiB), so payloads up to 15 MiB are accepted.

#### `PAYLOAD_SIZE_BYTES` is not `payloadStorageMaxPayloadBytes`

These two settings are easy to confuse but do completely different things:

| Setting | Where it lives | What it controls |
|---|---|---|
| `PAYLOAD_SIZE_BYTES` | app setting / `local.settings.json` | How many bytes of test data **this sample generates** for its demo payload. |
| `payloadStorageMaxPayloadBytes` | `host.json` | The **maximum** size the runtime will externalize. Payloads larger than this fail fast. |

If you raise `PAYLOAD_SIZE_BYTES` above `payloadStorageMaxPayloadBytes` (for example `PAYLOAD_SIZE_BYTES=11000000` against the default 10 MB cap), the run fails **by design** with the error above. Raise `payloadStorageMaxPayloadBytes` to at least the payload size you want to support.

## Prerequisites

- [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0)
Expand Down Expand Up @@ -132,7 +157,7 @@ The extension stores payload blobs with gzip content encoding, so Azure shows th
| `DTS_CONNECTION_STRING` | DTS emulator or Azure connection string | `Endpoint=http://localhost:8080;Authentication=None` |
| `TASKHUB_NAME` | Task hub name | `default` |
| `AzureWebJobsStorage` | Storage for Functions host state and payload blobs | `UseDevelopmentStorage=true` locally |
| `PAYLOAD_SIZE_BYTES` | Payload size used by the HTTP starter | `1572864` |
| `PAYLOAD_SIZE_BYTES` | Size of the test payload this sample generates. Keep it at or below the configured `payloadStorageMaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` |

## Deploy to Azure with AZD

Expand Down
3 changes: 2 additions & 1 deletion samples/durable-functions/dotnet/LargePayload/host.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@
"type": "azureManaged",
"connectionStringName": "DTS_CONNECTION_STRING",
"payloadStorageEnabled": true,
"payloadStorageThresholdBytes": 262144
"payloadStorageThresholdBytes": 262144,
"payloadStorageMaxPayloadBytes": 15728640
},
"hubName": "%TASKHUB_NAME%"
}
Expand Down
30 changes: 28 additions & 2 deletions samples/durable-functions/dotnet/LargePayloadFanOutFanIn/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ The sample enables these settings in `host.json`:
"type": "azureManaged",
"connectionStringName": "DTS_CONNECTION_STRING",
"payloadStorageEnabled": true,
"payloadStorageThresholdBytes": 262144
"payloadStorageThresholdBytes": 262144,
"payloadStorageMaxPayloadBytes": 15728640
},
"hubName": "%TASKHUB_NAME%"
}
Expand All @@ -56,6 +57,31 @@ When a payload exceeds `payloadStorageThresholdBytes`, the Durable Functions ext

The sample uses **three deterministic, low-compressibility 1.5 MiB activity payloads** by default and a **262,144-byte (256 KiB)** threshold so externalization happens before the payload approaches the DTS 1 MiB message boundary.

### Maximum payload size

`payloadStorageThresholdBytes` controls *when* a payload is externalized. A separate setting, `payloadStorageMaxPayloadBytes`, controls the **largest single payload** the runtime will externalize:

- If you don't set `payloadStorageMaxPayloadBytes`, the default maximum is **10 MB (10,485,760 bytes, shown as 10,240 KB in the error message)**.
- The cap applies to each **single** externalized payload (here, each activity's payload), not the combined total across all activities.
- A request whose payload exceeds the maximum **fails fast** — before the orchestration is scheduled — with an error like:

```text
Payload size 10742 KB exceeds the configured maximum of 10240 KB. Reduce the payload size or increase the max payload size limit.
```

- This is a **configurable limit, not a hard product limit**. To allow larger payloads, raise `payloadStorageMaxPayloadBytes` in `host.json`. This sample sets it to `15728640` (15 MiB), so each payload up to 15 MiB is accepted.

#### `PAYLOAD_SIZE_BYTES` is not `payloadStorageMaxPayloadBytes`

These two settings are easy to confuse but do completely different things:

| Setting | Where it lives | What it controls |
|---|---|---|
| `PAYLOAD_SIZE_BYTES` | app setting / `local.settings.json` | How many bytes of test data **each activity generates** for its demo payload. |
| `payloadStorageMaxPayloadBytes` | `host.json` | The **maximum** size of any single payload the runtime will externalize. Larger payloads fail fast. |

If you raise `PAYLOAD_SIZE_BYTES` above `payloadStorageMaxPayloadBytes` (for example `PAYLOAD_SIZE_BYTES=11000000` against the default 10 MB cap), the run fails **by design** with the error above. Raise `payloadStorageMaxPayloadBytes` to at least the per-activity payload size you want to support.

## Prerequisites

- [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0)
Expand Down Expand Up @@ -134,7 +160,7 @@ The extension stores payload blobs with gzip content encoding, so Azure shows th
| `DTS_CONNECTION_STRING` | DTS emulator or Azure connection string | `Endpoint=http://localhost:8080;Authentication=None` |
| `TASKHUB_NAME` | Task hub name | `default` |
| `AzureWebJobsStorage` | Storage for Functions host state and payload blobs | `UseDevelopmentStorage=true` locally |
| `PAYLOAD_SIZE_BYTES` | Payload size generated by each activity | `1572864` |
| `PAYLOAD_SIZE_BYTES` | Size of the test payload each activity generates. Keep it at or below the configured `payloadStorageMaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` |
| `ACTIVITY_COUNT` | Number of parallel activity invocations | `3` |

## Deploy to Azure with AZD
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@
"type": "azureManaged",
"connectionStringName": "DTS_CONNECTION_STRING",
"payloadStorageEnabled": true,
"payloadStorageThresholdBytes": 262144
"payloadStorageThresholdBytes": 262144,
"payloadStorageMaxPayloadBytes": 15728640
},
"hubName": "%TASKHUB_NAME%"
}
Expand Down
4 changes: 3 additions & 1 deletion samples/durable-task-sdks/dotnet/LargePayload/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ The sample uses a deterministic, low-compressibility **1.5 MiB** payload by defa

> `THRESHOLD_BYTES` is a sample-only environment variable. The sample reads it and assigns it to the SDK's `LargePayloadStorageOptions.ThresholdBytes`, which must stay at or below `1,048,576` bytes (1 MiB).

> The same options object also caps the largest **single** externalized payload with `LargePayloadStorageOptions.MaxPayloadBytes`, which defaults to **10 MB (10,485,760 bytes)**. A larger payload fails fast with `Payload size <N> KB exceeds the configured maximum of <M> KB. Reduce the payload size or increase the max payload size limit.`. `PAYLOAD_SIZE_BYTES` only sets how much test data this sample generates — it is *not* the cap. To externalize payloads larger than 10 MB, set `options.MaxPayloadBytes` higher in `Program.cs` where `AddExternalizedPayloadStore` is configured.

## Prerequisites

1. [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
Expand Down Expand Up @@ -93,7 +95,7 @@ The sample works out of the box locally, but you can override the defaults with
| `PAYLOAD_STORAGE_CONNECTION_STRING` | Storage connection string for payload blobs | `UseDevelopmentStorage=true` |
| `PAYLOAD_STORAGE_ACCOUNT_URI` | Blob account URI for identity-based storage access | unset |
| `PAYLOAD_CONTAINER_NAME` | Blob container used for externalized payloads | `durabletask-payloads` |
| `PAYLOAD_SIZE_BYTES` | Default payload size used by the run endpoint | `1572864` |
| `PAYLOAD_SIZE_BYTES` | Default size of the test payload the run endpoint generates. Keep it at or below `LargePayloadStorageOptions.MaxPayloadBytes` (default 10 MB); larger values fail by design. | `1572864` |
| `THRESHOLD_BYTES` | Blob offload threshold | `262144` |
| `PAYLOAD_STORAGE_MANAGED_IDENTITY_CLIENT_ID` | Optional user-assigned managed identity client ID for storage | unset |
| `ASPNETCORE_URLS` | Listen URLs for the web host | framework default |
Expand Down
Loading