Skip to content

Commit 4f58595

Browse files
andystaplesandystaples
committed
Merge remote-tracking branch 'origin/main' into andystaples/add-orchestration-rewind
# Conflicts: # durabletask/internal/helpers.py # durabletask/testing/in_memory_backend.py # durabletask/worker.py
2 parents 4507f64 + 465c4de commit 4f58595

210 files changed

Lines changed: 28143 additions & 2576 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.flake8‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[flake8]
2-
ignore = E501,C901
2+
ignore = E501,C901,W503
33
exclude =
44
.git
55
*_pb2*

‎.github/copilot-instructions.md‎

Lines changed: 71 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,20 +8,63 @@ building durable orchestrations. The repo contains two packages:
88
- `durabletask` — core SDK (in `durabletask/`)
99
- `durabletask.azuremanaged` — Azure Durable Task Scheduler provider (in `durabletask-azuremanaged/`)
1010

11+
## Changelog Requirements
12+
13+
- ALWAYS document user-facing changes in the appropriate changelog under
14+
`## Unreleased`.
15+
- Update `CHANGELOG.md` for core SDK changes and
16+
`durabletask-azuremanaged/CHANGELOG.md` for provider changes.
17+
- If a change affects both packages, update both changelogs.
18+
- Include changelog entries for externally observable outcomes only, such as
19+
new public APIs, behavior changes, bug fixes users can notice, breaking
20+
changes, and new configuration capabilities.
21+
- Do NOT document internal-only changes in changelogs, including CI/workflow
22+
updates, test-only changes, refactors with no user-visible behavior change,
23+
and implementation details that do not affect public behavior or API.
24+
- When in doubt, write the changelog entry in terms of user impact (what users
25+
can now do or what behavior changed), not implementation mechanism (how it
26+
was implemented internally).
27+
28+
Examples:
29+
- Include: "Added `get_orchestration_history()` to retrieve orchestration history from the client."
30+
- Exclude: "Added internal helper functions to aggregate streamed history chunks."
31+
1132
## Language and Style
1233

1334
- Python 3.10+ is required.
1435
- Use type hints for all public API signatures.
1536
- Follow PEP 8 conventions.
1637
- Use `autopep8` for Python formatting.
1738

39+
## Copyright Headers
40+
41+
Every new Python (`.py`) source file MUST begin with the following copyright
42+
header as the first two lines, followed by a blank line before any code,
43+
docstring, or imports:
44+
45+
```python
46+
# Copyright (c) Microsoft Corporation.
47+
# Licensed under the MIT License.
48+
```
49+
50+
This applies to all hand-written Python files, including `__init__.py` files,
51+
tests, and examples. The only exceptions are auto-generated protobuf files
52+
(`*_pb2.py` and `*_pb2_grpc.py`), which carry their own generated header.
53+
54+
## Python Type Checking
55+
56+
Before linting, check for and fix any Pylance errors in the files you
57+
changed. Use the editor's diagnostics (or the `get_errors` tool) to
58+
identify type errors and resolve them first — type safety takes
59+
priority over style.
60+
1861
## Python Linting
1962

2063
This repository uses [flake8](https://flake8.pycqa.org/) for Python
2164
linting. Run it after making changes to verify there are no issues:
2265

2366
```bash
24-
flake8 path/to/changed/file.py
67+
python -m flake8 path/to/changed/file.py
2568
```
2669

2770
## Markdown Style
@@ -57,39 +100,61 @@ repository root.
57100
To lint a single file:
58101

59102
```bash
60-
pymarkdown -c .pymarkdown.json scan path/to/file.md
103+
python -m pymarkdown -c .pymarkdown.json scan path/to/file.md
61104
```
62105

63106
To lint all Markdown files in the repository:
64107

65108
```bash
66-
pymarkdown -c .pymarkdown.json scan **/*.md
109+
python -m pymarkdown -c .pymarkdown.json scan **/*.md
67110
```
68111

69112
Install the linter via the dev dependencies:
70113

71114
```bash
72-
pip install -r dev-requirements.txt
115+
python -m pip install -r dev-requirements.txt
73116
```
74117

75118
## Building and Testing
76119

77120
Install the packages locally in editable mode:
78121

79122
```bash
80-
pip install -e . -e ./durabletask-azuremanaged
123+
python -m pip install -e . -e ./durabletask-azuremanaged
81124
```
82125

83126
Run tests with pytest:
84127

85128
```bash
86-
pytest
129+
python -m pytest
87130
```
88131

89132
## Project Structure
90133

91134
- `durabletask/` — core SDK source
135+
- `payload/` — public payload externalization API (`PayloadStore` ABC,
136+
`LargePayloadStorageOptions`, helper functions)
137+
- `extensions/azure_blob_payloads/` — Azure Blob Storage payload store
138+
(installed via `pip install durabletask[azure-blob-payloads]`)
139+
- `entities/` — durable entity support
140+
- `testing/` — in-memory backend for testing without a sidecar
141+
- `internal/` — protobuf definitions, gRPC helpers, tracing (not public API)
92142
- `durabletask-azuremanaged/` — Azure managed provider source
93143
- `examples/` — example orchestrations (see `examples/README.md`)
94144
- `tests/` — test suite
95145
- `dev-requirements.txt` — development dependencies
146+
147+
## Cross-Package Compatibility
148+
149+
The `durabletask-azuremanaged` package extends the core `durabletask`
150+
package (e.g. `DurableTaskSchedulerWorker` subclasses
151+
`TaskHubGrpcWorker`). When adding or changing features in
152+
`durabletask/`, always verify that `durabletask-azuremanaged` still
153+
works correctly:
154+
155+
- Check whether the azuremanaged worker, client, or tests override or
156+
depend on the code you changed.
157+
- Run the azuremanaged unit tests if they exist for the affected area.
158+
- If a new public API is added to the core SDK (e.g. a method on
159+
`OrchestrationContext`), confirm it is accessible through the
160+
azuremanaged package and add a test or example if appropriate.

‎.github/skills/release-prep/SKILL.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
name: release-prep
3+
description: >-
4+
Prepare a release for durabletask and durabletask.azuremanaged. Use when the
5+
user asks for release prep, version bumping, changelog updates, or release
6+
body drafting. Trigger phrases include: release prep, prepare vX.Y.Z,
7+
changelog for release, and draft GitHub release notes.
8+
---
9+
10+
# Release Prep
11+
12+
This skill prepares a coordinated release for both packages in this repository:
13+
14+
- `durabletask`
15+
- `durabletask.azuremanaged`
16+
17+
The skill accepts a target version (for example `1.4.0`) and performs the
18+
required changes consistently.
19+
20+
## Inputs
21+
22+
- `version`: Target semantic version (for example `1.4.0`)
23+
- Optional: `baseTag` overrides for comparison if tags are non-standard
24+
25+
If `version` is not provided, ask the user before continuing.
26+
27+
## Steps
28+
29+
### 1. Determine source range and collect commits
30+
31+
- Root package range: `v<previousVersion>..HEAD`
32+
- Azure managed package range: `azuremanaged-v<previousVersion>..HEAD`
33+
- Use commit subjects and touched files to classify each change as:
34+
- `durabletask` only
35+
- `durabletask.azuremanaged` only
36+
- shared/infra/docs changes
37+
38+
### 2. Update package versions
39+
40+
Update both project versions:
41+
42+
- `pyproject.toml` -> `version = "<version>"`
43+
- `durabletask-azuremanaged/pyproject.toml` -> `version = "<version>"`
44+
45+
Update azuremanaged dependency floors:
46+
47+
- `durabletask>=<version>`
48+
- `durabletask[azure-blob-payloads]>=<version>`
49+
50+
### 3. Update changelogs
51+
52+
- Add a new `## v<version>` section directly under `## Unreleased` in:
53+
- `CHANGELOG.md`
54+
- `durabletask-azuremanaged/CHANGELOG.md`
55+
- Ensure user-facing changes since the previous release tags are represented.
56+
- Keep entries concise and grouped by type (`ADDED`, `CHANGED`, `FIXED`, `REMOVED`) where
57+
applicable.
58+
- Exclude internal-only changes from changelogs (for example CI/workflow-only
59+
updates, test-only changes, and implementation refactors with no public
60+
behavior or API impact).
61+
62+
### 4. Validate
63+
64+
- Run diagnostics on changed markdown and TOML files.
65+
- Fix formatting or heading issues introduced by release prep changes.
66+
- Verify the final diff only contains release-prep updates.
67+
68+
### 5. Wait for merge and tags before release drafting
69+
70+
Before creating draft releases in GitHub UI, require explicit user
71+
confirmation of both conditions:
72+
73+
- The version-bump/release-prep PR is merged
74+
- Tags `v<version>` and `azuremanaged-v<version>` already exist in the target
75+
repository
76+
77+
If either condition is not met, stop after preparing release body text and ask
78+
the user to confirm once merge and tags are complete.
79+
80+
### 6. Draft GitHub release bodies
81+
82+
Draft two release body texts for the GitHub Releases UI (do not add files to
83+
the repository):
84+
85+
- Tag: `v<version>`
86+
- Tag: `azuremanaged-v<version>`
87+
88+
Match existing release structure:
89+
90+
- Title (`# v<version>` or `# azuremanaged-v<version>`)
91+
- `## What's Changed`
92+
- `## External Links`
93+
- `### Contributors`
94+
95+
Include:
96+
97+
- PyPI link for the exact release version
98+
- Full changelog compare link
99+
- Contributor handles from the commit range
100+
- Keep drafts in the assistant response (or PR comment) so they can be pasted
101+
directly into the Releases section
102+
- Keep the release body focused on user-facing changes and avoid internal-only
103+
details (CI/test updates or implementation-only notes)
104+
105+
106+
## Output
107+
108+
Return a short summary with:
109+
110+
- Updated files
111+
- Commit coverage confirmation
112+
- Any manual follow-ups (for example, tag creation or publishing)

‎.github/workflows/codeql.yml‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# For most projects, this workflow file will not need changing; you simply need
2+
# to commit it to your repository.
3+
#
4+
# You may wish to alter this file to override the set of languages analyzed,
5+
# or to provide custom queries or build logic.
6+
#
7+
# ******** NOTE ********
8+
# We have attempted to detect the languages in your repository. Please check
9+
# the `language` matrix defined below to confirm you have the correct set of
10+
# supported CodeQL languages.
11+
#
12+
name: "CodeQL Advanced"
13+
14+
on:
15+
push:
16+
branches: [ "main", "release/*" ]
17+
pull_request:
18+
branches: [ "main", "release/*" ]
19+
schedule:
20+
- cron: '29 7 * * 0'
21+
22+
jobs:
23+
analyze:
24+
name: Analyze (${{ matrix.language }})
25+
# Runner size impacts CodeQL analysis time. To learn more, please see:
26+
# - https://gh.io/recommended-hardware-resources-for-running-codeql
27+
# - https://gh.io/supported-runners-and-hardware-resources
28+
# - https://gh.io/using-larger-runners (GitHub.com only)
29+
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
30+
runs-on: ubuntu-latest
31+
permissions:
32+
# required for all workflows
33+
security-events: write
34+
35+
# required to fetch internal or private CodeQL packs
36+
packages: read
37+
38+
# only required for workflows in private repositories
39+
actions: read
40+
contents: read
41+
42+
strategy:
43+
fail-fast: false
44+
matrix:
45+
include:
46+
- language: actions
47+
build-mode: none
48+
- language: python
49+
build-mode: none
50+
# CodeQL supports the following values keywords for 'language': 'actions', 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'rust', 'swift'
51+
# Use `c-cpp` to analyze code written in C, C++ or both
52+
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
53+
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
54+
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
55+
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
56+
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
57+
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
58+
steps:
59+
- name: Checkout repository
60+
uses: actions/checkout@v4
61+
62+
# Add any setup steps before running the `github/codeql-action/init` action.
63+
# This includes steps like installing compilers or runtimes (`actions/setup-node`
64+
# or others). This is typically only required for manual builds.
65+
# - name: Setup runtime (example)
66+
# uses: actions/setup-example@v1
67+
68+
# Initializes the CodeQL tools for scanning.
69+
- name: Initialize CodeQL
70+
uses: github/codeql-action/init@v4
71+
with:
72+
languages: ${{ matrix.language }}
73+
build-mode: ${{ matrix.build-mode }}
74+
# If you wish to specify custom queries, you can do so here or in a config file.
75+
# By default, queries listed here will override any specified in a config file.
76+
# Prefix the list here with "+" to use these queries and those in the config file.
77+
78+
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
79+
# queries: security-extended,security-and-quality
80+
81+
# If the analyze step fails for one of the languages you are analyzing with
82+
# "We were unable to automatically build your code", modify the matrix above
83+
# to set the build mode to "manual" for that language. Then modify this step
84+
# to build your code.
85+
# ℹ️ Command-line programs to run using the OS shell.
86+
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
87+
- name: Run manual build steps
88+
if: matrix.build-mode == 'manual'
89+
shell: bash
90+
run: |
91+
echo 'If you are using a "manual" build mode for one or more of the' \
92+
'languages you are analyzing, replace this with the commands to build' \
93+
'your code, for example:'
94+
echo ' make bootstrap'
95+
echo ' make release'
96+
exit 1
97+
98+
- name: Perform CodeQL Analysis
99+
uses: github/codeql-action/analyze@v4
100+
with:
101+
category: "/language:${{matrix.language}}"

0 commit comments

Comments
 (0)