-
Notifications
You must be signed in to change notification settings - Fork 12
test: add gated Azure Functions host E2E suite (Storage backend) #303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
5ff8b58
test: add gated Azure Functions host E2E suite (Storage backend)
YunchuWang 146fa74
test: port full BasicNode Functions host E2E suite (Storage backend)
YunchuWang 2963134
ci: make Functions host E2E workflow manual (ad hoc) only
YunchuWang 43c5a2b
test(functions-e2e): drop ping.ts, use /admin/host/status readiness p…
YunchuWang e6331ff
test(functions-e2e): fail wrapper script fast on test-app build failure
YunchuWang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| name: 🧪 Functions Host E2E Tests | ||
|
|
||
| # Gated Azure Functions host E2E suite. It launches a real `func start` host for | ||
| # test/e2e-functions/test-app (backed by Azurite / AzureStorage) and drives the | ||
| # app over HTTP, porting the extension repo's `BasicNode` app + xUnit tests. | ||
| # | ||
| # The test-app depends on the PUBLISHED `durable-functions` + `@azure/functions` | ||
| # packages, so it installs and runs without any in-repo package build. The suite | ||
| # self-gates: each spec SKIPS cleanly when the Azure Functions Core Tools (`func`) | ||
| # or the Azurite storage emulator are unavailable, and RUNS when both are present. | ||
| # In CI below we install and start both, so the suite runs for real; the self-skip | ||
| # is the safety net that keeps the job green if a prerequisite fails to come up. | ||
| # | ||
| # Triggering: this suite is run ad hoc (manual dispatch) for now while it beds in, | ||
| # so it does not gate day-to-day PRs. Start it from the Actions tab via "Run | ||
| # workflow" (workflow_dispatch). To re-enable automatic runs on pull requests | ||
| # later, add a `pull_request` trigger scoped to `test/e2e-functions/**` and this | ||
| # workflow file. NOTE: workflow_dispatch only appears in the Actions UI once this | ||
| # file exists on the repository's default branch. | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| functions-e2e-tests: | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| node-version: ["22.x"] | ||
| name: "functions-e2e (node ${{ matrix.node-version }})" | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: 📥 Checkout code | ||
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 | ||
|
|
||
| - name: ⚙️ NodeJS - Install | ||
| uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| registry-url: "https://registry.npmjs.org" | ||
|
|
||
| # Root install provides jest + ts-jest used to run the spec files. The specs | ||
| # do not depend on the core durabletask-js packages, so no workspace build is | ||
| # required here. | ||
| - name: ⚙️ Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: 🔧 Install Azurite and Azure Functions Core Tools | ||
| run: npm install -g azurite azure-functions-core-tools@4 | ||
|
|
||
| - name: 🗄️ Start Azurite | ||
| run: | | ||
| mkdir -p /tmp/azurite | ||
| azurite --silent --location /tmp/azurite \ | ||
| --blobPort 10000 --queuePort 10001 --tablePort 10002 & | ||
| echo "Waiting for Azurite blob endpoint..." | ||
| for i in $(seq 1 30); do | ||
| if (echo > /dev/tcp/127.0.0.1/10000) >/dev/null 2>&1; then | ||
| echo "Azurite is up." | ||
| break | ||
| fi | ||
| sleep 1 | ||
| done | ||
|
|
||
| # Installs the PUBLISHED durable-functions / @azure/functions packages and | ||
| # compiles the ported BasicNode app to dist/ (consumed by `func start`). | ||
| - name: 📦 Install + build test-app | ||
| working-directory: test/e2e-functions/test-app | ||
| run: | | ||
| npm install | ||
| npm run build | ||
|
|
||
| - name: ✅ Run Functions host E2E tests | ||
| run: npm run test:e2e:functions:internal | ||
|
YunchuWang marked this conversation as resolved.
|
||
| timeout-minutes: 20 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| /** @type {import('ts-jest').JestConfigWithTsJest} */ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT License. | ||
| // | ||
| // Dedicated jest config for the gated Azure Functions host e2e suite. It is | ||
| // intentionally NOT part of the default `npm test` / workspaces test run and is | ||
| // only executed via `npm run test:e2e:functions:internal`. These specs drive a | ||
| // real `func start` host over HTTP and skip cleanly without the local toolchain. | ||
| module.exports = { | ||
| preset: "ts-jest", | ||
| testEnvironment: "node", | ||
| roots: ["<rootDir>/test/e2e-functions"], | ||
| testMatch: ["**/test/e2e-functions/**/*.spec.ts"], | ||
| moduleFileExtensions: ["ts", "tsx", "js", "jsx", "json", "node"], | ||
| transform: { | ||
| "^.+\\.tsx?$": [ | ||
| "ts-jest", | ||
| { | ||
| tsconfig: "tsconfig.base.json", | ||
| }, | ||
| ], | ||
| }, | ||
| globalSetup: "<rootDir>/test/e2e-functions/global-setup.ts", | ||
| globalTeardown: "<rootDir>/test/e2e-functions/global-teardown.ts", | ||
| // Host cold-start (extension-bundle download + worker spin-up) dominates. | ||
| testTimeout: 300_000, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| #!/bin/bash | ||
|
YunchuWang marked this conversation as resolved.
|
||
| # Script to run the gated Azure Functions host E2E tests. | ||
| # | ||
| # This mirrors scripts/test-e2e-azuremanaged.sh. The suite launches a real | ||
| # Azure Functions host ("func start") for test/e2e-functions/test-app, backed by | ||
| # Azurite (the local Azure Storage emulator), and drives it over HTTP. | ||
| # | ||
| # The test-app depends on the PUBLISHED durable-functions / @azure/functions | ||
| # packages, so it installs and builds without any in-repo package build. | ||
| # | ||
| # Prerequisites (the suite SKIPS cleanly if any are missing): | ||
| # - Azure Functions Core Tools ('func') v4 on PATH | ||
| # - Azurite reachable on 127.0.0.1:10000 (started here if the 'azurite' CLI is | ||
| # installed and the port is free) | ||
|
|
||
| set -uo pipefail | ||
|
|
||
| ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| TEST_APP_DIR="$ROOT_DIR/test/e2e-functions/test-app" | ||
|
|
||
| AZURITE_HOST="127.0.0.1" | ||
| AZURITE_BLOB_PORT="10000" | ||
|
|
||
| started_azurite="" | ||
|
|
||
| azurite_up() { | ||
| (exec 3<>"/dev/tcp/$AZURITE_HOST/$AZURITE_BLOB_PORT") 2>/dev/null && exec 3>&- 3<&- | ||
| } | ||
|
|
||
| if ! azurite_up; then | ||
| if command -v azurite >/dev/null 2>&1; then | ||
| echo "Starting Azurite..." | ||
| AZURITE_DATA="$(mktemp -d)" | ||
| azurite --silent --location "$AZURITE_DATA" \ | ||
| --blobPort 10000 --queuePort 10001 --tablePort 10002 & | ||
| started_azurite="$!" | ||
| for _ in $(seq 1 30); do | ||
| azurite_up && break | ||
| sleep 1 | ||
| done | ||
| else | ||
| echo "Azurite is not running and 'azurite' is not installed; the suite will skip." | ||
| fi | ||
| fi | ||
|
|
||
| # Install + build the ported BasicNode app against the published durable-functions | ||
| # package so 'func start' has a dist/ to serve. If this fails there is no app to | ||
| # run, so fail fast instead of letting the suite skip and masking the error. | ||
| echo "Installing + building test-app..." | ||
| if ! ( cd "$TEST_APP_DIR" && npm install && npm run build ); then | ||
| echo "test-app install/build failed." >&2 | ||
| if [ -n "$started_azurite" ]; then | ||
| echo "Stopping Azurite..." | ||
| kill "$started_azurite" 2>/dev/null || true | ||
| fi | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Running Functions host E2E tests..." | ||
| ( cd "$ROOT_DIR" && npm run test:e2e:functions:internal ) | ||
| status=$? | ||
|
|
||
| if [ -n "$started_azurite" ]; then | ||
| echo "Stopping Azurite..." | ||
| kill "$started_azurite" 2>/dev/null || true | ||
| fi | ||
|
|
||
| exit $status | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| # Azure Functions host E2E tests (gated) | ||
|
|
||
| This suite launches a **real Azure Functions host** (`func start`) for the ported | ||
| app under [`test-app/`](./test-app), backed by **Azurite** (the local Azure | ||
| Storage emulator) using the Durable Task **AzureStorage** provider, and drives the | ||
| app entirely over HTTP. The Node.js worker, the Functions host, and the Durable | ||
| extension all cooperate exactly as they would in production. | ||
|
|
||
| It is a faithful port of the Azure Functions host E2E suite from the | ||
| [`azure-functions-durable-extension`](https://github.com/Azure/azure-functions-durable-extension) | ||
| repo — the `BasicNode` app plus its xUnit test classes — adapted to Jest. Expected | ||
| strings and Node-specific bug annotations come from that repo's | ||
| `NodeTestLanguageLocalizer`. | ||
|
|
||
| ## Published `durable-functions` dependency | ||
|
|
||
| The [`test-app`](./test-app) is wired to the **published** npm packages | ||
| `durable-functions` (`^3.1.0`) and `@azure/functions` (`^4.11.2`), so it installs | ||
| and builds without any in-repo package build. This is what makes the suite | ||
| genuinely runnable. | ||
|
|
||
| To run the app against an **in-repo** build of the Azure Functions durable package | ||
| instead, change the `durable-functions` dependency in | ||
| [`test-app/package.json`](./test-app/package.json) to a `file:` reference (e.g. | ||
| `file:../../../packages/azure-functions-durable`) once that package exists on your | ||
| branch, then re-run `npm install`. A note to this effect lives in that | ||
| `package.json` (`comment_durable-functions`). | ||
|
|
||
| ## Gating / skip model | ||
|
|
||
| The suite **skips cleanly** (it never fails) unless everything it needs is present, | ||
| so it is safe to leave wired into CI and to run locally without setup: | ||
|
|
||
| - **Prerequisite detection** — `global-setup.ts` only starts the shared `func` | ||
| host when all of these hold (otherwise every spec skips): | ||
| 1. Azure Functions Core Tools (`func`) v4 is on `PATH`, | ||
| 2. Azurite is reachable on `127.0.0.1:10000`, and | ||
| 3. the test-app is installed and built (`test-app/node_modules` + `test-app/dist`). | ||
| - **One shared host** — `global-setup.ts` starts a single `func start` host for the | ||
| whole run (mirroring the C# `FunctionAppFixture`) and writes its base URL to a | ||
| preflight file; the specs drive it over HTTP and `global-teardown.ts` stops it. | ||
| - **Isolation** — these specs are only run by the dedicated | ||
| `jest.functions-e2e.config.js` via `npm run test:e2e:functions:internal`. They are | ||
| **not** part of the default `npm test` / workspaces test run, nor of | ||
| `test:e2e:internal` (which is scoped to `tests/e2e`). | ||
| - **CI** — `.github/workflows/functions-e2e-tests.yaml` installs `func` + Azurite, | ||
| installs/builds the test-app, and runs the suite on PRs that touch | ||
| `test/e2e-functions/**`. The self-skip is the safety net if a prerequisite fails | ||
| to come up. | ||
|
|
||
| ## What it covers | ||
|
|
||
| One spec per area, mirroring the extension repo's test classes. Because this is the | ||
| **Storage** backend, `[Trait("Node-DTS","Skip")]` and `[Trait("DTS","Skip")]` tests | ||
| are **included** (those skips apply only to the DTS backend); `[Trait("Node","Skip")]` | ||
| tests are `it.skip` with a comment citing the same reason. | ||
|
|
||
| | Spec | Ported from | Notes | | ||
| | ----------------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------- | | ||
| | `hello-cities.spec.ts` | `HelloCitiesTest` | activity chaining → `Hello Tokyo!` etc. | | ||
| | `activity-input-type.spec.ts` | `ActivityInputTypeTests` | `byte[]`/`int[]`/string/custom-class serialization | | ||
| | `error-handling.spec.ts` | `ErrorHandlingTests` | rethrow/catch/retry activity & entity; skips dotnet-only FailureDetails + custom-exception-properties tests | | ||
| | `external-event.spec.ts` | `ExternalEventTests` | raise to running/completed/missing (raise-to-completed error skipped per #645) | | ||
| | `suspend-resume.spec.ts` | `SuspendResumeTests` | suspend/resume running, suspended, completed | | ||
| | `terminate.spec.ts` | `TerminateOrchestratorTests` | terminate running/terminated/completed/nonexistent | | ||
| | `timeout.spec.ts` | `TimeoutTests` | `Task.any` activity-vs-timer race | | ||
| | `rewind.spec.ts` | `RewindOrchestratorTests` | fail → rewind → complete; invocation counts; rewind-only-failed | | ||
| | `is-replaying.spec.ts` | `IsReplayingTests` | `isReplaying` flags (ConditionalLog #564 and FanOutFanIn #679 skipped) | | ||
| | `large-output.spec.ts` | `LargeOutputOrchestratorTests` | 65 KB via status URI + 4.5 MB via query trigger | | ||
| | `orchestration-query.spec.ts` | `OrchestrationQueryTests` | `GetAllInstances` / `GetRunningInstances` | | ||
| | `purge.spec.ts` | `PurgeInstancesTests` | purge by time / by entity id (purge-without-start-time #644 skipped) | | ||
| | `class-based-entity.spec.ts` | `ClassBasedEntityTests` | class-based entity state | | ||
|
|
||
| ### Node bug annotations honored | ||
|
|
||
| - [#642](https://github.com/Azure/azure-functions-durable-js/issues/642) — entity | ||
| error text: inner "This entity failed"/"More information" assertions omitted. | ||
| - [#645](https://github.com/Azure/azure-functions-durable-js/issues/645) — | ||
| raise-external-event to a completed instance: error assertions omitted. | ||
| - [#564](https://github.com/Azure/azure-functions-durable-js/issues/564) — | ||
| `isReplaying` undefined before the first `yield`: `IsReplayingConditionalLog` | ||
| skipped. | ||
| - [#679](https://github.com/Azure/azure-functions-durable-js/issues/679) — | ||
| `IsReplayingFanOutFanIn` skipped. | ||
| - [#644](https://github.com/Azure/azure-functions-durable-js/issues/644) — purge | ||
| without a start time: those purge tests skipped. | ||
| - Node swallows suspend/resume/terminate of a **terminal** instance and returns | ||
| success (`200`); the specs assert that behavior. | ||
|
|
||
| The only test-app deviation from `BasicNode` is the published-dependency wiring | ||
| (see `test-app/package.json`); the Durable function code is otherwise kept close | ||
| to the source app. Host readiness is detected by polling `/admin/host/status` | ||
| for `state == "Running"`, the same way the extension's C# `FunctionAppProcess` | ||
| fixture does. | ||
|
|
||
| ## Running locally | ||
|
|
||
| You need the [Azure Functions Core Tools v4](https://learn.microsoft.com/azure/azure-functions/functions-run-local) | ||
| and [Azurite](https://learn.microsoft.com/azure/storage/common/storage-use-azurite). | ||
|
|
||
| ```bash | ||
| # 1. Start Azurite (blob/queue/table on 10000/10001/10002) | ||
| npm install -g azurite | ||
| azurite --silent --location /tmp/azurite --blobPort 10000 --queuePort 10001 --tablePort 10002 & | ||
|
|
||
| # 2. Install the Core Tools (if needed) | ||
| npm install -g azure-functions-core-tools@4 | ||
|
|
||
| # 3. Install root dev deps (jest + ts-jest) | ||
| npm ci | ||
|
|
||
| # 4. Install + build the test-app (against the published durable-functions) | ||
| cd test/e2e-functions/test-app | ||
| npm install | ||
| npm run build | ||
| cd - | ||
|
|
||
| # 5. Run the suite | ||
| npm run test:e2e:functions:internal | ||
| ``` | ||
|
|
||
| Or use the convenience wrapper, which starts Azurite (if the CLI is installed), | ||
| installs + builds the test-app, and runs the suite: | ||
|
|
||
| ```bash | ||
| npm run test:e2e:functions | ||
| ``` | ||
|
|
||
| If any prerequisite is missing, the suite skips with a `[functions-e2e]` note | ||
| instead of failing. | ||
|
|
||
| ## Follow-ups (deferred) | ||
|
|
||
| The following extension-repo test areas are **not** ported yet. Most need extra | ||
| infrastructure (an OTLP collector, multi-version host config, scheduled starts) or | ||
| an orchestration that is not part of `BasicNode`: | ||
|
|
||
| - **DistributedTracing / DistributedTracingEntities** — need an OTLP collector. | ||
| - **Versioning / EntityVersioning** — need a multi-version host configuration. | ||
| - **DedupeStatuses**, **GetOrchestrationHistory**, **HTTPFeature**, **Restart**, | ||
| **Scheduled** — not yet ported. | ||
| - **`PurgeOnlyPurgesTerminalOrchestrations`** — needs a `HelloActivityDIFailure` | ||
| orchestration that is not present in `BasicNode`. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.