Skip to content

Conversation

@pietern
Copy link
Contributor

@pietern pietern commented Jan 7, 2026

Changes

When deleting multiple secret scope ACLs, if one ACL returns NotFound (due to race conditions or eventual consistency), the code would return nil and exit the loop early. This prevented remaining ACLs from being deleted. Changed to continue to skip NotFound errors and proceed with remaining deletions.

Tests

Added local acceptance test that simulates the race condition by manually deleting one ACL before bundle deploy attempts to delete both. Test confirms that with the fix, all ACLs are properly deleted even when one returns NotFound.

## Changes

When deleting multiple secret scope ACLs, if one ACL returns NotFound
(due to race conditions or eventual consistency), the code would
`return nil` and exit the loop early. This prevented remaining ACLs
from being deleted. Changed to `continue` to skip NotFound errors and
proceed with remaining deletions.

## Tests

Added local acceptance test that simulates the race condition by
manually deleting one ACL before bundle deploy attempts to delete both.
Test confirms that with the fix, all ACLs are properly deleted even
when one returns NotFound.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
title "Manually delete one ACL to simulate race condition"
# This simulates a race condition where an ACL is deleted between when we
# list them and when bundle deploy tries to delete them
trace $CLI secrets delete-acl test-scope user1@example.com
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how can sequential call before "bundle deploy" simulate race condition?

If I understand correctly the race condition this refers too is the one in setACLs() function which indeed does list + delete,

# list them and when bundle deploy tries to delete them
trace $CLI secrets delete-acl test-scope user1@example.com

trace $CLI bundle deploy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: add readplan variant to this test, it's a few lines of bash to make it test that this still works with deploy --plan:

% git grep -i readplan acceptance/bundle/resources/jobs/delete_task/
acceptance/bundle/resources/jobs/delete_task/script:$CLI bundle deploy $(readplanarg out.plan_create.direct.json)
acceptance/bundle/resources/jobs/delete_task/script:$CLI bundle deploy $(readplanarg out.plan_update.direct.json)
acceptance/bundle/resources/jobs/delete_task/test.toml:EnvMatrix.READPLAN = ["", "1"]

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Jan 7, 2026

Commit: 9cc0aa3

Run: 20784408749

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 1 18 5 2 380 678 61:38
❌​ aws windows 4 23 2 379 676 85:56
🟨​ aws-ucws linux 14 9 2 532 555 306:28
🟨​ aws-ucws windows 19 4 2 534 553 92:22
🟨​ azure linux 2 4 16 3 377 677 31:23
❌​ azure windows 7 11 7 3 376 675 32:03
🟨​ azure-ucws linux 13 5 3 528 554 339:56
🟨​ azure-ucws windows 18 3 530 552 83:16
🟨​ gcp linux 13 5 3 370 683 28:45
❌​ gcp windows 2 18 3 370 681 25:07
40 interesting tests: 24 KNOWN, 11 FAIL, 4 flaky, 1 SKIP
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/deployment/bind/alert 🙈​S 🙈​S 🙈​S 🙈​S 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/generate/alert 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/basic 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/with_file 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
❌​ TestAccept/bundle/templates/default-python/combinations/classic ✅​p ❌​F ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=no/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=yes/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=no/PY=yes/READPLAN=1 ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=no/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=no/READPLAN=1 ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=yes/NBOOK=yes/PY=yes/READPLAN=1 ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=no/READPLAN= ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=no/READPLAN= ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=yes/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.10 ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11 ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12 ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13 ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/ssh/connection ❌​F ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🟨​ TestExport 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestExportWithFileFlag 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestImportDir 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestImportDirDoesNotOverwrite 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestImportDirWithOverwriteFlag 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestImportFileFormatAuto 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestImportFileFormatSource 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
Top 32 slowest tests (at least 2 minutes):
duration env testname
9:14 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
8:41 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:56 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:11 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic
6:34 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
6:23 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:52 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:50 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:49 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:37 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:35 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:33 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:24 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:19 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:19 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:07 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:17 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:23 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:12 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic
2:30 azure windows TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=yes/READPLAN=
2:29 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12
2:20 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:11 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
2:11 azure linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes/READPLAN=
2:10 gcp windows TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes/READPLAN=
2:08 gcp windows TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=no/READPLAN=
2:05 azure-ucws windows TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.10
2:05 azure windows TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=yes/READPLAN=
2:03 azure linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.12
2:02 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:01 azure-ucws windows TestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
2:00 azure-ucws linux TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=yes/READPLAN=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants