Pass DATABRICKS_CLI_PATH and resolve the platform-specific CLI binary

[rugpanov]

Why

• On Windows, databricks bundle deploys fail with databricks CLI not found because the SDK / Terraform provider can't locate the bundled CLI when running in a subprocess.
• The path that was forwarded was extensionless (bin/databricks), but the bundled Windows binary is databricks.exe. The Go SDK / Terraform provider do a literal file lookup and do not auto-append .exe the way Windows' CreateProcess does — so the lookup fails.
• The legacy .databricks/project.json persisted the bundled CLI path (databricksPath), and AuthProvider.fromJSON preferred it over the freshly resolved path. That snapshot is version-pinned (e.g. .../databricks.databricks-2.11.0/bin/...) and, on Windows, was stored without .exe, so it overrode the corrected path during legacy→bundle migration.

This builds on #1903 (which adds the DATABRICKS_CLI_PATH env var) by also making the path it forwards correct on Windows and not letting a stale persisted path win.

What

• Forward the resolved CLI path to subprocesses via the DATABRICKS_CLI_PATH env var so they don't fall back to a PATH search (DatabricksCliAuthProvider.toEnv).
• Make CliWrapper.cliPath return the platform-specific binary name: databricks.exe on win32, databricks elsewhere.
• In AuthProvider.fromJSON, always use the freshly resolved bundled CLI path and ignore the persisted databricksPath — it's an extension-managed, version-pinned value that is stale after any upgrade. (fromSdkConfig is intentionally left alone, since its databricksCliPath comes live from the user's .databrickscfg.)
• Added explanatory comments at each site.

Verification

• Added unit tests for: the platform-specific binary name, toEnv() exposing DATABRICKS_CLI_PATH, and fromJSON ignoring a stale persisted databricksPath. Suite passes in the VS Code test host (14 relevant tests green).
• Built the win32-x64 VSIX and confirmed the bundled extension/out/extension.js contains both the env var and the .exe path, with bin/databricks.exe packaged.

This pull request and its description were written by Isaac.

[anton-107]

On the persisted databricksPath path:

DatabricksCliAuthProvider.toJSON() writes databricksPath: this.cliPath into .databricks/project.json, and AuthProvider.fromJSON() reads it back as json.databricksPath ?? cli.cliPath. So for a Windows user who connected before this fix, project.json already holds an extensionless …/bin/databricks, and after upgrading the ?? won't fall through to the new .exe-aware cli.cliPath — meaning toEnv() would forward the stale extensionless path and reproduce the exact databricks CLI not found this PR fixes.

Two things I could not confirm from the diff:

1. Is project.json rewritten with a fresh cli.cliPath on every connect/activation? The persisted value is also absolute and version-pinned (databricks.databricks-<version>/bin/...), which would be stale across any upgrade on any platform — so I suspect it does get refreshed, but I'd like to confirm.
2. If so, is there a window where toEnv() runs against a freshly-loaded provider before that rewrite happens? If yes, existing Windows users would still hit the failure on the first connect post-upgrade.

If the persisted path can win over the getter here, it might be worth ignoring/normalizing databricksPath for the bundled-CLI case so CliWrapper.cliPath stays the single source of truth. Could you confirm the rewrite timing?

[rugpanov]

Good catch. Digging in, project.json turns out to be an outdated way we used to persist config - nothing writes it anymore. But for the small fraction of users who still have one lying around, it's read once during the legacy -> bundle migration (BundleProjectManager → ProjectConfigFile.load → fromJSON), and that's exactly where this can fail.

On your questions: nothing rewrites or revalidates the file, so a persisted value is taken as-is. It only falls through when the field is absent, it never checks whether the path still exists. And since the stored path is absolute and points into the versioned extension folder (…databricks.databricks-2.11.0/bin/…), it's already pointing at the wrong install after an upgrade. This doesn't hit modern users, though: once a databricks.yml exists, auth comes from a ProfileAuthProvider built off the profile name, which resolves cli.cliPath fresh and never touches databricksPath. So it only affects the small set still on an un-migrated project.json, and the migration is the only window - which is why the fix has to live on the read side.

So I did what you suggested and made cli.cliPath the single source of truth: fromJSON now ignores databricksPath (commit 0534870), with a test that feeds a stale extensionless path plus a fresh .exe one and asserts the getter wins. I left fromSdkConfig alone since its databricksCliPath comes live from the user's .databrickscfg - intentional config rather than a stale snapshot - but happy to change that too if you'd prefer.

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 1908
• Commit SHA: 58d02a403d4198f69f9939cb1ec823ea5f58937b

Checks will be approved automatically on success.

[rugpanov]

Superseded by #1910, which runs from an upstream branch so CI (JFrog OIDC) can run — fork PRs don't get OIDC tokens/secrets, so the checks here couldn't execute. Same commit and contents.