[ISSUE] select_spark_version(latest=True) doesn't return the latest DBR overall because it implicitly filters to Scala 2.12

[isharfme]

Description
A call to WorkspaceClient().clusters.select_spark_version(latest=True) does not return the latest DBR overall. Instead, it appears to apply an implicit default scala="2.12" filter first and then return the latest runtime within that filtered set. This is surprising because latest=True reads as if it should return the latest runtime available in the workspace.

Reproduction

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
print(w.clusters.select_spark_version(latest=True))
print(w.clusters.select_spark_version(latest=True, scala="2.13"))
16.4.x-scala2.12
18.2.x-scala2.13

Expected behavior
select_spark_version(latest=True) should return the latest DBR available in the workspace by default, e.g. 18.2.x-scala2.13 inb the example above

Is it a regression?
INo

Debug Logs
See the real results above

Other Information
The results are from Databricks Azure running on Serverless

[saisharan0103]

The example shows latest=True returning 16.4.x-scala2.12, while latest=True, scala="2.13" returns 18.2.x-scala2.13, so the default Scala filter is changing the meaning of latest.

For a PR, I would start by separating “latest overall” from Scala-scoped selection in select_spark_version. The regression test should cover latest=True with no scala returning the highest DBR available, while explicit scala="2.12" keeps the scoped behavior.