Consider requiring a version bump on bytewise mismatches against the latest version

[sunshowers]

Here:

• Bytewise equality means that two OpenAPI documents have exactly the same contents.
• Semantic equality means that two OpenAPI documents describe the same API shape up to wire compatibility.

Bytewise equality always implies semantic equality, but not vice versa. Some examples of semantic equality that cause OpenAPI documents to be bytewise unequal are doc comment changes and type renames.

---

The Dropshot API manager detects semantic equality using drift. This was primarily intended so that wire-compatible changes could be made to endpoint handlers for old blessed versions. But, as a side-effect, this property also applies to the latest blessed version.

For example, consider renaming a type but maintaining the same structure.

• Without drift, we'd have compared bytewise equality for generated JSON schemas, if the type was present in any blessed versions, we'd have produced an error. That would have led to the developer adding a new version with the new name of the type.

• With drift, we compare semantic equality. If it's just a rename and the schema is ultimately the same, we wouldn't have forced the developer to create a new version.

But what that means is that these wire-compatible changes get built up over time, and the next time there is a semantic change we'd see not just the specific change introduced in the PR, but also all of those other built up changes. This makes it hard to review the generated schema for changes. For example, see oxidecomputer/omicron#9434 (comment).

There are a few different options to handle this. The most straightforward one is, when comparing the latest version, use both semantic and bytewise equality. Then, for blessed APIs:

• For old versions, only compare semantic equality.
• For the latest version, compare both semantic and bytewise equality. This leads to this table of actions that applies to the latest version:

semantic equality	bytewise equality	current action	proposed action
unequal	unequal	error: must be semantically equal	error: must be semantically equal
equal	unequal	ok: no need to create a new version	error: must create a new version
equal	equal	ok	ok

Another option: for versioned APIs, also generate a non-load-bearing lockstep file which lives on the side and is purely to help out with reviews and such. This does have some advantages (e.g. trivial changes would immediately be reflected, it's easier to compare diffs, and such) but does have the major disadvantage of adding even more UX confusion. In today's update watercooler, we discussed maybe having a bot which can generate useful diffs on OpenAPI documents.