Avoid minor releases or redefine minor release to relax SemVer restrictions

[tedepstein]

Discussed on the TSC call, Sep. 26, 2019.

Here's a summary:

• I found it difficult to think through certain design challenges with JSON Schema alignment, especially w.r.t. nullable, because there was an apparent conflict between the need for a 3.1 release with this change vs. adhering strictly to Semantic Versioning.
• In our discussion, it came to light that this is a recurring problem with Semantic Versioning, and it has been criticized for these reasons:
  • It tends to take up time and energy with discussions and problem-solving that are often not the most important things for a team to focus on.
  • The rigid requirement of backward compatibility doesn't work well for language specifications in particular, where the intent of a minor release is to introduce minor, additive, or evolutionary changes. A minor release can be consistent with that intent, without adhering strictly to SemVer's backward compatibility requirement.
  • Where these changes might risk breakage, even for a very small group of users or implementations, even if those breakages are easily detected and repaired, Semantic Versioning forces a no-win situation, where you have to make it a major release, or defer the change to the next major release, or consciously make an exception to the semantic versioning policy (knowingly breaking a promise that you've made, and hoping the consequences won't be too bad).
  • To user communities who are especially sensitive to stability and vendor support, calling the release 3.1 vs. 4.0 can have a significant impact on the perception of that specification version, and willingness to adopt. Marketing understands this, and needs to weigh in the versioning decision -- whether to call it a major or a minor release -- so they can effectively manage the positioning and communication around the release. But these considerations can come in conflict with the rigid backward compatibility requirements of SemVer.
• Clarifications of the current specification are a tricky grey area. If the TSC meant for the specification to say one thing, but the specification clearly says something else, SemVer says we have to treat the correction as a breaking change, therefore put it into a major version release. If the specification itself is ambiguous or unclear, then it's more a matter of judgment as to whether the correction can go into a patch, minor or major release.
• We noted that this is the first minor release of the OpenAPI Specification since adopting Semantic Versioning in OAS 3.0. For this group, working on this specification, SemVer is still an experiment.

We discussed a few possible paths forward:

• Keep SemVer in the spec as the official policy, but don't worry so much about breaking changes, where we think the breakage only affects edge cases. Interpret SemVer as "best-effort SemVer." We promise to try not to break things in minor releases, and we promise to keep the impact of breakages as low as possible, to the best of our judgment.
• Ditch SemVer entirely, and write our own versioning policy, or find an alternative versioning policy that works well for our purposes.
• Avoid doing minor releases of the OpenAPI spec, at least most of the time. Make each significant specification release a new major version. Exercise reasonable precaution in making breaking changes, call out breaking changes explicitly, and try to get the user community used to the idea that a major release isn't something to be afraid of, it's just how we roll.

I'll add a fourth idea:

• Keep a modified version of SemVer in the spec. Say we adhere to Semantic Versioning, but minor releases may include breaking changes that are expected to have minor impact. With each minor release, explicitly call out the breaking changes in the spec itself (e.g. in a version history appendix) or in a supplementary document.

[sebastien-rosset]

Related to this, the 3.0.2 spec states:

Subsequent minor version releases of the OpenAPI Specification (incrementing the minor version number) SHOULD NOT interfere with tooling developed to a lower minor version and same major version. Thus a hypothetical 3.1.0 specification SHOULD be usable with tooling designed for 3.0.0.

However, the proposed 3.1.0 draft does not follow that promise. I understand the spec uses the term "SHOULD NOT", not "MUST NOT", so technically it is ok. As a result, it would be good to add a warning in 3.1 that it may break the 3.0 tooling (principle of least surprise).

As an example, in 3.0.2, the "type" attribute must be a string:

type - Value MUST be a string. Multiple types via an array are not supported.

In 3.1.0 the type can be an array:

type - Used to restrict the allowed types as specified in JSON Schema, but may be modified by the nullable keyword.

Thus it's very likely that 3.0 tooling will fail or have problems processing a 3.1 OAS spec, at least in the cases when a 3.1 spec uses a type array. There may be other cases as well.

[handrews]

@sebastien-rosset some of that has to do with the definition of "usable."

A valid 3.1 spec that is also a valid 3.0 spec SHOULD be usable with a 3.0 implementation. That is the case with type – if its value is a string, it is both a valid 3.0 and 3.1 field, and can be passed to either version of tooling. If its value is an array, it is valid 3.1 but not valid 3.0 and there is no expectation that a 3.0 implementation would handle it correctly.

Basically, you can take a valid 3.0 document, change the OpenAPI version field to say 3.1, and it should still behave the same. The other direction is not expected to work.

[sebastien-rosset]

ok, thanks for the clarification.

[tedepstein]

Here's the new language introduced to v3.0.3, the next planned patch release of the OpenAPI Specification:

Each new minor version of the OpenAPI Specification SHALL allow any OpenAPI document that is valid against any previous minor version of the Specification, within the same major version, to be updated to the new Specification version with equivalent semantics. Such an update MUST only require changing the openapi property to the new minor version.

For example, a valid OpenAPI 3.0.2 document, upon changing its openapi property to 3.1.0, SHALL be a valid OpenAPI 3.1.0 document, semantically equivalent to the original OpenAPI 3.0.2 document. New minor versions of the OpenAPI Specification MUST be written to ensure this form of backward compatibility.

[tedepstein]

Closing this for now.

Though we now have a good definition of backward compatibility, there is still a question of whether we should do minor releases in the future. I still think we should avoid them, because there are so many minor changes we'd like to make that would, strictly speaking, break backward compatibility, and/or consume extra bandwidth trying to solve those without breaking changes. But we agreed on a TSC call that we'd revisit this question as and when we're considering the possibility of another minor release.