From 905dacd44caf1f6979d217e59e3ec0be1d16abc8 Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 16:45:27 -0400 Subject: [PATCH 01/12] Addressing #1389, and clearing the way for PRs #2046 and #1977. This adds a formal proposal to clarify the semantics of nullable, providing the necessary background and links to related resources. --- proposals/003_Clarify-Nullable.md | 135 ++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 proposals/003_Clarify-Nullable.md diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md new file mode 100644 index 0000000000..c3da7b0667 --- /dev/null +++ b/proposals/003_Clarify-Nullable.md @@ -0,0 +1,135 @@ +# Clarify Semantics of `nullable` in OpenAPI 3.0 + + +## Metadata + +|Tag |Value | +|---- | ---------------- | +|Proposal |[003](https://github.com/OAI/OpenAPI-Specification/tree/master/proposals/003_Clarify-Nullable.md)| +|Authors|[Ted Epstein](https://github.com/tedepstein)| +|Review Manager |TBD| +|Status |Proposal| +|Implementations |N/A| +|Issues | [1900](https://github.com/OAI/OpenAPI-Specification/issues/1900), [1368](https://github.com/OAI/OpenAPI-Specification/issues/1368), [1389](https://github.com/OAI/OpenAPI-Specification/issues/1389), [1957](https://github.com/OAI/OpenAPI-Specification/pull/1957), [2046](https://github.com/OAI/OpenAPI-Specification/pull/2046), [1977](https://github.com/OAI/OpenAPI-Specification/pull/1977#issuecomment-533333957) | +|Previous Revisions |N/A | + +.Change Log + +|Date |Responsible Party |Description | +|Oct 31, 2019 | Ted Epstein | Initial proposal | + +## Introduction + +This proposal aims to clarify the semantics of the `nullable` keyword in OpenAPI 3.0. This clarification would resolve ambiguities, reinforce the intended alignment with JSON Schema, and guidance for schema validators, translators, and other tools. + +## Motivation + +The documentation of the `nullable` keyword is incomplete and ambiguous, leaving many questions unanswered, and causing significant difficulty in reconciling certain assumed semantics with JSON Schema. + +To summarize the problems: + +* `nullable: true` is an _expanding assertion_ that doesn't fit JSON Schema's constraint-based processing model. It is not clear how it interacts with other keywords, and within what scope. +* `nullable: false`, which is the default value, is not clearly defined, and could be interpreted in a way that breaks fundamental assumptions of JSON Schema. +* Different OpenAPI schema validators and other tool implementations are likely to have different behaviors, because the semantics of `nullable` are not specified well enough. +* OpenAPI Schema Objects cannot be interpreted correctly by standard JSON Schema processors, because of the above issues. +* Depending on the interpretation, `nullable` might interact with `oneOf` and `anyOf` in problemantic and counter-intuitive ways. + +The solution proposed herein should: +* Clarify the boundaries around `nullable`, so we know how it interacts with other assertions, applicators, subtypes and supertypes within its context. +* Clarify the meaning of `nullable: false`. +* Reaffirm the intended alignment of OpenAPI's Schema Object with JSON Schema, and reconcile `nullable` with JSON Schema semantics. + +Further details follow. + +### Expanding vs. Constraining Assertions + +`nullable: true` is an _expanding assertion_, meaning it has the effect of expanding the range of acceptable values. By contrast, JSON Schema's central operating principle is constraint-based, where constraints are cumulative, immutable, and each constraint has veto power to disallow some range of values. + +The semantics of constraining assertions are well defined by JSON Schema and implemented in many JSON Schema validators and other tools. But JSON Schema doesn't have expanding assertions, so those well-defined semantics don't apply to `nullable`. + +We would have to specify how `nullable` interacts with constraining assertions and boolean applicators like `allOf` and `anyOf`. + +### Interpretation of `nullable: false` + +The documentation specifies that `nullable: false` is the default, but doesn't clearly state what that means. + +One reasonable interpretation suggests that null values are disallowed unless `nullable` is explicitly set to `true`. This breaks a fundamental rule of JSON Schema, which states that an empty object `{}` is a valid schema that permits all values, with no constraints. Breaking that rule takes OpenAPI's Schema Object even further out of alignment with JSON Schema's processing model. + +However, the OpenAPI 3.0 specification doesn't explicitly say that untyped schemas disallow null values. + +Here are the relevant parts: + +#### Data Types +> Primitive data types in the OAS are based on the types supported by the JSON Schema Specification Wright Draft 00. Note that integer as a type is also supported and is defined as a JSON number without a fraction or exponent part. null is not supported as a type (see nullable for an alternative solution). Models are defined using the Schema Object, which is an extended subset of JSON Schema Specification Wright Draft 00. + +To say that null is "not supported _as a type_" would definitely disallow `type: "null"` in a schema object. But it doesn't necessarily mean that an untyped schema disallows _null values_. + +#### Definition of `nullable` +> Allows sending a null value for the defined schema. Default value is false. + +This uses the word "allows," but there's no mention of "disallows." To say that `nullable: true` _allows_ null where it would otherwise be prohibited, doesn't necessarily mean that `nullable: false` _disallows_ null where it would otherwise be allowed. + +`nullable: true` _modifies_ a typed schema by adding null to the allowed types. `nullable: false` could mean "no null values allowed" or it could just mean "no modification to the specified type assertion, if any." + +#### Schema Object +> The following properties are taken from the JSON Schema definition but their definitions were adjusted to the OpenAPI Specification. +> +> type - Value MUST be a string. Multiple types via an array are not supported. + +There is no specified adjustment to the `type` property that disallows null values. So it should defer to the JSON Schema specification, which says that, in the absence of a `type` assertion, any valid JSON value is allowed. + +So the 3.0 spec is ambiguous about null values. It's not clear whether the spec intended to disallow null values by default, even in untyped schemas. This looks more like an accidental oversight, or an unfortunate choice of words, than a clear intention. + +### Specific Questions + +Questions that are not answered by the current specification include the following: +* If a schema specifies `nullable: true` and `enum: [1, 2, 3]`, does that schema allow null values? (See [#1900](https://github.com/OAI/OpenAPI-Specification/issues/1900).) + +* Does an untyped schema (without a `type` keyword) allow null values by default? What effect, if any,does `nullable: true` have on an untyped schema? + +* Can `allOf` be used to define a nullable subtype of non-nullable base schema? (See [#1368](https://github.com/OAI/OpenAPI-Specification/issues/1368).) + +* Can `allOf` be used to define a non-nullable subtype of nullable base schema? + +* What is the correct translation of a nullable schema from OpenAPI into an equivalent JSON Schema? + +## Proposed solution + +We propose to clarify the 3.0 specification in the next patch release, to resolve these questions and align OpenAPI's Schema Object with JSON Schema's well-defined, constraint-based semantics. + +In our view, and consistent with the original intent, `nullable` should have a very limited, well-defined scope, optimized to satisfy a specific use case with minimal side effects. + +This is the proposed replacement for the `nullable` definition: +
+ +Field Name | Type | Description +---|:---:|--- +nullable | `boolean` | A `true` value expands the allowed type defined by the `type` keyword, only if `type` is explicitly defined within the same Schema Object. Other Schema Object constraints retain their defined behavior, and therefore may disallow the use of `null` as a value. A `false` value leaves the specified or default `type` unmodified. The default value is `false`. +
+ +## Detailed design + +According to the above specification, `nullable` only operates within a narrow scope, wherein its translation to JSON Schema is straightforward: + +* `nullable` is only meaningful if its value is `true`. +* `nullable: true` operates within a single Schema Object; it does not "override" or otherwise compete with supertype or subtype schemas defined with `allOf` or other applicators; and it cannot be directly "inherited" through those applicators. +* `nullable: true` is only meaningful in combination with a `type` assertion specified in the same Schema Object. `nullable` acts as a `type` modifier, allowing `null` in addition to the specified type. + +This also solves the issues of alignment with JSON Schema: + +* Since `type` is a constraint, JSON Schema's constraint-based processing model is fully applicable. Interactions between `type` and other constraining assertions and applicators are unambiguous, with each constraint having independent veto power. +* It is now clear that `nullable: false`, whether explicit or by default, _does not_ prohibit default values. Consistent with JSON Schema, an empty object allows all values, including `null`. + +## Backwards compatibility + +Spec revisions through 3.0.2 are ambiguous as described above, so any possible clarification has the potential to break existing implementations. + +With the clarification of `nullable: false`, we think the risk of actual breakage is miniscule, because the current ambiguity only affects untyped Schema Objects, which by their nature leave a lot of room for unexpected values. Any implementation that relies on schema validation to prevent null values should use explicitly typed schemas, and typed schemas unambiguously disallow `null` unless `nullable` is `true`. + +There might be a somewhat greater risk of breakage by specifying the effect of `nullable: true` as a `type` modifier. A more heavy-handed interpretation of `nullable: true`, [described here](https://github.com/OAI/OpenAPI-Specification/issues/1900#issuecomment-486772917), would make it equivalent to `allOf [s, type: null]` where `s` is the schema as specified (excluding `nullable`). This would allow nulls even where they would be prohibited by other schema keywords, like `enum`. But this interpretation introduces far greater complexity than the narrowly scoped `type` modifier. We are not aware of any OpenAPI schema validator that actually attempts this, and there is nothing in the OpenAPI spec that says `nullable` should override constraints. + +## Alternatives considered + +[Pull request #1977](https://github.com/OAI/OpenAPI-Specification/pull/1977#issuecomment-533333957) has some history of other approaches considered along the way. The first attempt assumed that `nullable: false` would prohibit null values, and attempted to work around this while maintaining backward compatibility. + +On closer inspection, the specification does not say anything about `null` values being disallowed. So we believe our interpretation is correct, and highly advantageous in its alignment with JSON Schema. \ No newline at end of file From cddbc1e845f413bcca2b1b568a35490300217f4b Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 16:54:45 -0400 Subject: [PATCH 02/12] Corrected table formatting. --- proposals/003_Clarify-Nullable.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index c3da7b0667..618d76fa16 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -16,6 +16,7 @@ .Change Log |Date |Responsible Party |Description | +|---- | ---------------- |------------| |Oct 31, 2019 | Ted Epstein | Initial proposal | ## Introduction From a64f81a198d3a07f16cd0b084933b382644e84fb Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 17:11:03 -0400 Subject: [PATCH 03/12] Minor tweaks and corrections. --- proposals/003_Clarify-Nullable.md | 32 +++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index 618d76fa16..ab64f3336e 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -21,7 +21,7 @@ ## Introduction -This proposal aims to clarify the semantics of the `nullable` keyword in OpenAPI 3.0. This clarification would resolve ambiguities, reinforce the intended alignment with JSON Schema, and guidance for schema validators, translators, and other tools. +This proposal aims to clarify the semantics of the `nullable` keyword in OpenAPI 3.0. This clarification would resolve ambiguities, reinforce the intended alignment with JSON Schema, and provide guidance for schema validators, translators, and other tools. ## Motivation @@ -30,25 +30,34 @@ The documentation of the `nullable` keyword is incomplete and ambiguous, leaving To summarize the problems: * `nullable: true` is an _expanding assertion_ that doesn't fit JSON Schema's constraint-based processing model. It is not clear how it interacts with other keywords, and within what scope. + * `nullable: false`, which is the default value, is not clearly defined, and could be interpreted in a way that breaks fundamental assumptions of JSON Schema. -* Different OpenAPI schema validators and other tool implementations are likely to have different behaviors, because the semantics of `nullable` are not specified well enough. -* OpenAPI Schema Objects cannot be interpreted correctly by standard JSON Schema processors, because of the above issues. -* Depending on the interpretation, `nullable` might interact with `oneOf` and `anyOf` in problemantic and counter-intuitive ways. + +* Different OpenAPI schema validators and other tool implementations are likely to have different behaviors because the semantics of `nullable` are not fully specified. + +* OpenAPI Schema Objects cannot be interpreted correctly by standard JSON Schema processors because of the above issues. + +* Depending on the interpretation, `nullable` might interact with `oneOf` and `anyOf` in problematic and counter-intuitive ways. The solution proposed herein should: -* Clarify the boundaries around `nullable`, so we know how it interacts with other assertions, applicators, subtypes and supertypes within its context. + +* Clarify the boundaries around `nullable` so we know how it interacts with other assertions, applicators, subtypes and supertypes within its context. + * Clarify the meaning of `nullable: false`. + * Reaffirm the intended alignment of OpenAPI's Schema Object with JSON Schema, and reconcile `nullable` with JSON Schema semantics. +* Allow a straightforward translation from `nullable` in OpenAPI to type arrays in JSON Schema. + Further details follow. ### Expanding vs. Constraining Assertions -`nullable: true` is an _expanding assertion_, meaning it has the effect of expanding the range of acceptable values. By contrast, JSON Schema's central operating principle is constraint-based, where constraints are cumulative, immutable, and each constraint has veto power to disallow some range of values. +`nullable: true` is an _expanding assertion_, meaning it has the effect of expanding the range of acceptable values. By contrast, JSON Schema's central operating principle is constraint-based, where _constraining assertions_ are cumulative, immutable, and each constraint has veto power to disallow some range of values. -The semantics of constraining assertions are well defined by JSON Schema and implemented in many JSON Schema validators and other tools. But JSON Schema doesn't have expanding assertions, so those well-defined semantics don't apply to `nullable`. +The semantics of constraining assertions are well-defined by JSON Schema and implemented in many JSON Schema validators and other tools. But JSON Schema doesn't have expanding assertions, so those well-defined semantics don't apply to `nullable`. -We would have to specify how `nullable` interacts with constraining assertions and boolean applicators like `allOf` and `anyOf`. +To address this, we need to translate `nullable: true` into a constraining assertion. Otherwise, we would have to specify in detail how `nullable` interacts with constraining assertions like `enum` and with boolean applicators like `allOf` and `anyOf`. ### Interpretation of `nullable: false` @@ -113,13 +122,16 @@ Field Name | Type | Description According to the above specification, `nullable` only operates within a narrow scope, wherein its translation to JSON Schema is straightforward: * `nullable` is only meaningful if its value is `true`. + * `nullable: true` operates within a single Schema Object; it does not "override" or otherwise compete with supertype or subtype schemas defined with `allOf` or other applicators; and it cannot be directly "inherited" through those applicators. + * `nullable: true` is only meaningful in combination with a `type` assertion specified in the same Schema Object. `nullable` acts as a `type` modifier, allowing `null` in addition to the specified type. This also solves the issues of alignment with JSON Schema: * Since `type` is a constraint, JSON Schema's constraint-based processing model is fully applicable. Interactions between `type` and other constraining assertions and applicators are unambiguous, with each constraint having independent veto power. -* It is now clear that `nullable: false`, whether explicit or by default, _does not_ prohibit default values. Consistent with JSON Schema, an empty object allows all values, including `null`. + +* It is now clear that `nullable: false`, whether explicit or by default, _does not_ prohibit null values. Consistent with JSON Schema, an empty object allows all values, including `null`. ## Backwards compatibility @@ -127,7 +139,7 @@ Spec revisions through 3.0.2 are ambiguous as described above, so any possible c With the clarification of `nullable: false`, we think the risk of actual breakage is miniscule, because the current ambiguity only affects untyped Schema Objects, which by their nature leave a lot of room for unexpected values. Any implementation that relies on schema validation to prevent null values should use explicitly typed schemas, and typed schemas unambiguously disallow `null` unless `nullable` is `true`. -There might be a somewhat greater risk of breakage by specifying the effect of `nullable: true` as a `type` modifier. A more heavy-handed interpretation of `nullable: true`, [described here](https://github.com/OAI/OpenAPI-Specification/issues/1900#issuecomment-486772917), would make it equivalent to `allOf [s, type: null]` where `s` is the schema as specified (excluding `nullable`). This would allow nulls even where they would be prohibited by other schema keywords, like `enum`. But this interpretation introduces far greater complexity than the narrowly scoped `type` modifier. We are not aware of any OpenAPI schema validator that actually attempts this, and there is nothing in the OpenAPI spec that says `nullable` should override constraints. +There might be a somewhat greater risk of breakage by specifying the effect of `nullable: true` as a `type` modifier. A more heavy-handed interpretation of `nullable: true`, [described here](https://github.com/OAI/OpenAPI-Specification/issues/1900#issuecomment-486772917), would make it equivalent to `allOf [s, type: null]` where `s` is the schema as specified (excluding `nullable`). This would allow nulls even where they would be prohibited by other schema keywords, like `enum`. But this interpretation introduces far greater complexity than the narrowly scoped `type` modifier. We are not aware of any OpenAPI schema validator that actually attempts this, and there is nothing in the OpenAPI spec that says `nullable` can override constraining assertions. ## Alternatives considered From 3f250477e9752a6b1046850d515cb52ba8b1551e Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 17:15:09 -0400 Subject: [PATCH 04/12] Correct Change Log heading. --- proposals/003_Clarify-Nullable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index ab64f3336e..81db0aee97 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -13,7 +13,7 @@ |Issues | [1900](https://github.com/OAI/OpenAPI-Specification/issues/1900), [1368](https://github.com/OAI/OpenAPI-Specification/issues/1368), [1389](https://github.com/OAI/OpenAPI-Specification/issues/1389), [1957](https://github.com/OAI/OpenAPI-Specification/pull/1957), [2046](https://github.com/OAI/OpenAPI-Specification/pull/2046), [1977](https://github.com/OAI/OpenAPI-Specification/pull/1977#issuecomment-533333957) | |Previous Revisions |N/A | -.Change Log +## Change Log |Date |Responsible Party |Description | |---- | ---------------- |------------| From d58009ab185d3001ccf42ee8743028aa26f8cc72 Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 17:41:52 -0400 Subject: [PATCH 05/12] Cleaned up notes about translation to JSON Schema and *Of inheritance semantics. --- proposals/003_Clarify-Nullable.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index 81db0aee97..ff4eb34b75 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -35,7 +35,7 @@ To summarize the problems: * Different OpenAPI schema validators and other tool implementations are likely to have different behaviors because the semantics of `nullable` are not fully specified. -* OpenAPI Schema Objects cannot be interpreted correctly by standard JSON Schema processors because of the above issues. +* Because of the above ambiguities, it is not clear how to translate an OpenAPI Schema Object into a standard JSON Schema for message validation and for other purposes. Some possible interpretations of the OpenAPI spec could make translating to JSON Schema much more difficult. * Depending on the interpretation, `nullable` might interact with `oneOf` and `anyOf` in problematic and counter-intuitive ways. @@ -123,10 +123,10 @@ According to the above specification, `nullable` only operates within a narrow s * `nullable` is only meaningful if its value is `true`. -* `nullable: true` operates within a single Schema Object; it does not "override" or otherwise compete with supertype or subtype schemas defined with `allOf` or other applicators; and it cannot be directly "inherited" through those applicators. - * `nullable: true` is only meaningful in combination with a `type` assertion specified in the same Schema Object. `nullable` acts as a `type` modifier, allowing `null` in addition to the specified type. +* `nullable: true` operates within a single Schema Object. It does not "override" or otherwise compete with supertype or subtype schemas defined with `allOf` or other applicators. It cannot be directly "inherited" through those applicators, and it cannot be applied to an inherited `type` constraint. + This also solves the issues of alignment with JSON Schema: * Since `type` is a constraint, JSON Schema's constraint-based processing model is fully applicable. Interactions between `type` and other constraining assertions and applicators are unambiguous, with each constraint having independent veto power. From d1e3d94aded3f2e48824e89df0f9c83273ff21a8 Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 17:55:27 -0400 Subject: [PATCH 06/12] Fix Change Log heading in the proposal template. --- proposals/000_OAS-proposal-template.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/000_OAS-proposal-template.md b/proposals/000_OAS-proposal-template.md index d6843f5bd2..da2a8b1eeb 100644 --- a/proposals/000_OAS-proposal-template.md +++ b/proposals/000_OAS-proposal-template.md @@ -13,7 +13,7 @@ |Issues |[{issueid}](https://github.com/OAI/OpenAPI-Specification/issues/{Issueid})| |Previous Revisions |[{revid}](https://github.com/OAI/OpenAPI-Specification/pull/{revid}) | -.Change Log +## Change Log |Date |Responsible Party |Description | |---- | ---------------- | ---------- | From c58f5bf95aeba88a9e65ca1bfad0eb482fbf2ad9 Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 18:15:36 -0400 Subject: [PATCH 07/12] Snappy answers to stupid questions. --- proposals/003_Clarify-Nullable.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index ff4eb34b75..1e513c6437 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -93,9 +93,10 @@ So the 3.0 spec is ambiguous about null values. It's not clear whether the spec ### Specific Questions Questions that are not answered by the current specification include the following: + * If a schema specifies `nullable: true` and `enum: [1, 2, 3]`, does that schema allow null values? (See [#1900](https://github.com/OAI/OpenAPI-Specification/issues/1900).) -* Does an untyped schema (without a `type` keyword) allow null values by default? What effect, if any,does `nullable: true` have on an untyped schema? +* Does an untyped schema (without a `type` keyword) allow null values by default? What effect, if any, does `nullable: true` have on an untyped schema? * Can `allOf` be used to define a nullable subtype of non-nullable base schema? (See [#1368](https://github.com/OAI/OpenAPI-Specification/issues/1368).) @@ -133,6 +134,32 @@ This also solves the issues of alignment with JSON Schema: * It is now clear that `nullable: false`, whether explicit or by default, _does not_ prohibit null values. Consistent with JSON Schema, an empty object allows all values, including `null`. +### Questions Answered + +Following are answers to the questions posed above, assuming the proposed clarification is adopted: + +#### If a schema specifies `nullable: true` and `enum: [1, 2, 3]`, does that schema allow null values? (See [#1900](https://github.com/OAI/OpenAPI-Specification/issues/1900).) + +No. The `nullable: true` assertion folds into the `type` assertion, which presumably specifies `integer` or `number`. + +While the modified `type` now allows `null`, the `enum` does not. Consistent with JSON schema, a value conforms to the schema only if it is valid against _all_ constraints. Any constraint, in this case `enum`, can cause a value to fail validation, even if that value meets all of the other constraints. + +#### Does an untyped schema (without a `type` keyword) allow null values by default? What effect, if any, does `nullable: true` have on an untyped schema? + +Yes, an untyped schema allows null values, in addition to all other types. `nullable: true` has no effect, because null values are already allowed. And `nullable: false` has no effect because it just leaves the `type` constraint unmodified. + +#### Can `allOf` be used to define a nullable subtype of non-nullable base schema? (See [#1368](https://github.com/OAI/OpenAPI-Specification/issues/1368).) + +No. Subtypes can add constraints, but not relax them. + +#### Can `allOf` be used to define a non-nullable subtype of nullable base schema? + +Yes. The subtype can specify a `type` without `nullable: true`, or can specify `not: {enum: [null]}`. + +#### What is the correct translation of a nullable schema from OpenAPI into an equivalent JSON Schema? + +A nullable type should translate into a type array with two string elements: the name of the type specified in the Schema Object, and `'null'`. + ## Backwards compatibility Spec revisions through 3.0.2 are ambiguous as described above, so any possible clarification has the potential to break existing implementations. From 92fa3a68abbd1d880807b4dc53d13d555f30c64a Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 19:45:16 -0400 Subject: [PATCH 08/12] Change single-quote 'null' to double-quote "null" Thanks, @handrews for the review. --- proposals/003_Clarify-Nullable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index 1e513c6437..99ffbc0c6f 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -158,7 +158,7 @@ Yes. The subtype can specify a `type` without `nullable: true`, or can specify ` #### What is the correct translation of a nullable schema from OpenAPI into an equivalent JSON Schema? -A nullable type should translate into a type array with two string elements: the name of the type specified in the Schema Object, and `'null'`. +A nullable type should translate into a type array with two string elements: the name of the type specified in the Schema Object, and `"null"`. ## Backwards compatibility @@ -172,4 +172,4 @@ There might be a somewhat greater risk of breakage by specifying the effect of ` [Pull request #1977](https://github.com/OAI/OpenAPI-Specification/pull/1977#issuecomment-533333957) has some history of other approaches considered along the way. The first attempt assumed that `nullable: false` would prohibit null values, and attempted to work around this while maintaining backward compatibility. -On closer inspection, the specification does not say anything about `null` values being disallowed. So we believe our interpretation is correct, and highly advantageous in its alignment with JSON Schema. \ No newline at end of file +On closer inspection, the specification does not say anything about `null` values being disallowed. So we believe our interpretation is correct, and highly advantageous in its alignment with JSON Schema. From e1c5bc9eb0f0a4aac95d8509190568f1bf4ec1c1 Mon Sep 17 00:00:00 2001 From: Ted Epstein Date: Thu, 31 Oct 2019 20:15:19 -0400 Subject: [PATCH 09/12] Clarified the proposed definition of nullable Somehow in our collaborative editing, we neglected to state that `nullable` adds `"null"` to the set of allowed types. We just said that it "expands" the `type` value, but didn't state the obvious (to us) manner of said expansion. Correcting that oversight in this commit. --- proposals/003_Clarify-Nullable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index 99ffbc0c6f..6a8f327b05 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -115,7 +115,7 @@ This is the proposed replacement for the `nullable` definition: Field Name | Type | Description ---|:---:|--- -nullable | `boolean` | A `true` value expands the allowed type defined by the `type` keyword, only if `type` is explicitly defined within the same Schema Object. Other Schema Object constraints retain their defined behavior, and therefore may disallow the use of `null` as a value. A `false` value leaves the specified or default `type` unmodified. The default value is `false`. +nullable | `boolean` | A `true` value adds `"null"` to the allowed type specified by the `type` keyword, only if `type` is explicitly defined within the same Schema Object. Other Schema Object constraints retain their defined behavior, and therefore may disallow the use of `null` as a value. A `false` value leaves the specified or default `type` unmodified. The default value is `false`.
## Detailed design From 63a71b5e9a39b9ae797716d0fed1957c3c0f5fa1 Mon Sep 17 00:00:00 2001 From: tedepstein Date: Fri, 1 Nov 2019 05:59:58 -0400 Subject: [PATCH 10/12] Corrected the alternative, heavy-handed interpretation of nullable. --- proposals/003_Clarify-Nullable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index 6a8f327b05..a7d7caec57 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -166,7 +166,7 @@ Spec revisions through 3.0.2 are ambiguous as described above, so any possible c With the clarification of `nullable: false`, we think the risk of actual breakage is miniscule, because the current ambiguity only affects untyped Schema Objects, which by their nature leave a lot of room for unexpected values. Any implementation that relies on schema validation to prevent null values should use explicitly typed schemas, and typed schemas unambiguously disallow `null` unless `nullable` is `true`. -There might be a somewhat greater risk of breakage by specifying the effect of `nullable: true` as a `type` modifier. A more heavy-handed interpretation of `nullable: true`, [described here](https://github.com/OAI/OpenAPI-Specification/issues/1900#issuecomment-486772917), would make it equivalent to `allOf [s, type: null]` where `s` is the schema as specified (excluding `nullable`). This would allow nulls even where they would be prohibited by other schema keywords, like `enum`. But this interpretation introduces far greater complexity than the narrowly scoped `type` modifier. We are not aware of any OpenAPI schema validator that actually attempts this, and there is nothing in the OpenAPI spec that says `nullable` can override constraining assertions. +There might be a somewhat greater risk of breakage by specifying the effect of `nullable: true` as a `type` modifier. A more heavy-handed interpretation of `nullable: true`, [described here](https://github.com/OAI/OpenAPI-Specification/issues/1900#issuecomment-486772917), would make it equivalent to `anyOf [s, {type: "null"}]` where `s` is the schema as specified (excluding `nullable`). This would allow nulls even where they would be prohibited by other schema keywords, like `enum`. But this interpretation introduces far greater complexity than the narrowly scoped `type` modifier. We are not aware of any OpenAPI schema validator that actually attempts this, and there is nothing in the OpenAPI spec that says `nullable` can override constraining assertions. ## Alternatives considered From 7d94b7c4ac0650318887d8f9588af50c7318a698 Mon Sep 17 00:00:00 2001 From: tedepstein Date: Fri, 1 Nov 2019 11:43:50 -0400 Subject: [PATCH 11/12] Added more explicit detail about the primary use case. --- proposals/003_Clarify-Nullable.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index a7d7caec57..d13177cc17 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -51,6 +51,14 @@ The solution proposed herein should: Further details follow. +### Primary Use Case for `nullable` + +A Schema Object allows values of any data type, unless the type is restricted by the `type` keyword. The `type` keyword restricts the schema to a single data type, which can be `"string"`, `"number"`, `"integer"`, `"boolean"`, `"array"`, or `"object"`, but cannot be `"null"`. + +Some APIs restrict values to a single data type, but also allow explicit null values. OpenAPI Schema Objects can allow explicit null values by combining the `type` and `nullable` keywords. A `nullable` value of `true` modifies a typed schema to allow non-null values of a given type, and also allow `null`. This was the envisioned use case, and the primary motivation for introducing `nullable` into the OpenAPI 3.0 spec. + +There may be other possible usage scenarios or consequences of the `nullable` keyword, the way it is specified, or the way in which the spec may be interpreted or implemented. In our view, these other scenarios should be considered side effects or oversights. To the best of our knowledge, the `nullable` keyword was not intended for any purpose other than to allow `null` in a typed schema. + ### Expanding vs. Constraining Assertions `nullable: true` is an _expanding assertion_, meaning it has the effect of expanding the range of acceptable values. By contrast, JSON Schema's central operating principle is constraint-based, where _constraining assertions_ are cumulative, immutable, and each constraint has veto power to disallow some range of values. @@ -108,7 +116,7 @@ Questions that are not answered by the current specification include the followi We propose to clarify the 3.0 specification in the next patch release, to resolve these questions and align OpenAPI's Schema Object with JSON Schema's well-defined, constraint-based semantics. -In our view, and consistent with the original intent, `nullable` should have a very limited, well-defined scope, optimized to satisfy a specific use case with minimal side effects. +In our view, and consistent with the original intent, `nullable` should have a very limited, well-defined scope. It should satisfy the primary use case, i.e. allowing `null` in a typed schema, with minimal side effects. This is the proposed replacement for the `nullable` definition:
From e456a98ec5a7d9ec696d548e3e9bf6e7584e002d Mon Sep 17 00:00:00 2001 From: tedepstein Date: Tue, 5 Nov 2019 10:37:36 -0500 Subject: [PATCH 12/12] Added a more complete explanation of the problems created by disallowing nulls by default. --- proposals/003_Clarify-Nullable.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/proposals/003_Clarify-Nullable.md b/proposals/003_Clarify-Nullable.md index d13177cc17..94457e9b85 100644 --- a/proposals/003_Clarify-Nullable.md +++ b/proposals/003_Clarify-Nullable.md @@ -73,7 +73,35 @@ The documentation specifies that `nullable: false` is the default, but doesn't c One reasonable interpretation suggests that null values are disallowed unless `nullable` is explicitly set to `true`. This breaks a fundamental rule of JSON Schema, which states that an empty object `{}` is a valid schema that permits all values, with no constraints. Breaking that rule takes OpenAPI's Schema Object even further out of alignment with JSON Schema's processing model. -However, the OpenAPI 3.0 specification doesn't explicitly say that untyped schemas disallow null values. +For example, if null values are disallowed by default, does the following `UTCDate` schema accept `null`? + +```yaml +components: + + schemas: + + OptionalDate: + type: string + format: date + nullable: true + + UTCDate: + allOf: + $ref: "#/components/schemas/OptionalDate" + not: + type: string + pattern: "^.*Z.*$" +``` + +`UTCDate` does not specify a type of its own, and does not directly specify `nullable: true`. So if `null` is disallowed by default, even for untyped schemas, then `UTCDate` won't accept nulls. If we want it to accept nulls, we have to repeat `nullable: true` in `UTCDate`. This is not at all intuitive for API designers, and it breaks with JSON Schema's rule that any value is allowed unless it's explicitly disallowed. + +On the other hand, we could say that `UTCDate` inherits `nullable: true` from `OptionalDate`, therefore null values are allowed. But this kind of inheritance logic is completely foreign to JSON Schema. So this behavior is also counterintuitive, though for a different reason. It's also difficult to implement. Any JSON Schema validator would need to be hacked in highly disruptive ways to retrofit this behavior. Or a preprocessor would have to be introduced to propagate the effect of `nullable: true` through the `*Of` inheritance hierarchy. + +Whichever semantics we choose, it gets very messy. + +### A closer look at `nullable: false` + +In fact, the OpenAPI 3.0 specification doesn't explicitly say that untyped schemas disallow null values. Here are the relevant parts: