WIP: Glossary and guide to referencing #69
Conversation
This includes both a defitions page (which could be renamed to glossary if folks like that better) and the start of a detailed guide to referencing and related topics. The referencing guide is more of a WIP than the glossary, and I'm happy to split the glossary into its own PR if folks like it enough to approve it. I just wanted to show it in context as this is how I wrote it all. I made the referencing information a section as it is too long and has too many sub-topics to be on a single page. I tried to extract the most common topics into the FAQ so that the basics can be found quickly, which might lead some folks to read the more deep-dive pages. Some pages stop rather abruptly as I was getting bogged down and figured it would be good to solicit feedback before writing too much more. I'm happy to rework any/all of this to suit what typical users want to know.
referencing/faq.md
Outdated
|
|
||
| In general, no. But in many common cases, yes, and tools exist that will attempt this. | ||
|
|
||
| There are two types of reasons why de-referenceing might not be possible: |
| There are two types of reasons why de-referenceing might not be possible: | ||
|
|
||
| * Technical reasons: | ||
| * Cyclic references are allowed in Schema Objects, and are used to describe recursive tree structures. JSON Schema itself is a recursive tree structure, so the JSON Schema metaschemas cannot be fully de-referenced (see draft-07 or earlier for how this works with `"$ref"`; draft 2020-12 uses a different technique) |
There was a problem hiding this comment.
This point about recursive references is important, and implies we can't ship a completely normalized document (all references resolved and inlined), and that some level of at least intra-document indirection must be supported. (eg; tools must be slightly more complicated than simple JSON / YAML parsers)
referencing/faq.md
Outdated
|
|
||
| ## Do I need to understand dynamic references? | ||
|
|
||
| This guide does **not** cover OAS 3.1 / JSON Schema draft 2020-12's dynamic references (`"$dynamicRef"` and `"$dynamicAnchor"`). Dynamic references seve a different purpose than discussed here. The following JSON Schema Blog posts explain some uses of dynamic references and anchors: |
There was a problem hiding this comment.
Now I want to make up a meaning for "seve" where this sentence actually makes sense... 🙃
referencing/what.md
Outdated
| | OAS Version | Object | Reference Keyword | Adjacent Keywords | Behavior | | ||
| | ----------- | ------ | ----------------- | ----------------- | -------- | | ||
| | 3.0 | Reference Object | `"$ref"` | ignored | logically replace the Reference Object with the reference target | | ||
| | 3.1 | Reference Object | `"$ref"` | `"summary"` and `"description"` allowed; others ignored | logically replace the Reference Object with a coply of the target, overwriting the target's `"summary"` and/or `"description"` fields with those of the Reference Object if they are present | |
referencing/what.md
Outdated
| | 3.x | Path Item Object | `"$ref"` | allowed under some circumstances | logically replace the Path Item Object containing the `"$ref"` with a Path Item Object that combines the fields of the target Path Item Object with the non-`"$ref"` fields of the Path Item Object containing the `"$ref"`, as long as none of those fields conflict | | ||
| | 3.1 | Schema Object | `"$ref"` | allowed | Apply the target Schema Object to the same instance location as the Schema Object containing the `"$ref"`, and combine the results with the results of other keywords in the Schema Object containing the `"$ref"` just as you would any other keyword results; this is more-or-less equivalent to using a one-element `"allOf"` | | ||
| | 3.x | Link Object | `"operationRef"` | allowed, except `"operationId"` | Treat the reference target as the target of the link described by the Link Object | | ||
| | 3.x | Descriminator Object | `"mapping"` | n/a | for each name under `"mapping"`, if the value is not the name of a Schema Object under the Components Object, treat it as a reference to the schema to use when the discriminator field matches the mapping name | |
There was a problem hiding this comment.
/Descriminator/Discrimator/
referencing/what.md
Outdated
|
|
||
| ## Non-reference linkages | ||
|
|
||
| The following connections between parts of the OAS require that some name or identifier has been seen. Due to ambiguities in the specification (which the OAI will hopefully be able to clarify in future patch releases), it is not always clear how to handle identifiers that are present in a _document_ that has been referenced, but are not present in the _exact JSON object_ targeted by any single reference. |
There was a problem hiding this comment.
Should "connections between parts of the OAS" be "connections between parts of the OAD"?
The latter sentence is hard for me to follow, it seems like a pretty abstract description of a problem which doesn't lead me to envision what a concrete example might look like. Maybe something like an Operation whose tags include a tag name not present in the OAD's tags object?
There was a problem hiding this comment.
This is great feedback, thanks! I will think on this. Maybe I should just add an example here (there might be one in my detailed processing model report that's suitable).
referencing/what.md
Outdated
| | 3.x | Components Object | Security Scheme name | Security Requirement Object | | ||
| | 3.x | Tag Object | `name` | Operation Object | | ||
| | 3.x | Operation Object | `operationId` | Link Object | | ||
| | 3.x | Server Object | n/a | Paths Object | |
There was a problem hiding this comment.
This table I had a bit of difficulty parsing at first - the headings and the relationship between them is a bit unclear. It might make more sense to switch the order around? I tried writing a reformulation that read more naturally to me (of just one row) but having done so I'm not sure it's better:
| OAS Version | Object containing identifier | Identifying Field | Identifies Object |
|---|---|---|---|
| 3.x | Operation Object | Each tags item |
A Tag Object with a corresponding name field |
Given that these are not very uniform, maybe a bullet list describing each in prose would work better.
- An Operation Object is linked to a Tag Object whose
namefield correspond to an item in the Operation'stagsarray.
etc
There was a problem hiding this comment.
Another link that comes to mind is between a Parameter Object with in=path and a corresponding template variable of the Paths Object key containing the operation - not sure if that quite fits, or if others may exist (I won't go hunting).
There was a problem hiding this comment.
Actually, in my notes Parameter Object and Server Variable Object are included as linkages. I left them out because they felt different, mostly because they connect things that are close together. But come to think of it, the Parameters Object can go in two places, and be $ref'd, so it's probably good to include. And might as well include the Server Variable Object even if it is more localized. There's also the parameters field in the Link Object which I'm not sure I noticed before now. I'll add all of these.
Feel free to go hunting! There might be more hiding somewhere, it's a big spec :-)
referencing/what.md
Outdated
|
|
||
| ## JSON Schema referencing and identification in OAS 3.1 | ||
|
|
||
| In OAS 3.1, JSON Schema combines some of the concerns of referencing and non-reference linkage. This is because in JSON Schema draft 2020-12 as included in OAS 3.1, the dialect can be set within the OAD, or through the `"$schema"` keyword in a parent Schema Object. Schemas can also be referenced by URIs declared in-schema by certain keywords. These can be URIs that cannot be used as URLs as they are not hosted separately from the containing document, or plain-name fragments that, unlike JSON Pointer fragments, can only be resolved if their delcaring keyword has been parsed. |
There was a problem hiding this comment.
Schemas can also be referenced by URIs declared in-schema by certain keywords.
This makes sense, though being stated in the most generalized terms is correct but harder to follow. A parenthetical might help clarify, maybe "declared in-schema by certain keywords (most commonly $id and $anchor)."
definitions.md
Outdated
|
|
||
| Up through OAS 3.0.3 and 3.1.0, the OAS contained a section titled "OpenAPI Document", with differing definintions depending the OAS version. The terms "OpenAPI Definition", "OpenAPI Description", and other variations and combinations were also used within the specification and/or the [learn.openapips.org](learn.openapis.org) site. | ||
|
|
||
| In September 2023, the OAI Technical Steering Committee (TSC) agreed to standardize on "OpenAPI Description" for the complete logical entity that descrribes an API, and to reserve the term "document" (lower-cased) for its common meaning. Resolving the debate between "OpenAPI Description" and "OpenAPI Definition" hinged on the observation that while "definition" is appropriate when the API code is generated from the OAD, it is not accurate when an OAD is written for an existing API. The term "description" is accurate in both cases. |
Co-authored-by: Karen Etheridge <github@froods.org>
|
@duncanbeevers @notEthan @karenetheridge @ndenny I think I have incorporated all feedback, please let me know if I missed something. @notEthan instead of making bullet points, I rearranged and renamed the table columns- please let me know if that is more readable. I'd like to keep all of those sorts of blocks of information in the same sort of table rather than switching formats if possible. I also added an example of names that look usable but probably won't be in many tools because the names are not directly referenced. It's such a weird thing that I'm not sure if that makes it less or more confusing, but it is at least concrete. |
definitions.md
Outdated
| - **document**: A local file, network resource, or other distinct entity in a particular format such as JSON or YAML | ||
| - **entry document**: The document in an OAD where processing begins, starting with an OpenAPI Object ([3.0](https://spec.openapis.org/oas/v3.0.3#openapi-object), [3.1](https://spec.openapis.org/oas/v3.1.0#openapi-object)) | ||
| - **OpenAPI Description** (**OAD**): One or more documents written according to a specific version of the OpenAPI Specification, that together describe an API | ||
| - **OpenAPI Initiative** (**OAI**): The standards body responsible for the development of the OpenAPI Specification _(not to be confused with the unrelated and more recent "OpenAI")_ |
There was a problem hiding this comment.
I suspect this should say "The organization". I think calling us a standards body is a bit presumptuous.
| - **OpenAPI Initiative** (**OAI**): The standards body responsible for the development of the OpenAPI Specification _(not to be confused with the unrelated and more recent "OpenAI")_ | ||
| - **OpenAPI Specification** (**OAS**): The formal requirements for the OpenAPI format, which exists in several versions (e.g. 3.0.3, 3.1.0) | ||
| - **reference**: A connection from one location in an OAD to another in which the reference target is identified by a URI; the `"$ref"` and `"operationRef"` keywords implement references in the OAS | ||
| - **reference removal**: The process of replacing references with their targets; not all references can be removed |
There was a problem hiding this comment.
suggest: reference inlining
| * Cyclic references can also occur when describing complex payloads such as `multipart/form-data` where media type descriptions can be nested | ||
| * References in Link Objects (`"operationRef"`) and Discriminator Objects (`"mapping"`) cannot be replaced with literal values, although there are alternatives to using references in these cases (`"operationId"` and mapping by schema component name, respectively, althouogh there can be challenges with arranging the documents properly for those techniques as well) | ||
| * Dynamic references in OAS 3.1 (which are not covered in detail by this guide) have runtime behavior which cannot always be predicted ahead of time | ||
| * Social reasons: |
There was a problem hiding this comment.
I didn't totally understand the objections in today's discussion, but I felt perhaps there was a desire to acknowledge these concerns, while not putting them front-and-center.
Rather than a rich breakdown of each of these valid social concerns, an extremely terse caveat could cover the same terrain from a slightly-higher elevation.
| * Social reasons: | |
| * Social reasons: | |
| Complex OpenAPI Descriptions may have many stakeholders, and be deployed and consumed in many contexts. Authors of OpenAPI Descriptions may need to consider these contexts when de-referencing. | |
| * Different owners; parts of an API may be owned by different teams, and may be deployed on different schedules | |
| * Security Policies; parts of an API may not be accessible for de-referencing by all consumers |
In the service of brevity, I sort of squashed-together the Large-scale and Independent Deployment bits of the original bullet points.
The Standards Compliance piece is the bullet I understand the least (in terms of how an OAD might need to be split-up), but if there's a way to describe it succinctly, or to roll it into one of the other bullets, that seems like a win.
Again, I'm just trying to respond to what felt like a desire to downplay the (over)-articulation of these caveats, and to highlight the Happy Path for casual readers of these documents.
The concerns seem valid, but those who are affected by those concerns will likely have their ears pricked by the presence of the keywords, and don't need their concerns articulated in this context.
There was a problem hiding this comment.
I like these suggestions.
My general point in these discussions has been to observe that in a compile-time contract context, it might be helpful to publish a version of the description that is optimized for the consumer's point of view. In this case, you might consider doing things like merging multiple documents into a single one, and/or you might exclude operations to which the consumer was not entitled. In this way, the points about different owners and different security polices would be opaque (dynamic, just-in-time contract probably wouldn't ever fit this simplest of contexts).
|
Closing this for now in favor of #79 for the definitions/glossary and doing more work on the references parts before re-submitting. |
This includes both a defitions page (which could be renamed to glossary if folks like that better) and the start of a detailed guide to referencing and related topics.
The referencing guide is more of a WIP than the glossary, and I'm happy to split the glossary into its own PR if folks like it enough to approve it. I just wanted to show it in context as this is how I wrote it all.
I made the referencing information a section as it is too long and has too many sub-topics to be on a single page. I tried to extract the most common topics into the FAQ so that the basics can be found quickly, which might lead some folks to read the more deep-dive pages.
Some pages stop rather abruptly as I was getting bogged down and figured it would be good to solicit feedback before writing too much more. I'm happy to rework any/all of this to suit what typical users want to know.