From b71153608b304029f7caf6d7b9f06111813e2f47 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 25 Oct 2022 21:18:44 +0100 Subject: [PATCH 1/6] Clean up AS registration definitions Move the object definition out to yaml files --- content/application-service-api.md | 78 +++++-------------- .../definitions/namespace_list.yaml | 28 +++++++ .../definitions/registration.yaml | 74 ++++++++++++++++++ 3 files changed, 123 insertions(+), 57 deletions(-) create mode 100644 data/api/application-service/definitions/namespace_list.yaml create mode 100644 data/api/application-service/definitions/registration.yaml diff --git a/content/application-service-api.md b/content/application-service-api.md index 255f5cbc3..18ec7867f 100644 --- a/content/application-service-api.md +++ b/content/application-service-api.md @@ -37,14 +37,8 @@ registration for suspicious regex strings. {{% /boxes/note %}} Application services register "namespaces" of user IDs, room aliases and -room IDs. These namespaces are represented as regular expressions. An -application service is said to be "interested" in a given event if one -of the IDs in the event match the regular expression provided by the -application service, such as the room having an alias or ID in the -relevant namespaces. Similarly, the application service is said to be -interested in a given event if one of the application service's -namespaced users is the target of the event, or is a joined member of -the room where the event occurred. +room IDs. An application service is said to be "interested" in a given event +if it matches any of the namespaces. An application service can also state whether they should be the only ones who can manage a specified namespace. This is referred to as an @@ -53,28 +47,12 @@ application services from creating/deleting entities in that namespace. Typically, exclusive namespaces are used when the rooms represent real rooms on another service (e.g. IRC). Non-exclusive namespaces are used when the application service is merely augmenting the room itself (e.g. -providing logging or searching facilities). Namespaces are represented -by POSIX extended regular expressions and look like: +providing logging or searching facilities). - users: - - exclusive: true - regex: "@_irc_bridge_.*" - -Application services may define the following namespaces (with none -being explicitly required): - -| Name | Description | -|----------|------------------------------------------------------------| -| users | Events which are sent from certain users. | -| aliases | Events which are sent in rooms with certain room aliases. | -| rooms | Events which are sent in rooms with certain room IDs. | - -Each individual namespace MUST declare the following fields: +The registration is represented by a series of key-value pairs, which +is normally encoded as an object in a YAML file. It has the following structure: -| Name | Description | -|------------|------------------------------------------------------------------------------------------------------------------------------------| -| exclusive | **Required** A true or false value stating whether this application service has exclusive access to events within this namespace. | -| regex | **Required** A regular expression defining which values this namespace includes. | +{{% definition path="api/application-service/definitions/registration" %}} Exclusive user and alias namespaces should begin with an underscore after the sigil to avoid collisions with other users on the homeserver. @@ -83,38 +61,24 @@ they represent in the reserved namespace. For example, `@_irc_.*` would be a good namespace to register for an application service which deals with IRC. -The registration is represented by a series of key-value pairs, which -this specification will present as YAML. See below for the possible -options along with their explanation: - - -| Name | Description | -|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------| -| id | **Required** A unique, user-defined ID of the application service which will never change. | -| url | **Required** The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required. | -| as_token | **Required** A unique token for application services to use to authenticate requests to Homeservers. | -| hs_token | **Required** A unique token for Homeservers to use to authenticate requests to application services. | -| sender_localpart | **Required** The localpart of the user associated with the application service. | -| namespaces | **Required** A list of `users`, `aliases` and `rooms` namespaces that the application service controls. | -| rate_limited | Whether requests from masqueraded users are rate-limited. The sender is excluded. | -| protocols | The external protocols which the application service provides (e.g. IRC). | - An example registration file for an IRC-bridging application service is below: - id: "IRC Bridge" - url: "http://127.0.0.1:1234" - as_token: "30c05ae90a248a4188e620216fa72e349803310ec83e2a77b34fe90be6081f46" - hs_token: "312df522183efd404ec1cd22d2ffa4bbc76a8c1ccf541dd692eef281356bb74e" - sender_localpart: "_irc_bot" # Will result in @_irc_bot:example.org - namespaces: - users: - - exclusive: true - regex: "@_irc_bridge_.*" - aliases: - - exclusive: false - regex: "#_irc_bridge_.*" - rooms: [] +```yaml +id: "IRC Bridge" +url: "http://127.0.0.1:1234" +as_token: "30c05ae90a248a4188e620216fa72e349803310ec83e2a77b34fe90be6081f46" +hs_token: "312df522183efd404ec1cd22d2ffa4bbc76a8c1ccf541dd692eef281356bb74e" +sender_localpart: "_irc_bot" # Will result in @_irc_bot:example.org +namespaces: + users: + - exclusive: true + regex: "@_irc_bridge_.*" + aliases: + - exclusive: false + regex: "#_irc_bridge_.*" + rooms: [] +``` {{% boxes/warning %}} If the homeserver in question has multiple application services, each diff --git a/data/api/application-service/definitions/namespace_list.yaml b/data/api/application-service/definitions/namespace_list.yaml new file mode 100644 index 000000000..4f45fc3f4 --- /dev/null +++ b/data/api/application-service/definitions/namespace_list.yaml @@ -0,0 +1,28 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +type: array +items: + type: object + title: Namespace + properties: + regex: + type: string + description: A POSIX regular expression defining which values this namespace includes. + exclusive: + type: boolean + description: A true or false value stating whether this application service has exclusive access to events within this namespace. + required: + - regex + - exclusive diff --git a/data/api/application-service/definitions/registration.yaml b/data/api/application-service/definitions/registration.yaml new file mode 100644 index 000000000..50eefdfba --- /dev/null +++ b/data/api/application-service/definitions/registration.yaml @@ -0,0 +1,74 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +type: object +title: Registration +properties: + id: + type: string + description: A unique, user-defined ID of the application service which will never change. + url: + type: string + description: The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required. + as_token: + type: string + description: A secret token that the application service will use to authenticate requests to the homeserver. + hs_token: + type: string + description: A secret token that the homeserver will use authenticate requests to the application service. + sender_localpart: + type: string + description: The localpart of the user associated with the application service. + namespaces: + type: object + title: Namespaces + description: The namespaces that the application service is interested in. + properties: + users: + allOf: + - $ref: namespace_list.yaml + - description: |- + A list of namespaces defining the user IDs that the application + service is interested in. Events will be sent to the AS if a + user matching one of the namespaces is the target of the event, + or is a joined member of the room where the event occurred. + rooms: + allOf: + - $ref: namespace_list.yaml + - description: |- + A list of namespaces defining the room IDs that the application + service is interested in. Events will be sent to the AS if they + take place in a room with an ID matching one of the namespaces. + aliases: + allOf: + - $ref: namespace_list.yaml + - description: |- + A list of namespaces defining the room aliases that the application + service is interested in. Events will be sent to the AS if they + take place in a room with an alias matching one of the namespaces. + rate_limited: + type: boolean + description: Whether requests from masqueraded users are rate-limited. The sender is excluded. + protocols: + type: array + description: The external protocols which the application service provides (e.g. IRC). + items: + type: string +required: + - id + - url + - as_token + - hs_token + - sender_localpart + - namespaces From d0442ec97b89a49ecc808a6cc48a018775e6cc98 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 25 Oct 2022 21:48:54 +0100 Subject: [PATCH 2/6] Clarify that appservices can only watch local users Spec for MSC3905 --- content/application-service-api.md | 10 ++++++++++ .../application-service/definitions/registration.yaml | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/content/application-service-api.md b/content/application-service-api.md index 18ec7867f..d9dbac5cd 100644 --- a/content/application-service-api.md +++ b/content/application-service-api.md @@ -80,6 +80,16 @@ namespaces: rooms: [] ``` +{{% boxes/note %}} +Application services can only register interest in *local* users (i.e., users +whose IDs end with the `server_name` of the local homeserver). Events affecting +users on other homeservers are not sent to an application service, even if the +user happens to match the one of the `users` namespaces (unless, of course, the +event affects a room that the application service is interested in for another +room - for example, because there is another user in the room that the +application service is interested in). +{{% /boxes/note %}} + {{% boxes/warning %}} If the homeserver in question has multiple application services, each `as_token` and `id` MUST be unique per application service as these are diff --git a/data/api/application-service/definitions/registration.yaml b/data/api/application-service/definitions/registration.yaml index 50eefdfba..5fab75d02 100644 --- a/data/api/application-service/definitions/registration.yaml +++ b/data/api/application-service/definitions/registration.yaml @@ -41,7 +41,7 @@ properties: - description: |- A list of namespaces defining the user IDs that the application service is interested in. Events will be sent to the AS if a - user matching one of the namespaces is the target of the event, + local user matching one of the namespaces is the target of the event, or is a joined member of the room where the event occurred. rooms: allOf: From 4ba23ed530938fb931d06b01022871a075f6fbb3 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 25 Oct 2022 21:50:23 +0100 Subject: [PATCH 3/6] changelog --- changelogs/application_service/newsfragments/1305.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/application_service/newsfragments/1305.clarification diff --git a/changelogs/application_service/newsfragments/1305.clarification b/changelogs/application_service/newsfragments/1305.clarification new file mode 100644 index 000000000..c9a3e755d --- /dev/null +++ b/changelogs/application_service/newsfragments/1305.clarification @@ -0,0 +1 @@ +Clarify that application services can only register an interest in local users. \ No newline at end of file From a344dfa20dd6c06035ed4df311d279cd881d612e Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Tue, 25 Oct 2022 22:18:41 +0100 Subject: [PATCH 4/6] Update changelogs/application_service/newsfragments/1305.clarification Co-authored-by: Travis Ralston --- changelogs/application_service/newsfragments/1305.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/application_service/newsfragments/1305.clarification b/changelogs/application_service/newsfragments/1305.clarification index c9a3e755d..9c45483f4 100644 --- a/changelogs/application_service/newsfragments/1305.clarification +++ b/changelogs/application_service/newsfragments/1305.clarification @@ -1 +1 @@ -Clarify that application services can only register an interest in local users. \ No newline at end of file +Clarify that application services can only register an interest in local users, as per [MSC3905](https://github.com/matrix-org/matrix-spec-proposals/issues/3905). \ No newline at end of file From a959df71c129fe5a9fb6fee9de10659512fa872d Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Tue, 1 Nov 2022 14:25:53 +0000 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Eric Eastwood --- content/application-service-api.md | 14 ++++++++------ .../definitions/registration.yaml | 8 ++++---- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/content/application-service-api.md b/content/application-service-api.md index d9dbac5cd..64cf7597a 100644 --- a/content/application-service-api.md +++ b/content/application-service-api.md @@ -81,13 +81,15 @@ namespaces: ``` {{% boxes/note %}} -Application services can only register interest in *local* users (i.e., users -whose IDs end with the `server_name` of the local homeserver). Events affecting -users on other homeservers are not sent to an application service, even if the -user happens to match the one of the `users` namespaces (unless, of course, the -event affects a room that the application service is interested in for another -room - for example, because there is another user in the room that the +For the `users` namespace, application services can only register interest in +*local* users (i.e., users whose IDs end with the `server_name` of the local +homeserver). Events affecting users on other homeservers are not sent to an application +service, even if the user happens to match the one of the `users` namespaces (unless, +of course, the event affects a room that the application service is interested in +for another room - for example, because there is another user in the room that the application service is interested in). + +For the `rooms` and `aliases` namespaces, all events in a matching room will be sent to the application service. {{% /boxes/note %}} {{% boxes/warning %}} diff --git a/data/api/application-service/definitions/registration.yaml b/data/api/application-service/definitions/registration.yaml index 5fab75d02..d9dfe7484 100644 --- a/data/api/application-service/definitions/registration.yaml +++ b/data/api/application-service/definitions/registration.yaml @@ -48,15 +48,15 @@ properties: - $ref: namespace_list.yaml - description: |- A list of namespaces defining the room IDs that the application - service is interested in. Events will be sent to the AS if they - take place in a room with an ID matching one of the namespaces. + service is interested in. All events sent in a room with an ID + which matches one of the namespaces will be sent to the AS. aliases: allOf: - $ref: namespace_list.yaml - description: |- A list of namespaces defining the room aliases that the application - service is interested in. Events will be sent to the AS if they - take place in a room with an alias matching one of the namespaces. + service is interested in. All events sent in a room with an alias + which matches one of the namespaces will be sent to the AS. rate_limited: type: boolean description: Whether requests from masqueraded users are rate-limited. The sender is excluded. From 321d55cc1646f488e8d02cb7912407b5eebb000f Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Tue, 1 Nov 2022 14:30:17 +0000 Subject: [PATCH 6/6] wrap text --- content/application-service-api.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/application-service-api.md b/content/application-service-api.md index 64cf7597a..2023af00b 100644 --- a/content/application-service-api.md +++ b/content/application-service-api.md @@ -89,7 +89,8 @@ of course, the event affects a room that the application service is interested i for another room - for example, because there is another user in the room that the application service is interested in). -For the `rooms` and `aliases` namespaces, all events in a matching room will be sent to the application service. +For the `rooms` and `aliases` namespaces, all events in a matching room will be +sent to the application service. {{% /boxes/note %}} {{% boxes/warning %}}