New define_keyword option

[badlop]

update 17 february: keyword feature greately improved, and now macros can be used inside strings.

Macros are replaced in the yconf library, this is very powerful at the YAML level... but unfortunately macros cannot be defined inside host_config.

This PR adds a new option, define_keyword, complementary to define_macro. As define_keyword internal code is implemented in ejabberd, define_keyword can be set in host_config.

In ejabberd there were already a few keywords hard-coded in specific options, for example some module hosts can use the keyword @host@. This PR allows all the options to use this and other keywords, no need to add code for each option.

Tests are also added for all those features.

---

Release Notes

Keywords predefined for all options

Some options in ejabberd supported the possibility to use hard-coded keywords. For example, many modules like mod_vcard could used HOST in their option hosts. Also, the captcha_cmd toplevel option could use VERSION and SEMVER keywords. This was implemented for each individual option.

Now those keywords are predefined and can be used by any option, and this is implemented in ejabberd core, no need to implement the keyword substitution in each option.

The predefined keywords are: HOST, HOME, VERSION and SEMVER.

For example, this configuration is possible without requiring any implementation in the option source code:

ext_api_url: "http://example.org/@VERSION@/api"

New define_keyword option

Now you can define your own keywords, similarly to how macros are defined:

define_keyword:
  SCRIPT: "captcha.sh"

captcha_cmd: "tools/@SCRIPT@"

Macros can be used inside string

Macros can be used inside string options, similarly to how keywords can be used.

This is now possible:

define_macro:
  SCRIPT: "captcha.sh"

captcha_cmd: "tools/@SCRIPT@"

Documentation

This is the documentation that could be included in the Docs site. Notice that the markdown and links in this github issue may not be perfect, as it's designed for the docs site.

Macros and Keywords

In the ejabberd configuration file, you can define a macro or keyword for a value (atom, integer, string...) and later use it when configuring an ejabberd option.

Macros is a feature implemented internally by the yconf library and are replaced early and transparently to ejabberd. However, macros cannot be defined inside host_config.

Keywords is a feature similar to macros, implemented by ejabberd itself, and are replaced after macro replacement. Keywords can be defined inside host_config for module options, but not for toplevel options. Keywords cannot be used in those toplevel options: hosts, loglevel, version.

First define_macro and define_keyword and then use them like this:

define_macro:
  NAME1: value1

define_keyword:
  NAME2: "value2"

some_option1: NAME1
other_option1: "I am @NAME1@"

some_option2: NAME2
other_option2: "I am @NAME2@"

where:

• NAME: should be specified in capital letters for convenience. Duplicated macro/keyword names are not allowed. If a macro is defined with the same name than a keyword, the macro is used.

• value: for all options, the value can be any valid YAML element. It is also possible to use as value the name of another macro.

• use a macro/keyword when configuring the option: simply set NAME instead of option value. Macros are processed after additional configuration files have been included, so it is possible to use macros that are defined in configuration files included before the usage.

• use inside a string: surround its name with @ characters

Let's see examples of all this in detail:

Atom

define_macro:
  ANON: both

define_keyword:
  TLS: optional

anonymous_protocol: ANON
s2s_use_starttls: TLS

The resulting configuration is:

anonymous_protocol: both
s2s_use_starttls: optional

Integer

define_macro:
  LOG_LEVEL_NUMBER: 5
  NUMBER_PORT_C2S: 5222

define_keyword:
  NUMBER_PORT_HTTP: 5280

loglevel: LOG_LEVEL_NUMBER

listen:
  -
    port: NUMBER_PORT_C2S
    module: ejabberd_c2s
  -
    port: NUMBER_PORT_HTTP
    module: ejabberd_http

The resulting configuration is:

loglevel: 5

listen:
  -
    port: 5222
    module: ejabberd_c2s
  -
    port: 5280
    module: ejabberd_http

Map

Option values can be any arbitrary YAML value:

define_macro:
  USERBOB:
    user:
      - bob@localhost

define_keyword:
  USERJAN:
    user:
      - jan@localhost

acl:
  admin: USERBOB
  moderator: USERJAN

The resulting configuration is:

acl:
  admin:
    user:
      - bob@localhost
  moderator:
    user:
      - jan@localhost

String

define_macro:
  NAME: "MUC Service"
  PERSISTENT: true

define_keyword:
  TITLE: "Example Room"

modules:
  mod_muc:
    name: NAME
    default_room_options:
      persistent: true
      title: TITLE

The resulting configuration is:

modules:
  mod_muc:
    name: "MUC Service"
    default_room_options:
      persistent: PERSISTENT
      title: "Example Room"

Inside string

A macro or keyword can be used inside an option string:

define_keyword:
  CMD: "captcha"

captcha_cmd: "tools/@CMD@.sh"

is equivalent to:

define_keyword:
  CMD: "tools/captcha.sh"

captcha_cmd: "@CMD@"

is equivalent to:

define_keyword:
  CMD: "tools/captcha.sh"

captcha_cmd: CMD

The resulting configuration in all the cases is:

captcha_cmd: tools/captcha.sh

Macro over keyword

If a macro and a keyword are defined with the same name, the macro definition takes precedence and the keyword definition is ignored:

define_macro:
  LANGUAGE: "bg"

define_keyword:
  LANGUAGE: "pt"

language: LANGUAGE

The resulting configuration is:

language: "bg"

Keyword inside macro

A macro definition can use a keyword:

define_macro:
  MACRO: "tools/@KEYWORD@"

define_keyword:
  KEYWORD: "captcha.sh"

captcha_cmd: MACRO

The resulting configuration is:

captcha_cmd: "tools/captcha.sh"

Predefined keywords

Several keywords are predefined automatically by ejabberd, so you can use them without need to define them explicitly:

• HOST: the virtual host name, for example "example.org". That keyword is only predefined for module options, not toplevel options.
• HOME: the home directory of the user running ejabberd, for example "/home/ejabberd"
• VERSION: ejabberd version number in XX.YY format, for example "24.05"
• SEMVER: ejabberd version number in semver format when compiled with Elixir’s mix ("24.5"), otherwise it's in XX.YY format ("24.05")

It is possible to overwrite predefined keywords, global or for a vhost like in this example:

host_config:
  localhost:
    define_keyword:
      VERSION: "1.2.3"

ext_api_url: "http://localhost/@VERSION@"

The resulting behaviour is equivalent to a configuration like:

host_config:
  localhost:
    ext_api_url: "http://localhost/1.2.3"

ext_api_url: "http://localhost/25.xx"

Macro and host_config

Macros can be used inside host_config:

define_macro:
  MYSQL_PORT: 1234
  PGSQL_PORT: 4567

host_config:
  mysql.localhost:
    sql_port: MYSQL_PORT
  pgsql.localhost:
    sql_port: MYSQL_PORT

The resulting configuration is:

host_config:
  mysql.localhost:
    sql_port: 1234
  pgsql.localhost:
    sql_port: 4567

!!! warning "Don't use macro defined in host_config"

Macros can not be **defined** inside host_config.
Use the previous method instead.
That problematic macro is not replaced:

``` yaml
host_config:
  mysql.localhost:
    define_macro:
      SQL_PORT: 1234
  pgsql.localhost:
    define_macro:
      SQL_PORT: 4567

sql_port: SQL_PORT

# [critical] Failed to start ejabberd application:
#   Invalid value of option sql_port:
#   Expected integer, got string instead
```

Keyword and host_config

Keywords can be used and defined inside host_config:

hosts:
 - localhost
 - example.org

define_keyword:
  HOSTNAME: "Generic Name"

host_config:
  example.org:
    define_keyword:
      HOSTNAME: "Example Host"

modules:
  mod_vcard:
    name: "vJUD of @HOSTNAME@"

The resulting configuration is:

host_config:
  localhost:
    modules:
      mod_vcard:
        name: "vJUD of Generic Name"
  example.org:
    modules:
      mod_vcard:
        name: "vJUD of Example Host"

!!! warning "Don't use in toplevel a keyword defined in host_config"

Keywords can be defined inside `host_config`,
but only if they are being used in module options,
not in toplevel options.
That problematic keyword is not replaced:

``` yaml
host_config:
  mysql.localhost:
    define_keyword:
      SQL_PORT: 1234
  pgsql.localhost:
    define_keyword:
      SQL_PORT: 4567

sql_port: SQL_PORT

# [critical] Failed to start ejabberd application:
#   Invalid value of option sql_port:
#   Expected integer, got string instead
```

[coveralls]

coverage: 33.499% (+0.1%) from 33.384%
when pulling c821339 on badlop:define_keyword
into ceee3d3 on processone:master.