Add TOML syntax

[FichteFoll]

TOML aims to be a minimal configuration file format that's easy to read due to obvious semantics.

TOML is comparable to JSON and YAML, which we both already have in the default packages, and it has gained significant traction from being used for packaging metadata in both the Python and Rust ecosystems, which we also already support in the default packages. TOML is, however, also used in other fashion such as generic configuration files by individual applications and recently even started showing up inline for Rust (unstable, see also #4164) and Python ("living standard").

A package that provides TOML syntax support exists on Package Control since 2013 and has been installed a total of 76k times with around 25 installations daily. I personally have been using it for years and cannot name any shortcomings right now besides the scope naming being a bit dated compared to what we agreed upon in the recent years (using entity.name.tag instead of meta.mapping.key), but I also have not investigated closer right now.

In order to properly support inline metadata highlighting inside the Rust and Python syntaxes, TOML must also be available in the default packages because otherwise it is not guaranteed to be available and we do not want to depend on third-party package. We may also want to built it in a way that allows the Python syntax to trivially extend it to strip away the comment punctuation that is necessary inside the file to not interfere with actual source code.

According to @deathaxe, Package Control 4 handles third-party packages with the same name as a default package "gracefully", but we are also in a position to adjust the PC code to provide commands that specifically target the integration of a new default package in the future, if necessary.

---

The previous attempt to get it into the default packages (#453) was shot down in 2016, but I believe that the situation has changed since then.

[deathaxe]

It is also a supported Markdown frontmatter syntax, already supported by MarkdownEditing.

Hugo requires it for instance.

[BenjaminSchaaf]

We are broadly in favor of adding TOML.

[deathaxe]

With regards to https://toml.io/en/v1.0.0#array-of-tables I wonder how strictly we want to follow a "meta.mapping.key" interpretation or how it corresponds/maps to JSON.

Current TOML, INI and GitConfig scope sections [section] as entity.name.section.

TOML actually calls it "table".

As [key . subkey . "sub.subkey"] maps to {"key": { "subkey": { "sub.subkey": { ... } } } }, each table name part would individually be scoped meta.mapping.key string as all key = value pairs would be, if we'd follow the JSON interpretation.

I guess general highlighting would probably be pretty unexpected.

TOML

JSON

[michaelblyons]

entity.name.section.table instead of the first meta.mapping. key string.unquoted ? I agree it's not expected if we match exactly to JSON.

[FichteFoll]

Semantically, I think an entity.name.section scope is a good choice for these tables because they affect all the keys given below them and that is also how they are generally used or treated like. You could compare them to INI in usage, despite having a different result technically since every table can also be written as a mapping. You may even argue that the whole path inside the table header should be a single entity.name.section (with punctuation.accessor on top).

[deathaxe]

... table header should be a single entity.name.section (with punctuation.accessor on top).

Makes sense, but also renders meta.mapping string scope for keys somewhat questionalble as those are of exactly same syntax.

see "Dotted keys":

name = "Orange"
physical.color = "orange"
physical.shape = "round"
site."google.com" = true
1.2."foo" = "value

If section headers are scoped as proposed, keys should follow such schema, too, which leads us back to entity.name.key or something along those lines.

Other than YAML, all keys are "strings" in JSON and TOML.

[FichteFoll]

Makes sense, but also renders meta.mapping string scope for keys somewhat questionalble as those are of exactly same syntax.

No, I don't see how this relates to my argument. My point is that the […] syntax denotes a section (of keys), unrelated to the presence of dot accessors.

For the dot-joined individual keys (which I didn't even know were legal until now and don't remember seeing anywhere 🙃 ), they would still be string keys of a mapping, same as JSON. The question I do not have an answer for right now is whether the full path should be scoped as string or just the last segment.

I saw that you opened a PR but I'm short on time these days so I won't be looking at that in detail. I might take a look at open discussions some time during the next week, though, like this one.