JSON vs Jason incompatibility

[maxsalven]

Elixir and Erlang/OTP versions

Erlang/OTP 27 [erts-15.2.3] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Elixir 1.18.3 (compiled with Erlang/OTP 27)

Operating system

Mac OS

Current behavior

input = ~c"{}"

Jason.decode!(input)
%{}

JSON.decode!(input)
** (FunctionClauseError) no function clause matching in JSON.decode!/1

    The following arguments were given to JSON.decode!/1:

        # 1
        ~c"{}"

    Attempted function clauses (showing 1 out of 1):

        def decode!(binary) when is_binary(binary)

    (elixir 1.18.3) lib/json.ex:327: JSON.decode!/1
    iex:4: (file)

Expected behavior

I'm having issues with third party modules causing crashes when trying to switch to JSON.

Should JSON decode a charlist for Jason compatibility?

[josevalim]

We don't aim to be 100% Jason compatible but we should discuss if we want to add this feature. Thoughts @michalmuskala? In this case, I think folks could explicitly convert to binary, since I believe this is what Jason does anyway.

[maxsalven]

I think the risk here is people like me just plug ‘JSON’ into every lib’s ‘json_library:’ config and expect it to (ideally!) work. Perhaps an educational piece is missing for either library authors or library consumers that things aren’t necessarily interchangeable.

[josevalim]

Right, what I am saying is that it is an unreasonable expectation and we should not do changes based solely on that, because even if we made charlists work, then something else will fail. We should make the decision that is best aligned with JSON itself, considering Jason compatibility as one of the factors.

[sabiwara]

Perhaps an educational piece is missing for either library authors or library consumers that things aren’t necessarily interchangeable.

I like the idea of adding an admonition block to state the non-goals for JSON in terms of compatibility.

[josevalim]

Generally I am not in favor of saying things that we don't do in the documentation. Because there are one thousand things we don't do. We aren't fully compatible with Jason, we are not compatible with the json package either, etc. But perhaps that would be a welcome contribution to Jason. Let's see @michalmuskala thoughts.

[maxsalven]

A separate but real world data point:

Absinthe.Plug supports a json_codec option, however they pass a keyword list (opts) as the second argument to encode!/2 which is Jason specific:
https://github.com/absinthe-graphql/absinthe_plug/blob/5de21da279938d1a10642f94d5e9c9fbdc846149/lib/absinthe/plug.ex#L607

Trying to use JSON here will raise as it expects the second argument to be a function (an encoder).

I think this supports the idea that cross compatibility is an unreasonable expectation, and we'll need library authors to explicitly upstream support for JSON in certain cases.

[whatyouhide]

I think maybe we could consider an explicit error (not a FunctionClauseError) for charlists, that would at least point people in the right direction. Agree we don't need to aim to be 100% compatible.

[michalmuskala]

Couple points to consider here.
Jason accepts actually iodata, not charlists. So to some extent, using it with ~c strings is actually wrong. If you try with some that contain unicode, e.g. Jason.decode!(~c'"żółć"') it will crash.
I think Elixir's behaviour of just accepting binaries is simpler & less error prone, I'd consider this "flexibility" on Jason's part actually an API deficiency ideally we won't replicate.
As a second point, if you actually do have proper iodata, you should probably use the "streaming" API of the json module with :json.decode_start/3 since that allows you to avoid allocating the entire iodata as one binary.