rustdoc-json: Improve Item::attrs to not use debug representation.

[aDotInTheVoid]

Currently, Item::attrs uses a debug-printing

Stringified versions of parsed attributes on this item.
Essentially debug printed (e.g. #[inline] becomes something similar to #[attr="Inline(Hint)"]).
Equivalent to the hir pretty-printing of attributes.

https://github.com/rust-lang/rust/blame/85abb276361c424d64743c0965242dd0e7b866d1/src/rustdoc-json-types/lib.rs#L123-L126

This isn't particularly friendly to users, who need to deal with an internal (and undocumented 😱) attr representation.

This was changed in #135726, as part of the a larger attribute rework (#131229)

MCVE:

#[repr(C)]
pub struct Foo(i32);

which produces (abridged):

    "1": {
      "attrs": ["#[attr=\"Repr([ReprC])\")]\n"],
      "inner": {
        "struct": {
          "generics": {"params": [], "where_predicates": []},
          "impls": [2, 4, 6, 8, 10, 12, 15, 19, 23, 26, 31, 36, 39],
          "kind": {"tuple": [null]}
        }
      },
      "name": "Foo",
    }

CC @jdonszelmann @obi1kenobi

[aDotInTheVoid]

So there's basically two options here:

1. Go back to how things used to be, and return attrs as a Vec<String> of how they're represented in the surface syntax.

   This would change

   #[repr(C)]
   #[repc(align(8)]
    pub struct Foo {}

   to

   #[repr(C, align(8))]
    pub struct Foo {}

   but that's fine

2. Expose attributes in a more structured way in rustdoc-json-types. Currently clean::Attributes wraps a list of hir::Attribute (modulo some stuff for doc-comments). This exposes both parsed and unparsed attributes. We could do something similar.

Doing option 2 would require a lot more design work, but is probably worthwhile (even if we do option 1 initially). Some questions:

• How do we want to deal with #[cfg], which for rustc are gone by HIR, but rustdoc has a kinda seperate treatment?
• Do we care to persist stuff like #[inline] at all?
• How do we treat stuff that exists in the interface between std and rustc that isn't exposed to normal crates? Should we include #[unstable] attrs? How about #[rustc_promotable]?
• How do we deal with tool attributes that rustc knows nothing of?

[jdonszelmann]

One advantage of option 1 is that it also improves hir pretty printing. I personally think having them available in a nice and structured way is pretty neat, and hir is not required to look nice. Furthermore, it's also a lot of code (I initially tried). But it in an advantage at least

[jdonszelmann]

Tools attrs are still accessible as tokens. If you try to hir pretty print they even still look like they used to. So that would fit in with option 1 I guess

[jdonszelmann]

@obi1kenobi are there any other attrs except repr you would care about with semver checks? Although I dont think thats the only reason to expose things like this (I'd be in favor of exposing things like inline) it'd be interesting to know if that's literally the only one right now.

[obi1kenobi]

I'm in favor of Option 1 initially, and Option 2 in the long run as a nice-to-have. We already have code that does attribute parsing, so while I'd love to eliminate it eventually, that isn't really a priority. Option 1 sounds like it can be done with less work, and speed is good :)

@obi1kenobi are there any other attrs except repr you would care about with semver checks? Although I dont think thats the only reason to expose things like this (I'd be in favor of exposing things like inline) it'd be interesting to know if that's literally the only one right now.

No, unfortunately we need many more. At minimum (probably not an exhaustive list): #[non_exhaustive], #[automatically_derived], #[must_use], #[doc(hidden)], #[deprecated], #[no_mangle] and #[export_name] (plus their new unsafe variants like #[unsafe(no_mangle)]) , #[repr(packed)], #[repr(C)], #[repr(transparent)], enum integer reprs like #[repr(i8)].

• How do we want to deal with #[cfg], which for rustc are gone by HIR, but rustdoc has a kinda seperate treatment?

If we can keep #[cfg] somehow, that would enable new lints like "this API is no longer available on this feature/OS/architecture etc." Currently, users have to build rustdoc JSON for each feature combo they want to check, on each platform they want to check, which is fine but could obviously be improved.

• Do we care to persist stuff like #[inline] at all?

If we can keep it, it'd be great! There are some cases where in the future a warning may be desirable as a lint if e.g. #[inline] was removed from some public API where it might affect optimization. I don't recall the #[inline] implementation details off the top of my head enough to flesh this out further.

Not a priority today though.

• How do we treat stuff that exists in the interface between std and rustc that isn't exposed to normal crates? Should we include #[unstable] attrs? How about #[rustc_promotable]?

I have no opinion on this, since none of cargo-semver-checks relies on it today. If we can present the attributes as-is (i.e. as pretty-printed Rust code, the status quo ante of other attributes) without a ton of work, I guess that'd be nice.

• How do we deal with tool attributes that rustc knows nothing of?

It's possible cargo-semver-checks might get its own tool attribute in the near future, so preserving tool attributes in as-is form (i.e. as pretty-printed Rust code, the status quo ante of other attributes) would be nice if possible. Users have expressed a desire to opt out of specific lints for specific types or modules, and tool attributes in rustdoc JSON would make that much easier to implement.

[obi1kenobi]

Thank you both for looking into this issue and helping address it! ❤

[jdonszelmann]

for the record, although I played devil's advocate a little and posted some arguments for option, I think option 2 gives a lot more flexibility and although I agree you already have parsing for option 1, with option 2 you potentially won't need any parsing at all nor any other tool which I think is very valuable

[obi1kenobi]

I agree option 2 would be nice! But my top priority is having whichever option we can ship faster :)