[libc++] Vendor communication: upcoming ABI breaks in std::expected and parts of <ranges> #70820
Comments
ldionne
added
the
libc++
Currently std::expected can have some padding bytes in its tail due to
[[no_unique_address]]. Those padding bytes can be used by other objects.
For example, in the current implementation:
sizeof(std::expected<std::optional<int>, bool>) ==
sizeof(std::expected<std::expected<std::optional<int>, bool>, bool>)
As a result, the data layout of an
std::expected<std::expected<std::optional<int>, bool>, bool>
can look like this:
+-- optional "has value" flag
| +--padding
/---int---\ | |
00 00 00 00 01 00 00 00
| |
| +- "outer" expected "has value" flag
|
+- expected "has value" flag
This is problematic because `emplace()`ing the "inner" expected can not
only overwrite the "inner" expected "has value" flag (issue #68552) but
also the tail padding where other objects might live.
This patch fixes the problem by ensuring that std::expected has no tail
padding, which is achieved by conditional usage of [[no_unique_address]]
based on the tail padding that this would create.
This is an ABI breaking change because the following property changes:
sizeof(std::expected<std::optional<int>, bool>) <
sizeof(std::expected<std::expected<std::optional<int>, bool>, bool>)
Before the change, this relation didn't hold. After the change, the relation
does hold, which means that the size of std::expected in these cases increases
after this patch. The data layout will change in the following cases where
tail padding can be reused by other objects:
class foo : std::expected<std::optional<int>, bool> {
bool b;
};
or using [[no_unique_address]]:
struct foo {
[[no_unique_address]] std::expected<std::optional<int>, bool> e;
bool b;
};
The vendor communication is handled in #70820.
Fixes: #70494
Co-authored-by: philnik777 <nikolasklauser@berlin.de>
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
|
All the changes related to this event have now been merged:
Effect on
|
Hi @llvm/libcxx-vendors,
This issue is not a traditional issue. It is a communication to vendors about upcoming ABI breaks in
std::expectedand some parts of<ranges>. It was recently brought to our attention in #70494 (see the associated PR for more discussion) that there is a problem with some of the library's usage of[[no_unique_address]], which can lead to objects overlapping with padding bytes of other objects and their contents being overwritten unexpectedly.We are aware of this issue in
std::expectedand instd::__movable_box, which is used to implement several views in<ranges>. Fixing this will require breaking the ABI for the types involved. We introducedstd::expectedand__movable_boxin LLVM 16 so this has been shipping for two releases. We are aiming to fix these issues in LLVM 18.Typically, we do not expect code to expose a lot of
<ranges>views at ABI boundaries, so we believe the__movable_boxABI break to have a limited fallout. However,std::expectedis a vocabulary type and is intended to appear at all kinds of API (and ABI) boundaries, in particular function result types. That ABI break is more likely to cause actual issues in the wild.At this moment, we want to make vendors aware of these upcoming changes so they can plan ahead and perhaps limit the blast radius in case they haven't shipped LLVM 16/17 yet. One possibility would be to disable
std::expectedin their downstream release or keep it experimental behind-fexperimental-library. We will update this thread as we gain more clarity on the exact nature of the upcoming ABI break (e.g. which types are impacted and under what circumstances).FAQ
Q: Are you going to allow vendors to opt into the old/new ABI for these types?
A: No. The current ABI has correctness issues which are serious enough that nobody should opt into the old ABI. This isn't a performance improvement or a slight conformance fix.
Q: Are you going to take this opportunity to enable other ABI breaks, effectively moving to "ABI v2"?
A: No. The intent of these changes is to keep the ABI break as narrow as possible with the hopes that the actual fallout will be limited.
<ranges>is seldom used at ABI boundaries.std::expectedshipped only recently, requires C++23 and we expect that few users will have had the time to depend on it in their ABI.Q: Is it possible to use some techniques to try and diagnose when folks have an ABI mismatch after shipping that ABI break?
A: Yes, to some extent. We will use ABI tags to influence the mangling of functions that contain a type that undergoes an ABI break. This means that trying to link two units using/providing a function whose signature contains e.g.
std::expectedwill fail at link time since the mangled name of the function will be different between LLVM 17 and LLVM 18. However, this technique has limitations. For example, if astd::expectedis contained in a user-defined class, the user-defined class will not get an ABI tag despite its layout potentially changing between LLVM 17 and LLVM 18.The text was updated successfully, but these errors were encountered: