Assertion in clang::Sema::MarkExpressionAsImmediateEscalating when there are diagnostic errors

[danakj]

I am working on a clang tool which runs clang::tooling::ClangTool on a bunch of files.

I am using LLVM 30e688e6d061bf0f2a6fcbe39d2d23560a363436 (HEAD as of today).

When one file has errors in it, which are reported through the diagnostics, and I continue on to the next file, I end up getting an assertion failure on a consteval UDL. When there are no diagnostic errors reported (which are unrelated to C++ parsing anyway, they are something from my tool), then the assert does not occur.

So I have stopped this bug by not continuing to loop through each TopLevelDecl once I get to the end of one and see errors. But in case you would like to know this assertion is being triggered, here is a report.

I don't have a minimal repro, it involves running Subdoc and inserting a bunch of comments that throw errors from the tool.

This is the token on which clang asserts in MarkExpressionAsImmediateEscalating:
https://github.com/chromium/subspace/blob/6243bc49617a92220d2bfafca364ac6b05ef72b9/sus/collections/vec.h#L966

Here's the stack trace:

% cmake --build out && tools/run_subdoc.sh 
[6/6] Linking CXX executable subdoc/subdoc_unittests
[1/1] /home/danakj/s/subspace/sus/num/i8_unittest.cc
In file included from /home/danakj/s/subspace/sus/num/i8_unittest.cc:18:
In file included from /home/danakj/s/subspace/sus/collections/array.h:28:
In file included from /home/danakj/s/subspace/sus/collections/iterators/array_iter.h:23:
In file included from /home/danakj/s/subspace/sus/iter/iterator_defn.h:25:
In file included from /home/danakj/s/subspace/sus/iter/__private/iter_compare.h:19:
In file included from /home/danakj/s/subspace/sus/option/option.h:2047:
In file included from /home/danakj/s/subspace/sus/iter/size_hint.h:19:
In file included from /home/danakj/s/subspace/sus/num/unsigned_integer.h:57:
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:597:1: error: malformed API comment: @doc.self used outside of a class
  597 | /// Satisfies the [`Add<@doc.self>`]($sus::num::Add) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:642:1: error: malformed API comment: @doc.self used outside of a class
  642 | /// Satisfies the [`Sub<@doc.self>`]($sus::num::Sub) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:687:1: error: malformed API comment: @doc.self used outside of a class
  687 | /// Satisfies the [`Mul<@doc.self>`]($sus::num::Mul) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:732:1: error: malformed API comment: @doc.self used outside of a class
  732 | /// Satisfies the [`Div<@doc.self>`]($sus::num::Div) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:775:1: error: malformed API comment: @doc.self used outside of a class
  775 | /// Satisfies the [`Rem<@doc.self>`]($sus::num::Rem) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:819:1: error: malformed API comment: @doc.self used outside of a class
  819 | /// Satisfies the [`BitAnd<@doc.self>`]($sus::num::BitAnd) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:860:1: error: malformed API comment: @doc.self used outside of a class
  860 | /// Satisfies the [`BitOr<@doc.self>`]($sus::num::BitOr) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:901:1: error: malformed API comment: @doc.self used outside of a class
  901 | /// Satisfies the [`BitXor<@doc.self>`]($sus::num::BitXor) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:942:1: error: malformed API comment: @doc.self used outside of a class
  942 | /// Satisfies the [`Shl<@doc.self>`]($sus::num::Shl) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:957:1: error: malformed API comment: @doc.self used outside of a class
  957 | /// Satisfies the [`Shr<@doc.self>`]($sus::num::Shr) concept.
      | ^
In file included from /home/danakj/s/subspace/sus/num/i8_unittest.cc:18:
In file included from /home/danakj/s/subspace/sus/collections/array.h:28:
In file included from /home/danakj/s/subspace/sus/collections/iterators/array_iter.h:23:
In file included from /home/danakj/s/subspace/sus/iter/iterator_defn.h:25:
In file included from /home/danakj/s/subspace/sus/iter/__private/iter_compare.h:19:
In file included from /home/danakj/s/subspace/sus/option/option.h:2047:
In file included from /home/danakj/s/subspace/sus/iter/size_hint.h:19:
In file included from /home/danakj/s/subspace/sus/num/unsigned_integer.h:73:
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:597:1: error: malformed API comment: @doc.self used outside of a class
  597 | /// Satisfies the [`Add<@doc.self>`]($sus::num::Add) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:642:1: error: malformed API comment: @doc.self used outside of a class
  642 | /// Satisfies the [`Sub<@doc.self>`]($sus::num::Sub) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:687:1: error: malformed API comment: @doc.self used outside of a class
  687 | /// Satisfies the [`Mul<@doc.self>`]($sus::num::Mul) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:732:1: error: malformed API comment: @doc.self used outside of a class
  732 | /// Satisfies the [`Div<@doc.self>`]($sus::num::Div) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:775:1: error: malformed API comment: @doc.self used outside of a class
  775 | /// Satisfies the [`Rem<@doc.self>`]($sus::num::Rem) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:819:1: error: malformed API comment: @doc.self used outside of a class
  819 | /// Satisfies the [`BitAnd<@doc.self>`]($sus::num::BitAnd) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:860:1: error: malformed API comment: @doc.self used outside of a class
  860 | /// Satisfies the [`BitOr<@doc.self>`]($sus::num::BitOr) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:901:1: error: malformed API comment: @doc.self used outside of a class
  901 | /// Satisfies the [`BitXor<@doc.self>`]($sus::num::BitXor) concept.
      | ^
/home/danakj/s/subspace/sus/num/__private/unsigned_integer_methods.inc:942:1: error: malformed API comment: @doc.self used outside of a class
  942 | /// Satisfies the [`Shl<@doc.self>`]($sus::num::Shl) concept.
      | ^
fatal error: too many errors emitted, stopping now
subdoc: /home/danakj/s/llvm/clang/lib/Sema/SemaExpr.cpp:18263: void clang::Sema::MarkExpressionAsImmediateEscalating(clang::Expr*): Assertion `!FunctionScopes.empty() && "Expected a function scope"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: out/subdoc/subdoc -p out --out docs --include-file-pattern /sus/ --exclude-file-pattern /third_party/ --css subdoc-test-style.css --copy-file subdoc/gen_tests/subdoc-test-style.css --copy-file web/logo.png --favicon logo.png;image/png --project-md sus/project.md --project-logo logo.png --project-name Subspace --ignore-bad-code-links i8_unittest
1.	/home/danakj/s/subspace/sus/collections/vec.h:966:42: current parser token '1_usize'
2.	/home/danakj/s/subspace/sus/collections/vec.h:68:1: parsing namespace 'sus'
3.	/home/danakj/s/subspace/sus/collections/vec.h:83:1: parsing struct/union/class body 'sus::collections::Vec'
 #0 0x000055df2f0461ab llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/danakj/s/subspace/out/subdoc/subdoc+0x52d61ab)
 #1 0x000055df2f0435eb llvm::sys::RunSignalHandlers() (/home/danakj/s/subspace/out/subdoc/subdoc+0x52d35eb)
 #2 0x000055df2f043715 SignalHandler(int) Signals.cpp:0:0
 #3 0x00007f5dc5e5a540 (/lib/x86_64-linux-gnu/libc.so.6+0x3c540)
 #4 0x00007f5dc5ea812c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007f5dc5e5a4a2 raise ./signal/../sysdeps/posix/raise.c:27:6
 #6 0x00007f5dc5e444b2 abort ./stdlib/abort.c:81:7
 #7 0x00007f5dc5e443d5 _nl_load_domain ./intl/loadmsgcat.c:1177:9
 #8 0x00007f5dc5e533a2 (/lib/x86_64-linux-gnu/libc.so.6+0x353a2)
 #9 0x000055df2d5abbfa clang::Sema::MarkExpressionAsImmediateEscalating(clang::Expr*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x383bbfa)
#10 0x000055df2d5b0817 clang::Sema::CheckForImmediateInvocation(clang::ActionResult<clang::Expr*, true>, clang::FunctionDecl*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3840817)
#11 0x000055df2d9866f0 clang::Sema::BuildLiteralOperatorCall(clang::LookupResult&, clang::DeclarationNameInfo&, llvm::ArrayRef<clang::Expr*>, clang::SourceLocation, clang::TemplateArgumentListInfo*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3c166f0)
#12 0x000055df2d5e7617 clang::Sema::ActOnNumericConstant(clang::Token const&, clang::Scope*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3877617)
#13 0x000055df2cf46ad4 clang::Parser::ParseCastExpression(clang::Parser::CastParseKind, bool, bool&, clang::Parser::TypeCastState, bool, bool*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31d6ad4)
#14 0x000055df2cf497b9 clang::Parser::ParseCastExpression(clang::Parser::CastParseKind, bool, clang::Parser::TypeCastState, bool, bool*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31d97b9)
#15 0x000055df2cf49978 clang::Parser::ParseAssignmentExpression(clang::Parser::TypeCastState) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31d9978)
#16 0x000055df2cf18431 clang::Parser::ParseCXXMemberInitializer(clang::Decl*, bool, clang::SourceLocation&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31a8431)
#17 0x000055df2cf2968c clang::Parser::ParseCXXClassMemberDeclaration(clang::AccessSpecifier, clang::ParsedAttributes&, clang::Parser::ParsedTemplateInfo const&, clang::ParsingDeclRAIIObject*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31b968c)
#18 0x000055df2cf2a572 clang::Parser::ParseCXXClassMemberDeclarationWithPragmas(clang::AccessSpecifier&, clang::ParsedAttributes&, clang::TypeSpecifierType, clang::Decl*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31ba572)
#19 0x000055df2cf2aace clang::Parser::ParseCXXMemberSpecification(clang::SourceLocation, clang::SourceLocation, clang::ParsedAttributes&, unsigned int, clang::Decl*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31baace)
#20 0x000055df2cf2cd2e clang::Parser::ParseClassSpecifier(clang::tok::TokenKind, clang::SourceLocation, clang::DeclSpec&, clang::Parser::ParsedTemplateInfo const&, clang::AccessSpecifier, bool, clang::Parser::DeclSpecContext, clang::ParsedAttributes&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31bcd2e)
#21 0x000055df2cefcfe3 clang::Parser::ParseDeclarationSpecifiers(clang::DeclSpec&, clang::Parser::ParsedTemplateInfo const&, clang::AccessSpecifier, clang::Parser::DeclSpecContext, clang::Parser::LateParsedAttrList*, clang::ImplicitTypenameContext) (/home/danakj/s/subspace/out/subdoc/subdoc+0x318cfe3)
#22 0x000055df2cf71232 clang::Parser::ParseSingleDeclarationAfterTemplate(clang::DeclaratorContext, clang::Parser::ParsedTemplateInfo const&, clang::ParsingDeclRAIIObject&, clang::SourceLocation&, clang::ParsedAttributes&, clang::AccessSpecifier) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3201232)
#23 0x000055df2cf7eaad clang::Parser::ParseTemplateDeclarationOrSpecialization(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::AccessSpecifier) (.part.0) ParseTemplate.cpp:0:0
#24 0x000055df2cf7ed93 clang::Parser::ParseDeclarationStartingWithTemplate(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::AccessSpecifier) (/home/danakj/s/subspace/out/subdoc/subdoc+0x320ed93)
#25 0x000055df2cf0b77e clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::ParsedAttributes&, clang::SourceLocation*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x319b77e)
#26 0x000055df2cec9528 clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3159528)
#27 0x000055df2cf21564 clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31b1564)
#28 0x000055df2cf21451 clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31b1451)
#29 0x000055df2cf2471d clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) (/home/danakj/s/subspace/out/subdoc/subdoc+0x31b471d)
#30 0x000055df2cf0b850 clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::ParsedAttributes&, clang::SourceLocation*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x319b850)
#31 0x000055df2cec9528 clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x3159528)
#32 0x000055df2ceca4e5 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x315a4e5)
#33 0x000055df2ceba6da clang::ParseAST(clang::Sema&, bool, bool) (/home/danakj/s/subspace/out/subdoc/subdoc+0x314a6da)
#34 0x000055df2cc25c89 clang::FrontendAction::Execute() (/home/danakj/s/subspace/out/subdoc/subdoc+0x2eb5c89)
#35 0x000055df2cba2782 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x2e32782)
#36 0x000055df2cb6dfbd clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x2dfdfbd)
#37 0x000055df2cb66fec clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) (/home/danakj/s/subspace/out/subdoc/subdoc+0x2df6fec)
#38 0x000055df2cb6a33d clang::tooling::ToolInvocation::run() (/home/danakj/s/subspace/out/subdoc/subdoc+0x2dfa33d)
#39 0x000055df2cb6c4cc clang::tooling::ClangTool::run(clang::tooling::ToolAction*) (/home/danakj/s/subspace/out/subdoc/subdoc+0x2dfc4cc)
#40 0x000055df2c64215b subdoc::run_files(clang::tooling::CompilationDatabase const&, sus::collections::Vec<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, subdoc::RunOptions const&) (/home/danakj/s/subspace/out/subdoc/subdoc+0x28d215b)
#41 0x000055df2c5f0148 main (/home/danakj/s/subspace/out/subdoc/subdoc+0x2880148)
#42 0x00007f5dc5e456ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#43 0x00007f5dc5e45785 call_init ./csu/../csu/libc-start.c:128:20
#44 0x00007f5dc5e45785 __libc_start_main ./csu/../csu/libc-start.c:347:5
#45 0x000055df2c5ee351 _start (/home/danakj/s/subspace/out/subdoc/subdoc+0x287e351)
tools/run_subdoc.sh: line 14: 1990666 Aborted                 (core dumped) out/subdoc/subdoc -p out --out docs --include-file-pattern /sus/ --exclude-file-pattern /third_party/ --css subdoc-test-style.css --copy-file subdoc/gen_tests/subdoc-test-style.css --copy-file web/logo.png --favicon "logo.png;image/png" --project-md sus/project.md --project-logo logo.png --project-name Subspace --ignore-bad-code-links i8_unittest

[cor3ntin]

Could you point me to the definition of 1_usize? the bug is probably somewhere around that. Better yet, a reduction or a preprocessed source file would help a lot.

[danakj]

This is the UDL:

https://github.com/chromium/subspace/blob/6243bc49617a92220d2bfafca364ac6b05ef72b9/sus/num/__private/literals.h#L44-L49

It is defined for _usize here:

https://github.com/chromium/subspace/blob/6243bc49617a92220d2bfafca364ac6b05ef72b9/sus/num/unsigned_integer.h#L258-L259

So you get

sus::num::usize inline consteval operator""_usize(unsigned long long val) {
  if (val > static_cast<unsigned long long>(sus::num::usize::MAX_PRIMITIVE))
    throw "Integer literal out of bounds for sus::num::usize";
  return sus::num::usize(static_cast<decltype(sus::num::usize::primitive_value)>(val));
}

Note that it is consteval.

[danakj]

A simple repro is hard as this seems to need a clang tool that will continue after errors in one file into processing the AST of another file. If I do not emit any diagnostic errors, the assert does not occur, if I stop after errors in one file, the assert does not occur.

If you need to reproduce to get any further I can help you reproduce with subdoc if you would like.

[cor3ntin]

I trigger the crash with

int consteval operator""_foo(unsigned long long) {
    if(true) throw "";
    return 0;
}
struct C {
    static constexpr auto a =1_foo;
};

https://godbolt.org/z/d5bMrY51c

[llvmbot]

@llvm/issue-subscribers-clang-frontend

[danakj]

Interesting, note that the throw is not being hit when I am reproducing it (the value is 1 which is in range), but hopefully that helps explain something.