Invalid cyclical TypeSpec in referenced assembly causes stack overflow

[nguerrera]

Although the EMCA-335 CLI specification does not permit TypeSpec tokens as modifiers, they are allowed in practice by ilasm, peverify, the CLR, and Roslyn. However, they open up the possibility of circular signatures, and the compiler hits a stack overflow while trying to decode one.

I have started to document such spec issues in dotnet/corefx#5435 and I'm now filing the bugs I found while experimenting to write that up.

Native csc.exe had the same problem.

Repro steps

• git clone https://github.com/nguerrera/signature-repros
• cd signature-repros
• build.bat
• csc.exe UseSignatureCycle2.cs /r:SignatureCycle2.exe

Expected result

• Compilation fails with suitable error message.

Actual result

• Stack overflow

cc @tmat @gafter @AlekseyTs

[gafter]

/cc @jaredpar

[tmat]

Adding a test to Roslyn.

[jaredpar]

Closing as #8027 addressed this.

[nguerrera]

@jaredpar As far as I can tell, a skipped test was added, but the issue was not resolved. Another related bug was fixed, but I think this one is still unresolved.

[jaredpar]

@tmat can you comment here? The PR claims to address the bug, does change some product code but does add two tests: one skipped and one active. I only saw the non-skipped parts when I was looking at this bug yesterday.

[gafter]

Reopening; the PR quoted as fixing this actually just added a (skipped) test for it. I'm not saying I know for sure the problem still exists, but I don't have any evidence that it has been fixed, so I'm reopening this for investigation and possible fixing.

[tmat]

I am not aware of any fix we made to address this.

[gafter]

The compilers have not been hardened against broken metadata. This should be fixed in the context of a (hypothetical) effort to harden the compilers against different kinds of broken metadata.