Initial implementation of ANTLR4 Lexer Support

[lkishalmi]

Take number 2.

This is a bunch of collected knowledge, help to integrate ANTLR v4 based Lexers into NetBeans. This PR does not include the change of the existing ANTLR v4 based languages. If this one goes through, I'd provide separate PR-s for:

• ANTLR Language Support
• TOML Support
• JSON Support

I could use this on my upcoming projects for:

• Terraform, for two or more lexers
• GoLang Lexer
• Maybe: HTTP Lexer, may create a new Language Support tutorial featuring that simple grammar.

[mbien]

looks good to me. Although, the last time i wrote a grammar was over a decade ago for the NetBeans GL and CL packs - so I am not going to be able to spot bugs just by looking at it :)

consider adding the test to the CI ide-modules-test job:
https://github.com/apache/netbeans/blob/master/.github/workflows/main.yml#L341
since they won't start running automatically.

[matthiasblaesing]

I know this is already in, but one thing I thought about when looking at this was: are we prepared to make this API yet or should this be a purely internal package (i.e. Friend-only API without external users)?

[lkishalmi]

Well, this one does not introduce something big/groundbreaking. It's only 4 classes. Two of them: AbstractAntlrLexerBridge and LexerInputCharStream are actually implementations of an existing API. The AbstractAntlrLexerBridge.LexerState duties and usage is strict and well defined, that leaves AntlrTokenSequence as new here. That one is also simple and straight forward enough.

I could imagine that there could be bugs in the implementation, though I'd not expect the API change without keeping backward compatibility any time soon. (Unless ANTLR forces us to do so...)

[lkishalmi]

Hmm. Could make the LexerInputCharStream package private though...