Add no-piggyback-errors flag to syslog-parser() and syslog related source drivers.

[bazsi]

Problem:
In case syslog-parser() fails, the entire log message is cleared: all name-value pairs, tags, along with partially parsed values already extracted from the message when the parsing failed. This also clears anything that was set prior to syslog-parser() and unrelated to syslog parsing (e.g. $SOURCEIP or $SOURCE or even $TRANSPORT).

Then, syslog-ng adds some error information to the message and then attributes it to itself, as if it was coming from syslog-ng itself, but not via the internal() driver, rather by the driver which actually receives the message.

This behaviour makes sense in cases where the syslog parser is embedded in source drivers, as in that case the message should not have any other things yet.

It is not intuitive with syslog-parser() though, as in this case, we might have requested store-raw-message or even associated various name-value pairs to the message already.

Solution

This PR adds a no-piggyback-errors flag to syslog-parser (both the embedded case and the syslog-parser case). This flag is not enabled by default, as that would be an incompatible change with syslog-parser().

With no-piggyback-errors then the message will not be attributed to syslog-ng in case of errors. Actually it retains everything that was present at the time of the parse error, potentially things that were already extracted.

So $MSG remains that was set (potentially the raw message), $HOST may or may not be extracted, likewise for $PROGRAM, $PID, $MSGID, etc.

The error is still indicated via $MSGFORMAT set to "syslog-error" (but note #250, which will change this to syslog:error).

Others:

This branch also:

• adds predefined tags to indicate various parsing issues for RFC5424 (similarly to what we have for RFC3164).
• fixes that such piggybacked error messages lacked a $HOST value in case they were generated by a syslog-parser(), in contrast when they are generated as a part of a source driver.
• added a log_msg_clear_sdata() function, which did not exist before. there was a state of this branch where that would have been used, but now it is not. it is exercised by a unit test though, so I kept it here. could be dropped from the series.

[bazsi]

I'm thinking about retaining even more information in case parsing fails. The current behaviour drops everything, for instance the original sending IP address does seem useful when you want to diagnose parse errors.

[bazsi]

I am finished with the branch now and mostly happy how it shaped up. @mbonsack would be great to get your input.

[mbonsack]

LGTM based on our review this past weekend.

[MrAnno]

LGTM otherwise.

Another idea that may probably be a separate PR but could be attached to this new flag:

We should allow a syslog-parser() mode where we don't fall back to parsing bsdsyslog when we fail to parse the message as IETF syslog. This could be useful for writing complex parser logic (I saw something like this assumed in @jszigetvari 's filterx codes).

[bazsi]

LGTM otherwise.

Another idea that may probably be a separate PR but could be attached to this new flag:

We should allow a syslog-parser() mode where we don't fall back to parsing bsdsyslog when we fail to parse the message as IETF syslog. This could be useful for writing complex parser logic (I saw something like this assumed in @jszigetvari 's filterx codes).

it is already possible, there's a flag called 'no-rfc3164-fallback'

[bazsi]

if you can re-approve @MrAnno we could merge this so the work on the dependant PR can continue.
thanks