Fluentbit output to elasticsearch is missing milliseconds on @timestamp

[FarhadF]

Bug Report

Describe the bug

Fluentbit output to elasticsearch is missing milliseconds on @timestamp

config:

[FILTER]
    Name parser
    Match *
    Key_Name log
    Parser json
    Reserve_Data True

[OUTPUT]
    Name es
    Match *
    Host XXX
    Logstash_Prefix my_prefix
    Logstash_Format On
    Type _doc
    Trace_Output On

Version: Fluent Bit v1.4.2

Example record(please note the 000 in @timestamp (this is from the trace output of es plugin):

{"@timestamp":"2020-08-19T09:39:29.000Z","level":"error","timestamp":"2020-08-19T09:39:29.996959864Z","message":"Failed to retrieve queue url"}

Actual log line from the application:

{"level":"error","timestamp":"2020-08-19T09:39:29.996959864Z","message":"Failed to retrieve queue url"}

[FarhadF]

Things I tried:

• changing Flush to 0.5, still no milliseconds.
• set Time_Key_Format
• trying my config locally (just changed INPUT to tail to read from file) -> this works correctly

{"@timestamp":"2020-08-19T11:47:06.540Z","level":"error","timestamp":"2020-08-19T09:57:29.996959864Z","message":"Failed to retrieve queue url"}

[FarhadF]

One workaround would be to parse application timestamp key in parser and replace @timestamp

[zihengCat]

@FarhadF Hi, check this PR #2544 out.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.