We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parsed log fields are ignored, and raw logs are written instead.
I have the following fluentbit configuration:
[SERVICE] Flush 1 Daemon Off Log_Level warning HTTP_Server true Parsers_File fluentbit/conf/parsers.conf [INPUT] Name tail Tag logfiles.data Path /var/log/auth.log Path_Key path Key_Name log Parser auth [OUTPUT] Name stdout Match * [OUTPUT] Name Loki Match * Url http://loki:3100/api/prom/push
Contents of fluentbit/conf/parsers.conf:
fluentbit/conf/parsers.conf
[PARSER] Name auth Format regex Regex ^(?<time>[^ ]* \d{2} \d{2}:\d{2}:\d{2}) (?<host>[\S]*) (?<program>[^\[\] ]*)(\[(?<pid>\d*)\])?: (?<message>.*)$ Time_Key time Time_Format %b %d %H:%M:%S Time_Keep true
Contents of /var/log/auth.log:
/var/log/auth.log
Aug 22 13:26:45 plutonium systemd-logind[814]: Removed session 6422.
I expected loki to store the stringified parsed data, similar to what stdout gives:
logfiles.data: [1566480405.000000000, {"path"=>"/var/log/auth.log", "time"=>"Aug 22 13:26:45", "host"=>"plutonium", "program"=>"systemd-logind", "pid"=>"814", "message"=>"Removed session 6422."}]
{"log":"Aug 22 13:26:05 plutonium systemd-logind[814]: Removed session 6422.","path":"/var/log/auth.log"}
Other outputs, behave like this:
[{u'pid': u'814', u'host': u'plutonium', u'program': u'systemd-logind', u'time': u'Aug 22 13:26:45', u'date': 1566480405.0, u'path': u'/var/log/auth.log', u'message': u'Removed session 6422.'}]
logfiles.data: [1566480405.000000, {"path":"/var/log/auth.log", "time":"Aug 22 13:26:45", "host":"plutonium", "program":"systemd-logind", "pid":"814", "message":"Removed session 6422."}]
Ubuntu 16.04.6 LTS
Within docker, started with, command:
/fluent-bit/bin/fluent-bit -c /fluent-bit/etc/fluent-bit-loki.conf -e /usr/lib/x86_64-linux-gnu/out_loki.so
And these folder binds:
"/fluentbit/conf/fluent-bit.conf:/fluent-bit/etc/fluent-bit-loki.conf:rw", "/fluentbit/conf/:/fluentbit/conf/:rw", "/var/log/:/var/log/:rw"
Fluent Bit v1.1.3
Fluent Bit v1.1.3 Copyright (C) Treasure Data level=info caller=out_loki.go:86 [flb-go]="Starting fluent-bit-go-loki" version="(version=0.2.0, branch=master, revision=5b25616)" level=info caller=out_loki.go:87 [flb-go]="provided parameter" URL=http://loki:3100/api/prom/push level=info caller=out_loki.go:88 [flb-go]="provided parameter" BatchWait= level=info caller=out_loki.go:89 [flb-go]="provided parameter" BatchSize= level=info caller=out_loki.go:90 [flb-go]="provided parameter" Labels= level=info caller=out_loki.go:91 [flb-go]="provided parameter" LogLevel= level=info caller=out_loki.go:92 [flb-go]="provided parameter" RemoveKeys=
Grafana v6.4.0-pre (299a0e2) loki_build_info{branch="master",goversion="go1.11.4",revision="481906c4",version="master-481906c"} 1.0
The text was updated successfully, but these errors were encountered:
This fluent-bit-go-loki plugin is merged into loki upstream. Could you test it instead? https://github.com/grafana/loki/tree/master/cmd/fluent-bit
Sorry, something went wrong.
No branches or pull requests
Problem
Parsed log fields are ignored, and raw logs are written instead.
Steps to replicate
I have the following fluentbit configuration:
Contents of
fluentbit/conf/parsers.conf
:Contents of
/var/log/auth.log
:Expected Behavior or What you need to ask
I expected loki to store the stringified parsed data, similar to what stdout gives:
Other outputs, behave like this:
Using Fluentd and loki plugin versions
Ubuntu 16.04.6 LTS
Within docker, started with, command:
And these folder binds:
Fluent Bit v1.1.3
The text was updated successfully, but these errors were encountered: