Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(output): fluent-bit integrate OpenSearch plugin #298

Merged
merged 1 commit into from
May 23, 2022
Merged

feat(output): fluent-bit integrate OpenSearch plugin #298

merged 1 commit into from
May 23, 2022

Conversation

mangoGoForward
Copy link
Member

Signed-off-by: mango xu.weiKyrie@foxmail.com

What this PR does / why we need it:

Fluent-bit integrate OpenSearch plugin

Which issue(s) this PR fixes:

Fixes #284

Does this PR introduced a user-facing change?

Action Required: Should upgrade fluent-bit image version to 1.9

// Prefix keys with this string
LogstashPrefixKey string `json:"logstashPrefixKey,omitempty"`
// When enabled, mapping types is removed and Type option is ignored. Types are deprecated in APIs in v7.0. This options is for v7.0 or later.
SuppressTypeName string `json:"suppressTypeName,omitempty"`
Copy link
Member

@wenchajun wenchajun May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.fluentbit.io/manual/pipeline/outputs/opensearch
When enabled, mapping types is removed and Type option is ignored.
The value defaults to off.This appears to have only on and off values, which seems to be possible with bool types?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I would to fix now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@wenchajun
Copy link
Member

The corresponding crd file does not seem to be generated here

openSearchOutput := ClusterOutput{
TypeMeta: metav1.TypeMeta{
APIVersion: "fluentbit.fluent.io/v1alpha2",
Kind: "Output",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind shoud be ClusterOutput?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there should be ClusterOutput. I noticed that there are many place used Output, Input or Filter in fluent-bit config, seems they should be also need add a Cluster prefix, and do we need solve also in this PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Pls help to correct it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input, output, filter parser CRDs are all changed to clusterxxx in v1.0.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

},
ObjectMeta: metav1.ObjectMeta{
Name: "opensearch_output_0",
Namespace: "testnamespace",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No namespace I think if use cluster CRD.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got, I will fix. thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Signed-off-by: mango <xu.weiKyrie@foxmail.com>
@benjaminhuo
Copy link
Member

Thank you for your contribution @mangoGoForward !
If this PR requires fluentbit v1.9.0+, we can release a new version of the operator after fluentbit v1.9.4 is release to fix fluent/fluent-bit#5426 @wenchajun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fluentbit integrated OpenSearch plugin
5 participants