Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Fluentd to v1.15.3 #556

Merged
merged 1 commit into from
Feb 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
VERSION?=$(shell cat VERSION | tr -d " \t\n\r")
# Image URL to use all building/pushing image targets
FB_IMG ?= kubesphere/fluent-bit:v2.0.8
FD_IMG ?= kubesphere/fluentd:v1.14.6
FD_IMG ?= kubesphere/fluentd:v1.15.3
FO_IMG ?= kubesphere/fluent-operator:$(VERSION)
FD_IMG_BASE ?= kubesphere/fluentd:v1.14.6-arm64-base
FD_IMG_BASE ?= kubesphere/fluentd:v1.15.3-arm64-base

ARCH ?= arm64

Expand Down
2 changes: 1 addition & 1 deletion charts/fluent-operator/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ description: A Helm chart for Kubernetes
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 2.0.0
version: 2.0.1

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
Expand Down
2 changes: 1 addition & 1 deletion charts/fluent-operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ fluentd:
port: 24224
image:
repository: "kubesphere/fluentd"
tag: "v1.14.6"
tag: "v1.15.3"
replicas: 1
forward:
port: 24224
Expand Down
14 changes: 7 additions & 7 deletions cmd/fluent-watcher/fluentd/Dockerfile.amd64
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ RUN echo $(ls -al /code)
RUN CGO_ENABLED=0 go build -i -ldflags '-w -s' -o /fluentd/fluentd-watcher /code/cmd/fluent-watcher/fluentd/main.go

# Fluentd main image
FROM alpine:3.13
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.14.6"
FROM alpine:3.16
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.15.3"

# Do not split this into multiple RUN!
# Docker creates a layer for every RUN-Statement
Expand All @@ -23,11 +23,11 @@ RUN apk update \
build-base linux-headers \
ruby-dev gnupg \
&& echo 'gem: --no-document' >> /etc/gemrc \
&& gem install oj -v 3.10.18 \
&& gem install json -v 2.4.1 \
&& gem install async-http -v 0.54.0 \
&& gem install ext_monitor -v 0.1.2 \
&& gem install fluentd -v 1.14.6 \
&& gem install oj -v 3.13.22 \
&& gem install json -v 2.6.2 \
&& gem install async -v 1.30.3 \
&& gem install async-http -v 0.56.6 \
&& gem install fluentd -v 1.15.3 \
&& gem install bigdecimal -v 1.4.4 \
# NOTE: resolv v0.2.1 includes the fix for CPU spike issue due to DNS resolver.
# This hack is needed for Ruby 2.6.7, 2.7.3 and 3.0.1. (alpine image is still kept on 2.7.3)
Expand Down
19 changes: 10 additions & 9 deletions cmd/fluent-watcher/fluentd/Dockerfile.arm64
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ RUN apk add curl --no-cache
RUN curl -sL -o qemu-6.0.0.balena1-aarch64.tar.gz https://github.com/balena-io/qemu/releases/download/v6.0.0%2Bbalena1/qemu-6.0.0.balena1-aarch64.tar.gz && echo "$QEMU_DOWNLOAD_SHA256 *qemu-6.0.0.balena1-aarch64.tar.gz" | sha256sum -c - | tar zxvf qemu-6.0.0.balena1-aarch64.tar.gz -C . && mv qemu-6.0.0+balena1-aarch64/qemu-aarch64-static .

# Fluentd main image
FROM arm64v8/ruby:2.7-slim-bullseye
FROM arm64v8/ruby:3.1-slim-bullseye
COPY --from=builderqemu /go/qemu-aarch64-static /usr/bin/
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.14.6"
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.15.3"
ENV TINI_VERSION=0.18.0

# Do not split this into multiple RUN!
Expand All @@ -32,11 +32,12 @@ RUN apt-get update \
" \
&& apt-get install -y --no-install-recommends $buildDeps \
&& echo 'gem: --no-document' >> /etc/gemrc \
&& gem install oj -v 3.10.18 \
&& gem install json -v 2.4.1 \
&& gem install async-http -v 0.54.0 \
&& gem install ext_monitor -v 0.1.2 \
&& gem install fluentd -v 1.14.6 \
&& gem install oj -v 3.13.22 \
&& gem install json -v 2.6.2 \
&& gem install rexml -v 3.2.5 \
&& gem install async -v 1.30.3 \
&& gem install async-http -v 0.56.6 \
&& gem install fluentd -v 1.15.3 \
&& gem install bigdecimal -v 1.4.4 \
&& gem install resolv -v 0.2.1 \
&& gem install elasticsearch -v 7.13.3 \
Expand All @@ -55,8 +56,8 @@ RUN apt-get update \
fluent-plugin-aws-elasticsearch-service \
fluent-plugin-opensearch \
&& echo "plugin installed." \
&& wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
&& cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
&& wget -O /tmp/jemalloc-5.3.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/5.3.0/jemalloc-5.3.0.tar.bz2 \
&& cd /tmp && tar -xjf jemalloc-5.3.0.tar.bz2 && cd jemalloc-5.3.0/ \
&& ./configure && make \
&& mv lib/libjemalloc.so.2 /usr/lib \
&& apt-get purge -y --auto-remove \
Expand Down
12 changes: 7 additions & 5 deletions cmd/fluent-watcher/fluentd/Dockerfile.arm64.base
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ RUN apk add curl --no-cache
RUN curl -sL -o qemu-6.0.0.balena1-aarch64.tar.gz https://github.com/balena-io/qemu/releases/download/v6.0.0%2Bbalena1/qemu-6.0.0.balena1-aarch64.tar.gz && echo "$QEMU_DOWNLOAD_SHA256 *qemu-6.0.0.balena1-aarch64.tar.gz" | sha256sum -c - | tar zxvf qemu-6.0.0.balena1-aarch64.tar.gz -C . && mv qemu-6.0.0+balena1-aarch64/qemu-aarch64-static .

# Fluentd main image
FROM arm64v8/ruby:2.7-slim-bullseye
FROM arm64v8/ruby:3.1-slim-bullseye
COPY --from=builderqemu /go/qemu-aarch64-static /usr/bin/
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.14.6"
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.15.3"
ENV TINI_VERSION=0.18.0

# Do not split this into multiple RUN!
Expand All @@ -23,9 +23,11 @@ RUN apt-get update \
" \
&& apt-get install -y --no-install-recommends $buildDeps \
&& echo 'gem: --no-document' >> /etc/gemrc \
&& gem install oj -v 3.10.18 \
&& gem install json -v 2.4.1 \
&& gem install async-http -v 0.54.0 \
&& gem install oj -v 3.13.22 \
&& gem install json -v 2.6.2 \
&& gem install rexml -v 3.2.5 \
&& gem install async -v 1.30.3 \
&& gem install async-http -v 0.56.6 \
&& gem install ext_monitor -v 0.1.2 \
&& gem install bigdecimal -v 1.4.4 \
&& gem install resolv -v 0.2.1 \
Expand Down
6 changes: 3 additions & 3 deletions cmd/fluent-watcher/fluentd/Dockerfile.arm64.quick
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ RUN echo $(ls -al /code)
RUN CGO_ENABLED=0 go build -i -ldflags '-w -s' -o /fluentd/fluentd-watcher /code/cmd/fluent-watcher/fluentd/main.go

# Fluentd main image
FROM kubesphere/fluentd:v1.14.6-arm64-base
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.14.6"
FROM kubesphere/fluentd:v1.15.3-arm64-base
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.15.3"
ENV TINI_VERSION=0.18.0

# Do not split this into multiple RUN!
Expand All @@ -19,7 +19,7 @@ RUN buildDeps=" \
make gcc g++ libc-dev \
wget bzip2 gnupg dirmngr\
" \
&& gem install fluentd -v 1.14.6 \
&& gem install fluentd -v 1.15.3 \
&& apt-get purge -y --auto-remove \
-o APT::AutoRemove::RecommendsImportant=false \
$buildDeps \
Expand Down
15 changes: 8 additions & 7 deletions cmd/fluent-watcher/fluentd/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb

FROM alpine:3.17.1
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.14.6"
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.15.3"

# Do not split this into multiple RUN!
# Docker creates a layer for every RUN-Statement
Expand All @@ -16,11 +16,12 @@ RUN apk update \
build-base linux-headers \
ruby-dev gnupg \
&& echo 'gem: --no-document' >> /etc/gemrc \
&& gem install oj -v 3.10.18 \
&& gem install json -v 2.4.1 \
&& gem install async-http -v 0.54.0 \
&& gem install ext_monitor -v 0.1.2 \
&& gem install fluentd -v 1.14.6 \
&& gem install oj -v 3.13.22 \
&& gem install json -v 2.6.2 \
&& gem install rexml -v 3.2.5 \
&& gem install async -v 1.30.3 \
&& gem install async-http -v 0.56.6 \
&& gem install fluentd -v 1.15.3 \
&& gem install bigdecimal -v 1.4.4 \
# NOTE: resolv v0.2.1 includes the fix for CPU spike issue due to DNS resolver.
# This hack is needed for Ruby 2.6.7, 2.7.3 and 3.0.1. (alpine image is still kept on 2.7.3)
Expand All @@ -42,7 +43,7 @@ RUN apk update \
fluent-plugin-opensearch \
fluent-plugin-grafana-loki \
&& apk del .build-deps \
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem /usr/lib/ruby/gems/2.*/gems/fluentd-*/test
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem /usr/lib/ruby/gems/3.*/gems/fluentd-*/test

RUN addgroup -S fluent && adduser -S -G fluent fluent \
# for log storage (maybe shared with host)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 3
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down
2 changes: 1 addition & 1 deletion manifests/fluentd/fluentd-cluster-cfg-output-es.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 1
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down
2 changes: 1 addition & 1 deletion manifests/fluentd/fluentd-cluster-cfg-output-kafka.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 1
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 1
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down
2 changes: 1 addition & 1 deletion manifests/fluentd/fluentd-mixed-cfgs-output-es.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 1
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down
2 changes: 1 addition & 1 deletion manifests/fluentd/fluentd-namespaced-cfg-output-es.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
bind: 0.0.0.0
port: 24224
replicas: 1
image: kubesphere/fluentd:v1.14.6
image: kubesphere/fluentd:v1.15.3
fluentdCfgSelector:
matchLabels:
config.fluentd.fluent.io/enabled: "true"
Expand Down