Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mimir.rules.kubernetes] When mimir is unavailable when the component start, it never tries again #1076

Open
QuentinBisson opened this issue Jun 18, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@QuentinBisson
Copy link

Request

Component startup should retry when connection to mimir is unavailable during the compoent starts but it does not and we have to manually restart the pod:

ts=2024-06-18T13:27:21.649522346Z level=error msg="failed to list rules from mimir" component=mimir.rules.kubernetes.local err="Get \"http://mimir-ruler.mimir.svc:8080/prometheus/config/v1/rules\": dial tcp 172.31.62.76:8080: connect: connection refused"
ts=2024-06-18T13:27:21.649584571Z level=error msg="starting up component failed" component=mimir.rules.kubernetes.local err="Get \"http://mimir-ruler.mimir.svc:8080/prometheus/config/v1/rules\": dial tcp 172.31.62.76:8080: connect: connection refused"

Use case

We need this because we deploy the agent at the same time as Mimir and we have to manually intervene

@QuentinBisson QuentinBisson added the enhancement New feature or request label Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant