Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue with prometheus remote write #378

Closed
tsearle opened this issue Nov 7, 2023 · 9 comments
Closed

issue with prometheus remote write #378

tsearle opened this issue Nov 7, 2023 · 9 comments
Assignees
Labels
bug Something isn't working

Comments

@tsearle
Copy link
Contributor

tsearle commented Nov 7, 2023

when using vector to remote write to qryn a 500 server error occurs

Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: {"level":30,"time":1699369189621,"pid":547,"hostname":"ip-10-150-3-53","name":"qryn","reqId":"req-1mx","req":{"method":"POST","url":"/api/v1/prom/write","hostname":"localhost:3100","remoteAddress":"127.0.0.1","remotePort":59928},"msg":"incoming request"}
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: (node:547) [FSTDEP017] FastifyDeprecation: You are accessing the deprecated "request.routerPath" property. Use "request.routeOptions.url" instead. Property "req.routerPath" will be removed in `fastify@5`.
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: (node:547) [FSTDEP018] FastifyDeprecation: You are accessing the deprecated "request.routerMethod" property. Use "request.routeOptions.method" instead. Property "req.routerMethod" will be removed in `fastify@5`.
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: (node:547) [FSTDEP017] FastifyDeprecation: You are accessing the deprecated "request.routerPath" property. Use "request.routeOptions.url" instead. Property "req.routerPath" will be removed in `fastify@5`.
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: (node:547) [FSTDEP018] FastifyDeprecation: You are accessing the deprecated "request.routerMethod" property. Use "request.routeOptions.method" instead. Property "req.routerMethod" will be removed in `fastify@5`.
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: (node:547) [FSTDEP017] FastifyDeprecation: You are accessing the deprecated "request.routerPath" property. Use "request.routeOptions.url" instead. Property "req.routerPath" will be removed in `fastify@5`.
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: {"level":50,"time":1699369189622,"pid":547,"hostname":"ip-10-150-3-53","name":"qryn","reqId":"req-1mx","err":"undefined parser\nError: undefined parser\n    at Parser.fn (/usr/lib/node_modules/qryn/parsers.js:263:15)\n    at ContentTypeParser.run (/usr/lib/node_modules/qryn/node_modules/fastify/lib/contentTypeParser.js:177:27)\n    at handleRequest (/usr/lib/node_modules/qryn/node_modules/fastify/lib/handleRequest.js:41:33)\n    at next (/usr/lib/node_modules/qryn/node_modules/fastify/lib/hooks.js:339:7)\n    at Object.<anonymous> (/usr/lib/node_modules/qryn/qryn_node.js:109:7)\n    at next (/usr/lib/node_modules/qryn/node_modules/fastify/lib/hooks.js:345:30)\n    at preParsingHookRunner (/usr/lib/node_modules/qryn/node_modules/fastify/lib/hooks.js:368:3)\n    at runPreParsing (/usr/lib/node_modules/qryn/node_modules/fastify/lib/route.js:587:5)\n    at next (/usr/lib/node_modules/qryn/node_modules/fastify/lib/hooks.js:237:9)\n    at /usr/lib/node_modules/qryn/node_modules/@fastify/cors/index.js:201:12","msg":"undefined parser"}
Nov 07 14:59:49 ip-10-150-3-53 qryn[547]: {"level":30,"time":1699369189622,"pid":547,"hostname":"ip-10-150-3-53","name":"qryn","reqId":"req-1mx","res":{"statusCode":500},"responseTime":0.9872800000011921,"msg":"request completed"}
@lmangani lmangani added the bug Something isn't working label Nov 7, 2023
@tsearle
Copy link
Contributor Author

tsearle commented Nov 7, 2023

POST /api/v1/prom/write HTTP/1.1.
x-prometheus-remote-write-version: 0.1.0.
content-encoding: snappy.
content-type: application/x-protobuf.
user-agent: Vector/0.33.1 (x86_64-unknown-linux-gnu 3cc27b9 2023-10-30 16:50:49.747931844).
accept-encoding: identity.
host: localhost:3100.
content-length: 1825.

@akvlad
Copy link
Collaborator

akvlad commented Nov 8, 2023

endpoint = "https://localhost:8087/api/v1/write"

It's a strange endpoint in the vector example.
The route we have this interceptor is: /api/prom/remote/write.
I'll retest vector and ask @lmangani if we should add an alias /api/v1/write -> `/api/prom/remote/write``

For now please try to change the endpoint route to /api/prom/remote/write.

@lmangani
Copy link
Collaborator

lmangani commented Nov 8, 2023

@tsearle our supported APIs are documented here

There are indeed many aliases from the various platforms, and we'd be glad to add any established aliases in qryn

@lmangani
Copy link
Collaborator

lmangani commented Nov 8, 2023

@tsearle could you confirm the last results + version of vector you are using for us to reproduce this fully?

@tsearle
Copy link
Contributor Author

tsearle commented Nov 8, 2023

still no difference

Nov 08 10:18:06 ip-10-150-3-90 qryn[594]: {"level":30,"time":1699438686447,"pid":594,"hostname":"ip-10-150-3-90","name":"qryn","reqId":"req-103d","req":{"method":"GET","url":"/api/prom/remote/write","hostname":"localhost:3100","remoteAddress":"127.0.0.1","remotePort":48828},"msg":"incoming request"} Nov 08 10:18:06 ip-10-150-3-90 qryn[594]: {"level":50,"time":1699438686448,"pid":594,"hostname":"ip-10-150-3-90","name":"qryn","reqId":"req-103d","err":"Unsupported Content-Encoding: snappy\nFastifyCompressError: Unsupported Content-Encoding: snappy\n at Object.preParsing (/usr/lib/node_modules/qryn/node_modules/@fastify/compress/index.js:322:24)\n at next (/usr/lib/node_modules/qryn/node_modules/fastify/lib>

@tsearle
Copy link
Contributor Author

tsearle commented Nov 8, 2023

vector 0.33.1-1 amd64

running on debian 12

@lmangani
Copy link
Collaborator

lmangani commented Nov 8, 2023

Thanks @tsearle the issue was related to the healthcheck performed by the latest vector
@akvlad provided a fix with #380 available in qryn:3.0.11

@lmangani lmangani closed this as completed Nov 8, 2023
@lmangani
Copy link
Collaborator

lmangani commented Nov 8, 2023

@all-contributors please add @tsearle for testing

Copy link

@lmangani

I've put up a pull request to add @tsearle! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants