OTEL_RESOURCE_ATTRIBUTES is ignored when using OTEL_RESOURCE_ATTRIBUTES_* (e.g. OTEL_RESOURCE_ATTRIBUTES_POD_NAME)

[Synthenses]

Describe the bug

When OTEL_RESOURCE_ATTRIBUTES envierment variable is ignored when OTEL_RESOURCE_ATTRIBUTES_* (e.g. OTEL_RESOURCE_ATTRIBUTES_POD_NAME) is also defind

Steps to reproduce

Default spring boot project with opentelemetry-spring-boot-starter v2.2.0-alpha and following the configuration in the application.properties
otel.resource.attributes=environment=dev,xyz=foo
otel.resource.attributes.pod.name=TestPod

Define a Env Variable Like OTEL_RESOURCE_ATTRIBUTES_POD_NAME=TestPod and OTEL_RESOURCE_ATTRIBUTES=env=dev,xyz=foo

REPO
Here you can see in the console log that only TestPod is contained in the Json, if you remove this from the application.properties you can find dev and foo in the log

Expected behavior

log collected by opentelemetry should contain the resource attributes set via OTEL_RESOURCE_ATTRIBUTES envierment variable

Actual behavior

log collected by opentelemetry does not contain the resource attributes set via OTEL_RESOURCE_ATTRIBUTES envierment variable, only the resource attribute pod.name=TestPod

Javaagent or library instrumentation version

opentelemetry-spring-boot-starter v2.2.0-alpha

Environment

JDK: Temurin-17.0.9+9
OS: Windows 10
Spring Boot: 3.2.4

Additional context

Found Bug by deploying an application with spirng starter boot starter to kubernetes, which is injected via inject sdk which generating env variabl OTEL_RESOURCES_ATTRIBUTES

[laurit]

cc @zeitlinger @jeanbisutti

[breedx-splk]

Hey @Synthenses thanks for the report. A couple of questions:

1. What's up with the syntax otel.resource.attributes."pod.name"=TestPod? Is that a special spring thing? Doesn't seem like the quotes should be needed there, huh?
2. Is it your expectation that you can add attributes to the resource by simply appending them to the property prefix or the env var prefix? If so, where did you read that?
3. Does it happen with system properties as well, or only with spring application.properties?

[trask]

oh, right, I believe this feature was removed in #10453, when we consolidated the spring boot starter to use the standard autoconfigure module configuration, looks like we missed that in the release notes

[Synthenses]

Hey @Synthenses thanks for the report. A couple of questions:

1. What's up with the syntax [`otel.resource.attributes."pod.name"=TestPod`](https://github.com/Synthenses/OTEL_BUG/blob/master/src/main/resources/application.properties#L3)? Is that a special spring thing? Doesn't seem like the quotes should be needed there, huh?

2. Is it your expectation that you can add attributes to the resource by simply appending them to the property prefix or the env var prefix? If so, where did you read that?

3. Does it happen with system properties as well, or only with spring application.properties?

1. the quotes are not requiered but the bug still occures
2. Thats not my expection i want to write it like OTEL_RESOURCE_ATTRIBUTES=env=dev,xyz=foo because the autoinstrumentation with inject sdk does add the attributes like this as env variable in kubernetes Pods, and also inject OTEL_RESOURCE_ATTRIBUTES_POD_NAME there I noticed that it will only read the Pod_Name
3. It has the same behavior with System props, env vars, and application.properties

[zeitlinger]

We're using spring logic to read the resource attributes - so one could argue that it's a spring boot bug.
However, this is a case where spring has to decide if the property should be ready from a string (as in the env var) or a map (as in the yaml file).
We have a map converter to help spring convert the string to a map.

In the end, I think spring decides to give priority to the map - but this is not an acceptable solution for us - because our contract is that the env var will always be honored - so I'll fix this on our side.