Baggage is lost between Filter and Controller when using Spring-Boot with Undertow

[rvenancio]

Describe the bug

When using auto instrumentation with the Java Agent with Spring-boot using undertow as application server, if we set the baggage in a Filter, it will not reach the Controllers, for instance:

@Component
public class LogFilter extends OncePerRequestFilter implements Ordered {

    private static final Logger LOG = LoggerFactory.getLogger(LogFilter.class);

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
        LOG.info("### LogFilter start ###");

        Baggage baggage = Baggage.builder().put("user.name", "jack").build();
        LOG.info("----> Baggage before LogFilter: {}", OtelUtils.getAll());

        try(Scope ignored = baggage.makeCurrent()) {
            LOG.info("----> Baggage inside LogFilter TRY: {}", OtelUtils.getAll());
            filterChain.doFilter(request, response);
            LOG.info("----> Baggage inside LogFilter TRY after doFilter: {}", OtelUtils.getAll());
        }

        LOG.info("----> Baggage after LogFilter: {}", OtelUtils.getAll());

        LOG.info("### LogFilter end ###");
    }

    @Override
    public int getOrder() {
        return -15;
    }
}

@RestController
public class FacadeController {
    private static final Logger LOG = LoggerFactory.getLogger(FacadeController.class);

    public FacadeController() {
    }

    @PostMapping(value = "/submit")
    public ResponseEntity<Void> submit(@RequestHeader(name = CORRELATION_ID_KEY) String correlationId) {
        LOG.debug("New request with correlation-id: {}", correlationId);

        LOG.info("----> All Baggage: {}", Baggage.fromContext(Context.current())
                .asMap()
                .entrySet()
                .stream()
                .collect(Collectors.toMap(
                        Map.Entry::getKey,
                        entry -> entry.getValue().getValue())));

        // send correlation-id in request payload
        Map<String, String> requestBody = new HashMap<>();
        requestBody.put(CORRELATION_ID_KEY, correlationId);

        return new ResponseEntity<>(HttpStatus.OK);
    }
}

As side note, this works as expected using a Tomcat. The problem only occurs when using undertow.

After some debugging, the problem is that the Java Agent seems to be replacing the ThreadLocal Storage with a context that does not have the Baggage. As shown in the image bellow:

Steps to reproduce

Setup a Spring-boot project with a OncePerRequestFilter and a Controller.

Expected behavior

Baggage should reach the Controller:

http.route=/submit, server.port=9090, url.scheme=http, thread.name=XNIO-1 I/O-1, user_agent.original=PostmanRuntime/7.36.1, network.protocol.version=1.1, network.peer.port=54890}, capacity=128, totalAddedValues=14}
2024-01-24 11:29:29.454 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ### LogFilter start ###
2024-01-24 11:29:29.455 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ----> Baggage before LogFilter: {}
2024-01-24 11:29:29.455 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ----> Baggage inside LogFilter TRY: {user.name=jack}
2024-01-24 11:29:29.456 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.FacadeController : ----> All Baggage: {{user.name=jack}}

Actual behavior

Baggage is empty in the Controller:

http.route=/submit, server.port=9090, url.scheme=http, thread.name=XNIO-1 I/O-1, user_agent.original=PostmanRuntime/7.36.1, network.protocol.version=1.1, network.peer.port=54890}, capacity=128, totalAddedValues=14}
2024-01-24 11:29:29.454 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ### LogFilter start ###
2024-01-24 11:29:29.455 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ----> Baggage before LogFilter: {}
2024-01-24 11:29:29.455 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.LogFilter : ----> Baggage inside LogFilter TRY: {user.name=jack}
2024-01-24 11:29:29.456 INFO 37092 --- [ XNIO-1 task-1] com.wit.otel.facade.FacadeController : ----> All Baggage: {}

Javaagent or library instrumentation version

at least higher than 1.30, both SDK and Javaagent

Environment

JDK: 8
OS: Windows 11 and MacOS Sonoma
Spring Boot starter web: 2.7.17

Additional context

No response

[laurit]

Could you provide a sample app that allows reproducing this, as a github repo or zip.

[rvenancio]

@laurit here it is: https://github.com/rvenancio/spring-boot-otel-baggage-demo
Check the steps in the README.