Net attributes getters changes (in preparation for HTTP spec impl) #6503

mateuszrzeszutek · 2022-08-23T16:57:30Z

I started working on implementing the most recent HTTP spec changes, and I think I ran into several problems with the current shape of our net.* attributes extractors.

I added a couple of methods for determining the host name/port/socket to the NetServerAttributesGetter (I think these will be needed in the RPC semconv too)
I looked through most of our instrumentations, and I came to a conclusion that in most places the InetSocketAddress retrieved from the library code actually represents the socket address, not the logical name/port. It is especially true for most of the HTTP instrumentations, since in these the peer name/port is supposed to be extracted from the URL, not from the socket. I decided to change the InetSocket... classes so that they only extract net.sock.* attributes from the InetSocketAddress classes, and leave peerName/peerPort for the user to implement.

@trask @lmolkova please take a look at this draft - I'm quite interested in your opinion on the InetSocketAddress changes in particular.

(the 2nd commit contains the actual changes, the first one is just cleanup)

lmolkova · 2022-08-24T06:05:30Z

Looks good!

One thing I'm not sure about is keeping logical net.peer.name, net.peer.port (and host pair) in NetAttribute*.
E.g. unix domain socket instrumentation would never have logical attributes but would have to implement methods anyway.

The alternative is something like LogicalPeerAttribute*, LogicalHostAttribute* interfaces - HTTP and other conventions would implement them directly. gRPC might be tricky since it can work over local sockets.

I don't have a strong opinion, methods are already there and more abstractions won't make life easier. I also assume that every network-related semantic convention would need logical attributes, so current approach in this PR is fine too.

mateuszrzeszutek · 2022-08-24T10:05:35Z

The alternative is something like LogicalPeerAttribute*, LogicalHostAttribute* interfaces - HTTP and other conventions would implement them directly. gRPC might be tricky since it can work over local sockets.

I don't have a strong opinion, methods are already there and more abstractions won't make life easier. I also assume that every network-related semantic convention would need logical attributes, so current approach in this PR is fine too.

Hmm, I think I'll leave them in the Net*Getter interfaces for now; some of the socket attributes have conditions that depend on logical attributes, e.g. net.sock.peer.addr is recommended "If available and different from net.peer.name and if net.sock.peer.addr is set."

mateuszrzeszutek · 2022-08-30T15:43:20Z

@open-telemetry/java-instrumentation-maintainers this PR is ready for review. It is absurdly huge though (net semconv is used in the vast majority of all instrumentations after all) -- I tried to split it into 3 commits (instrumentation-api changes, actual getter implementation changes, fixing tests) to make it more readable, but if you have any alternative idea please let me know -- I'll try making this PR more manageable.

trask

will keep reviewing, just wanted to leave first comments

...rc/main/java/io/opentelemetry/instrumentation/testing/junit/http/AbstractHttpServerTest.java

...rc/main/java/io/opentelemetry/instrumentation/testing/junit/http/AbstractHttpClientTest.java

testing-common/src/main/groovy/io/opentelemetry/instrumentation/test/base/HttpServerTest.groovy

...mconv/src/main/java/io/opentelemetry/instrumentation/api/instrumenter/net/NetAttributes.java

trask

a couple more comments, sorry for the pace..

...in/java/io/opentelemetry/instrumentation/api/instrumenter/net/NetClientAttributesGetter.java

trask · 2022-09-08T04:21:31Z

.../opentelemetry/instrumentation/apachedubbo/v2_7/internal/DubboNetClientAttributesGetter.java

-    // dubbo 3 doesn't set remote address for client calls
-    if (address == null) {
-      URL url = request.url();
-      address = InetSocketAddress.createUnresolved(url.getHost(), url.getPort());
-    }


this is a nice example where the new semantic conventions cleared things up

...lemetry/instrumentation/api/instrumenter/net/InetSocketAddressNetClientAttributesGetter.java

...o/opentelemetry/javaagent/instrumentation/apachehttpasyncclient/ApacheHttpClientRequest.java

...ain/java/io/opentelemetry/instrumentation/apachehttpclient/v4_3/ApacheHttpClientRequest.java

trask · 2022-09-08T04:47:03Z

.../opentelemetry/javaagent/instrumentation/couchbase/v2_6/CouchbaseNetworkInstrumentation.java

-          }
-        }
-
+        requestInfo.setPeerAddress(channelHandlerContext.channel().remoteAddress());


mateuszrzeszutek · 2022-09-08T10:44:04Z

a couple more comments, sorry for the pace..

No problem at all! This PR is so huge that I don't expect anybody to review it in one go 🙈 😅

trask

😅 that was not the straight-forward refactoring I was expecting, thx for going through all of those instrumentations to map them correctly

...tation/liberty/compile-stub/src/main/java/com/ibm/wsspi/http/channel/HttpRequestMessage.java

instrumentation/jedis/jedis-4.0/javaagent/src/test/groovy/Jedis40ClientTest.groovy

trask · 2022-09-08T19:32:00Z

...lemetry/instrumentation/api/instrumenter/net/InetSocketAddressNetClientAttributesGetter.java

+    if (remoteAddress instanceof Inet6Address) {
+      return "inet6";
+    }
+    return null;


@lmolkova should we report inet when that is known? the spec says we don't need to, but it feels like we may be losing something by not reporting it?

I think I understand now.

It's conditionally required

If different than inet and if any of net.sock.peer.addr or net.sock.host.addr are set. Consumers of telemetry SHOULD accept both IPv4 and IPv6 formats for the address in net.sock.peer.addr if net.sock.family is not set. This is to support instrumentations that follow previous versions of this document.

So if consumers see net.sock.peer.addr or net.sock.host.addr, and they do not see any net.sock.family, then they are allowed to assume that the net.sock.family is inet.

...n/grpc-1.6/library/src/main/java/io/opentelemetry/instrumentation/grpc/v1_6/GrpcRequest.java

...brary/src/main/java/io/opentelemetry/instrumentation/grpc/v1_6/TracingClientInterceptor.java

...metry/javaagent/instrumentation/reactornetty/v1_0/ReactorNettyNetClientAttributesGetter.java

trask · 2022-09-08T20:42:37Z

...emetry/javaagent/instrumentation/netty/v4/common/client/NettyConnectNetAttributesGetter.java

-    if (remoteAddress == null) {
-      remoteAddress = request.remoteAddressOnStart();
+  public String peerName(NettyConnectionRequest request, @Nullable Channel channel) {
+    SocketAddress requestedAddress = request.remoteAddressOnStart();


it seems like all netty operations are at socket layer, can you clarify why this works for net.peer.name?

My reasoning here was that in the establishing connection phase, the socket address passed at the start (before the DNS lookup & connection occurs) is actually a logical address

trask · 2022-09-11T22:25:36Z

@mateuszrzeszutek any reason not to merge this for 1.18.0?

mateuszrzeszutek · 2022-09-12T07:56:24Z

If you're fine with it then I'm fine with it too 👍

…pen-telemetry#6503) * Net attributes getters changes: instrumentation-api-semconv changes * Net attributes getters changes: getter implementations * Net attributes getters changes: test fixes * Remove net.sock.host.name * code review comments * default getter methods & getPeerSocketAddress() method name * set authority in grpc earlier

anuragagarwal561994 · 2023-01-28T21:37:32Z

@mateuszrzeszutek this is an issue with the apache httpasyncclient instrumentation.

So we have used target.getAddress() here but the method was added in http core v4.3

If I see on maven central with apache httpasyncclient v4.1 https://mvnrepository.com/artifact/org.apache.httpcomponents/httpasyncclient/4.1

The default version that comes along for httpcore is 4.4.x but that doesn't mean that the user doesn't have a way to override this.

Hence someone using httpasyncclient 4.1.x with < 4.4.x version of httpcore, things will break.

trask · 2023-01-30T03:26:56Z

@anuragagarwal561994 can you open a new issue for this comment so it does not get lost? thx!

anuragagarwal561994 · 2023-01-30T08:09:43Z

@trask done

mateuszrzeszutek force-pushed the net-attributes branch 23 times, most recently from a46d26a to 312cc62 Compare August 30, 2022 10:23

Mateusz Rzeszutek added 2 commits August 30, 2022 14:06

Net attributes getters changes: instrumentation-api-semconv changes

3abaf5e

Net attributes getters changes: getter implementations

e925d1b

mateuszrzeszutek force-pushed the net-attributes branch from 312cc62 to b3285ad Compare August 30, 2022 12:06

Net attributes getters changes: test fixes

6574cbe

mateuszrzeszutek force-pushed the net-attributes branch from b3285ad to 6574cbe Compare August 30, 2022 14:01

mateuszrzeszutek marked this pull request as ready for review August 30, 2022 15:41

mateuszrzeszutek requested a review from a team as a code owner August 30, 2022 15:41

mateuszrzeszutek requested a review from lmolkova August 30, 2022 15:43

trask reviewed Aug 31, 2022

View reviewed changes

Mateusz Rzeszutek added 2 commits August 31, 2022 10:31

Remove net.sock.host.name

47b0731

code review comments

2957133

trask reviewed Sep 8, 2022

View reviewed changes

default getter methods & getPeerSocketAddress() method name

11eaf00

trask approved these changes Sep 8, 2022

View reviewed changes

set authority in grpc earlier

84f87a7

mateuszrzeszutek mentioned this pull request Sep 9, 2022

Avoid creating a new URI instance in gRPC and Reactor Netty instrumentations #6568

Closed

trask added this to the v1.18.0 milestone Sep 11, 2022

trask merged commit cfdbe75 into open-telemetry:main Sep 12, 2022

mateuszrzeszutek deleted the net-attributes branch September 13, 2022 09:12

mateuszrzeszutek mentioned this pull request Dec 19, 2022

Capture net.host.* attributes where possible #6287

Closed

anuragagarwal561994 mentioned this pull request Jan 30, 2023

GetAddress not present in httpcore < 4.3 #7674

Closed

anuragagarwal561994 mentioned this pull request Feb 28, 2023

Adds commons module for apache httpclient #7914

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Net attributes getters changes (in preparation for HTTP spec impl) #6503

Net attributes getters changes (in preparation for HTTP spec impl) #6503

mateuszrzeszutek commented Aug 23, 2022 •

edited

Loading

lmolkova commented Aug 24, 2022 •

edited

Loading

mateuszrzeszutek commented Aug 24, 2022

mateuszrzeszutek commented Aug 30, 2022

trask left a comment

trask left a comment

trask Sep 8, 2022

trask Sep 8, 2022

mateuszrzeszutek commented Sep 8, 2022

trask left a comment

trask Sep 8, 2022

trask Sep 11, 2022

trask Sep 8, 2022

mateuszrzeszutek Sep 9, 2022

trask commented Sep 11, 2022

mateuszrzeszutek commented Sep 12, 2022

anuragagarwal561994 commented Jan 28, 2023

trask commented Jan 30, 2023

anuragagarwal561994 commented Jan 30, 2023

Net attributes getters changes (in preparation for HTTP spec impl) #6503

Net attributes getters changes (in preparation for HTTP spec impl) #6503

Conversation

mateuszrzeszutek commented Aug 23, 2022 • edited Loading

lmolkova commented Aug 24, 2022 • edited Loading

mateuszrzeszutek commented Aug 24, 2022

mateuszrzeszutek commented Aug 30, 2022

trask left a comment

Choose a reason for hiding this comment

trask left a comment

Choose a reason for hiding this comment

trask Sep 8, 2022

Choose a reason for hiding this comment

trask Sep 8, 2022

Choose a reason for hiding this comment

mateuszrzeszutek commented Sep 8, 2022

trask left a comment

Choose a reason for hiding this comment

trask Sep 8, 2022

Choose a reason for hiding this comment

trask Sep 11, 2022

Choose a reason for hiding this comment

trask Sep 8, 2022

Choose a reason for hiding this comment

mateuszrzeszutek Sep 9, 2022

Choose a reason for hiding this comment

trask commented Sep 11, 2022

mateuszrzeszutek commented Sep 12, 2022

anuragagarwal561994 commented Jan 28, 2023

trask commented Jan 30, 2023

anuragagarwal561994 commented Jan 30, 2023

mateuszrzeszutek commented Aug 23, 2022 •

edited

Loading

lmolkova commented Aug 24, 2022 •

edited

Loading