Emit package events

[jack-berg]

Resolves #9253.

[jack-berg]

I could use some advice on whether this is the best way to get a hook to inspect every class that's loaded.

The idea is basically:

• For each class, determine the URL of the containing JAR
• Add each unique JAR to a processing queue
• Process each JAR by finding / extracting various metadata and emitting an event with event.domain=jvm, event.name=info
• If disabled, dont add the ClassFileTransformer, and don't start the processing thread

[jack-berg]

This rather crude test is really just meant to show where I'm at with this. The following content is printed to the console when you run it:

{event.domain="package", event.name="info", package.checksum="a96d6978cb02654c4d45b14babcd7777566fdd89", package.description="Gradle", package.path="gradle-base-services-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="950782352b1e7b490948b60f6e28435748aa4e8c", package.description="Gradle", package.path="gradle-logging-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="15cf5acff6d5c3e6d8d9389dfaa104696d5f93cd", package.description="Gradle", package.path="gradle-process-services-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="b5a4b6d16ab13e34a88fae84c35cd5d68cac922c", package.description="slf4j-api", package.name="org.slf4j:slf4j-api", package.path="slf4j-api-1.7.30.jar", package.type="jar", package.version="1.7.30"}
{event.domain="package", event.name="info", package.checksum="1524e41b99b14b6e79312e67d5ca97fd173cde10", package.description="Gradle", package.path="gradle-logging-api-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="e200977bcea40414ce41e3fbf22c0a5c5ff45d05", package.description="Gradle", package.path="gradle-native-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="cbe87e9ddf4da3b8fb0619e1c2cdfb5e73c18b46", package.description="Gradle", package.path="gradle-enterprise-logging-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="18751f6367987dcb341e0293885a7d78509c27ab", package.description="Gradle", package.path="gradle-build-operations-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="8d465dda8d734dee9820e8cf83f1465eaaab4f99", package.description="Gradle", package.path="gradle-enterprise-workers-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="cf4a57cfcee7fb3e6eb98091f7ce1beb0de54347", package.description="Gradle", package.path="gradle-testing-base-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="77d5d61f0994ba87f92dc73b0f0157cef9503b18", package.description="Gradle", package.path="gradle-testing-jvm-infrastructure-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="66a92110e0267b4af39bbe4f68033ffbb1273c17", package.path="native-platform-0.22-milestone-24.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="78fd0713dc18f5c6af1bebb7f8ced189c9fca78d", package.description="Gradle", package.path="gradle-files-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="48123c815efde7fb26ad9db44a4179505fbc27ce", package.description="Gradle", package.path="gradle-file-temp-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="d58bebff8cbf70ff52b59208586095f467656c30", package.name="org.slf4j:jul-to-slf4j", package.path="jul-to-slf4j-1.7.30.jar", package.type="jar", package.version="1.7.30"}
{event.domain="package", event.name="info", package.checksum="1b2514d2e8a13f4c795939528bb30ef48cd99837", package.description="Gradle", package.path="gradle-testing-junit-platform-8.3.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="0c6b206e80cfd97e66a1364003724491c757b92f", package.path="kryo-2.24.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="0ce1edb914c94ebc388f086c6827e8bdeec71ac2", package.description="Commons Lang by The Apache Software Foundation", package.name="commons-lang:commons-lang", package.path="commons-lang-2.6.jar", package.type="jar", package.version="2.6"}
{event.domain="package", event.name="info", package.checksum="89a1922534ed102be1fb2a8c0b2c6151297a12bf", package.description="junit-platform-launcher by junit.org", package.path="junit-platform-launcher-1.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="276c4edcf64fabb5a139fa7b4f99330d7a93b804", package.description="junit-platform-engine by junit.org", package.path="junit-platform-engine-1.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="d533ff2c286eaf963566f92baf5f8a06628d2609", package.description="junit-platform-commons by junit.org", package.path="junit-platform-commons-1.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="90587932d718fc51a48112d33045a18476c542ad", package.description="junit-jupiter-engine by junit.org", package.path="junit-jupiter-engine-5.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="af4e0a3cb6901aa53d6401003fc10638014b39b1", package.description="junit-vintage-engine by junit.org", package.path="junit-vintage-engine-5.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="653d1b0abf992ac0b3b7262991a8b73bce648497", package.description="spock-core by spockframework.org", package.path="spock-core-2.4-M1-groovy-4.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="8b528fa0a49297582932bdd9096c84dedf71cf25", package.path="junit-pioneer-1.9.1.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="8ac9e16d933b6fb43bc7f576336b8f4d7eb5ba12", package.description="JUnit by JUnit", package.path="junit-4.13.2.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="2fe4ba3d31d5067878e468c96aa039005a9134d3", package.description="junit-jupiter-api by junit.org", package.path="junit-jupiter-api-5.10.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="7c85ec12689813fdcfb66ab9b7533184e169ce1d", package.description="testing-common by OpenTelemetry", package.path="opentelemetry-testing-common-1.30.0-alpha-SNAPSHOT.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="68151d4465c12183db8edc4f9d8f0a878bada16b", package.description="logs", package.path="opentelemetry-sdk-logs-1.29.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="4d512d413b426037b7027fd39426542748ed4581", package.description="Groovy: a powerful, multi-faceted language for the JVM by The Apache Software Foundation", package.path="groovy-4.0.14.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="b7414c6d6a05b9f65b964689abd19a2c236783a6", package.description="spock-junit4 by spockframework.org", package.path="spock-junit4-2.4-M1-groovy-4.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="1820c0968dba3a11a1b30669bb1f01978a91dedc", package.description="hamcrest by hamcrest.org", package.path="hamcrest-2.2.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="2c39784846001a9cffd6c6b89c78de62c0d80fb8", package.name="org.awaitility:awaitility", package.path="awaitility-4.2.0.jar", package.type="jar", package.version="4.2.0"}
{event.domain="package", event.name="info", package.checksum="41eb7184ea9d556f23e18b5cb99cad1f8581fc00", package.description="slf4j-api", package.name="org.slf4j:slf4j-api", package.path="slf4j-api-2.0.7.jar", package.type="jar", package.version="2.0.7"}
{event.domain="package", event.name="info", package.checksum="e003985c3b9a8865c6bbc938864235cdeea3f516", package.description="Logback Classic Module by QOS.ch", package.name="ch.qos.logback:logback-classic", package.path="logback-classic-1.3.8.jar", package.type="jar", package.version="1.3.8"}
{event.domain="package", event.name="info", package.checksum="27d73ef908f832790b9eda936dd6476cc22e47c5", package.description="Logback Core Module by QOS.ch", package.name="ch.qos.logback:logback-core", package.path="logback-core-1.3.8.jar", package.type="jar", package.version="1.3.8"}
{event.domain="package", event.name="info", package.checksum="45010687a1181dc886fd12403e48cf76e94c65b1", package.description="all", package.path="opentelemetry-api-1.29.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="a4cf6857f268b9637ea330fffc70c1e6421d1d55", package.description="context", package.path="opentelemetry-context-1.29.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="85811b4757731b2fd4d8ff24ec802dcb03b895d5", package.description="instrumentation-api by OpenTelemetry", package.path="opentelemetry-instrumentation-api-1.30.0-SNAPSHOT.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="5e810d79e96c0615813d5ed2859413222c5340b4", package.description="instrumentation-api-semconv by OpenTelemetry", package.path="opentelemetry-instrumentation-api-semconv-1.30.0-alpha-SNAPSHOT.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="8ee51f51d9c1c959b537c8dba67d7a524204b974", package.description="semconv", package.path="opentelemetry-semconv-1.29.0-alpha.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="6bb59616f1180286bc2ccf40e34d636984581ba9", package.description="metrics", package.path="opentelemetry-sdk-metrics-1.29.0.jar", package.type="jar"}
{event.domain="package", event.name="info", package.checksum="5cc1be17aed4e1e396c6b5359518f369a42ebc37", package.path="protobuf-java-3.23.4.jar", package.type="jar"}

Notice that some jars don't have a version package.name, package.version, or package.description. These fields are missing if we can't find certain things in the JAR, like the manifest "Implementation-Name" and "Implementation-Vendor fields", or the pom.properties file containing the groupid and artifactId.

We can probably get clever and do a little better than what I have currently, but there are going to be limits.

[mateuszrzeszutek]

What about ear? Some people might still use that

[jack-berg]

Oh didn't know about ear. Let me see if I can't get the test module to build an .ear so I can confirm the logic works.

[laurit]

The ear isn't usually an issue because app servers will extract them during the deploy. The exception is jboss that uses a custom url protocol that can represent nested archives. Similarly war files are almost always extracted to support ServletContext.getRealPath, I think only weblogic may deploy (there is a checkbox in the weblogic admin console) from packaged war, but that won't affect class loading as they extract WEB-INF/lib and also package classes from WEB-INF/classes into a jar.

[mateuszrzeszutek]

It would be nice to add some Spring Boot tests at some point: Spring Boot jars include the dependencies' jar files inside the main uberjar

[jack-berg]

It would be nice to add some Spring Boot tests at some point: Spring Boot jars include the dependencies' jar files inside the main uberjar

Can see the output of this in the smoke tests, right? The output looks normal to be for the spring boot smoke tests.

[tylerbenson]

I noticed you're not getting Implementation-Version. Is that intentional?

[jack-berg]

The version is being grabbed from the pom, so yes intentional. Although I don't feel particularly strongly about how the package.description field is derived.

[jack-berg]

Hey @open-telemetry/java-instrumentation-maintainers got buried for a couple weeks and lost track of this. Wondering if there's interest in getting this landing (in something close to its current form) and then iterating. There's obviously a lot to do, including starting and seeing through a semantic-conventions discussion, but I think this is a pretty good start and although it will be subject to some churn, can help provide useful insight today.