Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed esexporter serialized document to avoid conflicting keys #33454

Merged
merged 11 commits into from
Jun 11, 2024

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented Jun 10, 2024

Description:
Sort() or Dedup() call was missing before runnin Serialize() with ECS mapping enabled

Link to tracking Issue:
Fixes #33264

Testing: Added UT

Documentation: -

Comment on lines 234 to 245
expectedDocFields := pcommon.NewMap()
err = expectedDocFields.FromRaw(map[string]any{
"service.name": "foo.bar",
"service.version": "1.1.0",
"agent.name": "otlp",
"host.os.platform": "darwin",
"host.os.full": "Mac OS Mojave",
"host.os.name": "Mac OS X",
"host.os.version": "10.14.1",
"host.os.type": "macos",
"host.name": "localhost",
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something obvious, but is expectedDocFields being used in this test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes they are not. leftovers

m := encodeModel{
mode: MappingECS,
dedot: true,
dedup: true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this test testing de-duplication? I ran it with setting dedup: false and it still passes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you run it without a fix, resulting document is malformed.
the fix was really in calling Sort or Dedup before Serailize

Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, can you also undo the change in #32917 ?

@carsonip
Copy link
Contributor

Link to tracking Issue: #33264

Do you mean "Fixes #33264" ?

@michalpristas
Copy link
Contributor Author

michalpristas commented Jun 11, 2024

do we need changelog for this one?
i will add one

Copy link
Member

@andrzej-stencel andrzej-stencel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, the fix is to sort the document fields before serializing when dedot is enabled (the default).

@andrzej-stencel andrzej-stencel merged commit c963057 into open-telemetry:main Jun 11, 2024
162 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 11, 2024
t00mas pushed a commit to t00mas/opentelemetry-collector-contrib that referenced this pull request Jun 18, 2024
…telemetry#33454)

**Description:** 
`Sort()` or `Dedup()` call was missing before runnin `Serialize()` with
ECS mapping enabled

**Link to tracking Issue:** 
Fixes open-telemetry#33264 

**Testing:** Added UT

**Documentation:** -
cparkins pushed a commit to AmadeusITGroup/opentelemetry-collector-contrib that referenced this pull request Jul 11, 2024
…telemetry#33454)

**Description:** 
`Sort()` or `Dedup()` call was missing before runnin `Serialize()` with
ECS mapping enabled

**Link to tracking Issue:** 
Fixes open-telemetry#33264 

**Testing:** Added UT

**Documentation:** -
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[exporter/elasticsearch] Regression: Duplicate Key in JSON After #31694
5 participants