[exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request #11459

sfc-gh-sili · 2024-10-16T08:06:54Z

Description

This PR changes mergeBatchFunc and mergeBatchSplit function as a member function of batchRequest.

Link to tracking issue

Testing

Documentation

codecov · 2024-10-18T00:30:45Z

Codecov Report

Attention: Patch coverage is 89.24731% with 10 lines in your changes missing coverage. Please review.

Project coverage is 91.41%. Comparing base (5cd035b) to head (bd64227).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
exporter/exporterhelper/internal/request.go	83.63%	7 Missing and 2 partials ⚠️
exporter/exporterhelper/metrics_batch.go	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11459      +/-   ##
==========================================
- Coverage   91.47%   91.41%   -0.07%     
==========================================
  Files         433      433              
  Lines       23616    23657      +41     
==========================================
+ Hits        21603    21625      +22     
- Misses       1645     1658      +13     
- Partials      368      374       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bogdandrutu · 2024-10-18T17:14:41Z

exporter/internal/request.go

+	Merge(context.Context, BatchRequest) (BatchRequest, error)
+	MergeSplit(context.Context, exporterbatcher.MaxSizeConfig, BatchRequest) ([]BatchRequest, error)


Please document the functions.

Sounds good.

bogdandrutu · 2024-10-18T17:15:02Z

exporter/internal/request.go

+	Merge(context.Context, BatchRequest) (BatchRequest, error)
+	MergeSplit(context.Context, exporterbatcher.MaxSizeConfig, BatchRequest) ([]BatchRequest, error)


I think (maybe next PR) we can make the second func just Split.

I tried this some time ago. AFAIR there was a performance hit coming from the Merge->Split sequence of calls. But I'd be happy if we can make it work with no problems

dmitryax · 2024-10-18T19:02:53Z

exporter/exporterhelper/exporterhelperprofiles/profiles_batch_test.go

+func (req *tracesRequest) Merge(_ context.Context, _ exporterhelper.BatchRequest) (exporterhelper.BatchRequest, error) {
+	return nil, nil
+}
+
+// MergeSplit splits and/or merges the profiles into multiple requests based on the MaxSizeConfig.
+func (req *tracesRequest) MergeSplit(_ context.Context, _ exporterbatcher.MaxSizeConfig, _ exporterhelper.BatchRequest) (
+	[]exporterhelper.BatchRequest, error) {
+	return nil, nil
+}


Is there a test where tracesRequest needs the BatchRequest implementation?

I agree this is confusing. tracesRequest in this file is solely for testing that merging profilesBatchRequest and tracesRequest would fail. Let me rename it to dummyRequest.

dmitryax · 2024-10-18T20:38:42Z

exporter/exporterhelper/internal/batch_sender.go

 	}
-	return bs.sendMergeBatch(ctx, req)
+	return bs.sendMergeBatch(ctx, req.(internal.BatchRequest))


Custom Request not implementing batching function would panic here. Let's return an error instead?

It is ok for the moment to panic I believe since we are experimental. We will make it required to implement this to setup batching soon.

@sfc-gh-sili for the moment move the functions on Request.

@bogdandrutu Thanks!
Do you mean instead of having Request and BatchRequest, we would like to have Merge() and MergeSplitBatch() as a required function and then just have Request? For users who do not do batching they can leave Merge() and MergeSplit() as noop I assume?

For users who do not do batching they can leave Merge() and MergeSplit() as noop I assume?

Correct.

.chloggen/merge-function-as-requet-method.yaml

bogdandrutu · 2024-10-21T19:20:44Z

Please rebase

…pen-telemetry#11459)  #### Description This PR changes `mergeBatchFunc` and `mergeBatchSplit` function as a member function of `batchRequest`.  #### Link to tracking issue  #### Testing  #### Documentation

sfc-gh-sili force-pushed the sili-merge branch 3 times, most recently from 39122c8 to 6f5643a Compare October 17, 2024 09:31

sfc-gh-sili marked this pull request as ready for review October 17, 2024 09:31

sfc-gh-sili requested a review from a team as a code owner October 17, 2024 09:31

sfc-gh-sili requested a review from mx-psi October 17, 2024 09:31

sfc-gh-sili changed the title ~~mergeBatchFunc moved to request~~ [exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request Oct 17, 2024

mx-psi requested review from bogdandrutu and dmitryax October 17, 2024 10:55

bogdandrutu approved these changes Oct 18, 2024

View reviewed changes

bogdandrutu reviewed Oct 18, 2024

View reviewed changes

dmitryax reviewed Oct 18, 2024

View reviewed changes

sfc-gh-sili force-pushed the sili-merge branch from 075207b to 52d6e24 Compare October 18, 2024 22:10

sfc-gh-sili requested a review from dmitryax October 21, 2024 01:16

dmitryax approved these changes Oct 21, 2024

View reviewed changes

dmitryax reviewed Oct 21, 2024

View reviewed changes

.chloggen/merge-function-as-requet-method.yaml Show resolved Hide resolved

.chloggen/merge-function-as-requet-method.yaml Show resolved Hide resolved

sfc-gh-sili added 7 commits October 21, 2024 12:52

mergeBatchFunc moved to request

f7c608d

Add chlog

0f5e73b

fix change log

9546974

Fix test

8d28975

Edits according to feedback

c74b52d

Made merge() and mergesplit() required method

2b2c528

Updated change log

a130237

sfc-gh-sili force-pushed the sili-merge branch from 315fe8b to 2a6163d Compare October 21, 2024 19:55

Mod tidied

bd64227

sfc-gh-sili force-pushed the sili-merge branch from 2a6163d to bd64227 Compare October 21, 2024 19:57

bogdandrutu merged commit 2efeae4 into open-telemetry:main Oct 21, 2024
48 of 49 checks passed

github-actions bot added this to the next release milestone Oct 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request #11459

[exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request #11459

sfc-gh-sili commented Oct 16, 2024 •

edited

Loading

codecov bot commented Oct 18, 2024 •

edited

Loading

bogdandrutu Oct 18, 2024

sfc-gh-sili Oct 18, 2024

bogdandrutu Oct 18, 2024

dmitryax Oct 18, 2024 •

edited

Loading

dmitryax Oct 18, 2024

sfc-gh-sili Oct 18, 2024

dmitryax Oct 18, 2024

bogdandrutu Oct 18, 2024

bogdandrutu Oct 18, 2024

sfc-gh-sili Oct 18, 2024 •

edited

Loading

bogdandrutu Oct 18, 2024

bogdandrutu commented Oct 21, 2024

		Merge(context.Context, BatchRequest) (BatchRequest, error)
		MergeSplit(context.Context, exporterbatcher.MaxSizeConfig, BatchRequest) ([]BatchRequest, error)

[exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request #11459

[exporter] moved mergeBatchFunc and mergeBatchSplitFunc to request #11459

Conversation

sfc-gh-sili commented Oct 16, 2024 • edited Loading

Description

Link to tracking issue

Testing

Documentation

codecov bot commented Oct 18, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmitryax Oct 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfc-gh-sili Oct 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bogdandrutu commented Oct 21, 2024

sfc-gh-sili commented Oct 16, 2024 •

edited

Loading

codecov bot commented Oct 18, 2024 •

edited

Loading

dmitryax Oct 18, 2024 •

edited

Loading

sfc-gh-sili Oct 18, 2024 •

edited

Loading