Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find break limits for DocBlock tests on fly #104213

Merged
merged 3 commits into from
Jan 12, 2024
Merged

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Jan 10, 2024

The test failure is related to #104159, where we had an overestimate of the RAM usage for X-ArrayVector. Instead of updating the break limits, this PR uses the breaker utility that @nik9000 wrote to dynamically compute the limits on-the-fly.

Closes #104191

@dnhatn dnhatn added >test Issues or PRs that are addressing/adding tests :Analytics/ES|QL AKA ESQL v8.13.0 labels Jan 10, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 10, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)


public void testBuildBreaks() {
testBuildBreaks(ByteSizeValue.ofBytes(between(0, MAX_BUILD_BREAKS_LIMIT)));
testBuildBreaks(ByteSizeValue.ofBytes(randomLongBetween(0, MAX_BUILD_BREAKS_LIMIT)));
}

public void testBuildBreaksMax() {
testBuildBreaks(ByteSizeValue.ofBytes(MAX_BUILD_BREAKS_LIMIT));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need the MAX limit test if we're going to find this on the fly. The act of finding it confirms that it breaks.

private static long MAX_SHARD_SEGMENT_DOC_MAP_BREAKS = -1;

@Before
public void calculateBreakLimit() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd move this to the tests that need them. If the build process doesn't properly release on failure this could leave behind un-released blocks. Better to have that fail in the test, i think.

@dnhatn
Copy link
Member Author

dnhatn commented Jan 11, 2024

@nik9000 I've reworked these tests as you suggested. Can you take another look? Thanks!

@dnhatn dnhatn requested a review from nik9000 January 11, 2024 18:06
Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@dnhatn
Copy link
Member Author

dnhatn commented Jan 12, 2024

Thank you, Nik!

@dnhatn dnhatn merged commit 4f261fc into elastic:main Jan 12, 2024
15 checks passed
@dnhatn dnhatn deleted the fix-doc-break branch January 12, 2024 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test Issues or PRs that are addressing/adding tests v8.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] DocVectorTests testShardSegmentDocMapBreaks failing
3 participants