Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure in CachedBlobContainerIndexInputTests.testRandomReads #70763

Closed
danhermann opened this issue Mar 23, 2021 · 2 comments · Fixed by #70668
Closed

Failure in CachedBlobContainerIndexInputTests.testRandomReads #70763

danhermann opened this issue Mar 23, 2021 · 2 comments · Fixed by #70668
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI

Comments

@danhermann
Copy link
Contributor

Build scan: https://gradle-enterprise.elastic.co/s/3g6dnmunvdxgw

Repro line: ./gradlew ':x-pack:plugin:searchable-snapshots:test' --tests "org.elasticsearch.index.store.cache.CachedBlobContainerIndexInputTests.testRandomReads" -Dtests.seed=B14197630B410474 -Dtests.security.manager=true -Dtests.locale=id-ID -Dtests.timezone=America/Indiana/Indianapolis -Druntime.java=11

Reproduces locally?: Yes

Applicable branches: master

Failure history: So far, just once on 3/23/21

Failure excerpt:


java.lang.AssertionError: Expected at most 1 ranges fetched from the source |  
-- | --
   | Expected: a value less than or equal to <1L> |  
   | but: <2L> was greater than <1L>

at __randomizedtesting.SeedInfo.seed([B14197630B410474:84433590E1EDA05E]:0) |  
-- | --
   |   | at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18) |  
   |   | at org.junit.Assert.assertThat(Assert.java:956) |  
   |   | at org.elasticsearch.index.store.cache.CachedBlobContainerIndexInputTests.testRandomReads(CachedBlobContainerIndexInputTests.java:148)
@danhermann danhermann added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI labels Mar 23, 2021
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Mar 23, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@ywelsch
Copy link
Contributor

ywelsch commented Mar 23, 2021

Will be fixed by #70668

dakrone added a commit to dakrone/elasticsearch that referenced this issue Mar 23, 2021
@dakrone
Mute AbstractSearchableSnapshotsTestCase and all sub tests
a2942c0 
Relates to elastic#70763 and can be unmuted when elastic#70668 is unmuted
@dakrone dakrone mentioned this issue Mar 23, 2021
Merged
dakrone added a commit that referenced this issue Mar 23, 2021
@dakrone
Mute AbstractSearchableSnapshotsTestCase and all sub tests (#70773)
3cf2dd5 
Relates to #70763 and can be unmuted when #70668 is unmuted
dakrone added a commit that referenced this issue Mar 23, 2021
@dakrone
Mute AbstractSearchableSnapshotsTestCase and all sub tests (#70773)
a36d86d 
Relates to #70763 and can be unmuted when #70668 is unmuted
@ywelsch ywelsch mentioned this issue Mar 24, 2021
Merged
ywelsch added a commit that referenced this issue Mar 24, 2021
@ywelsch
Always use CacheService for caching metadata blobs (#70668)
296ac1a 
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for
caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for
this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support).

This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when
reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data
(which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty.

Closes #70728
Closes #70763
ywelsch added a commit to ywelsch/elasticsearch that referenced this issue Mar 24, 2021
@ywelsch
Always use CacheService for caching metadata blobs (elastic#70668)
ddd115c 
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for
caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for
this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support).

This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when
reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data
(which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty.

Closes elastic#70728
Closes elastic#70763
@ywelsch ywelsch mentioned this issue Mar 24, 2021
Merged
ywelsch added a commit that referenced this issue Mar 24, 2021
@ywelsch
Always use CacheService for caching metadata blobs (#70668) (#70795)
4b67753 
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for
caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for
this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support).

This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when
reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data
(which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty.

Closes #70728
Closes #70763
ywelsch added a commit to ywelsch/elasticsearch that referenced this issue Mar 26, 2021
@ywelsch
Always use CacheService for caching metadata blobs (elastic#70668) (e…
822db8c 
…lastic#70795)

This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for
caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for
this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support).

This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when
reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data
(which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty.

Closes elastic#70728
Closes elastic#70763
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Always use CacheService for caching metadata blobs ywelsch/elasticsearch
3 participants
@ywelsch @elasticmachine @danhermann