SearchableSnapshotDirectoryStatsTests failures #70728
Labels
:Distributed Coordination/Snapshot/Restore
Anything directly related to the `_snapshot/*` APIs
Team:Distributed (Obsolete)
Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
>test-failure
Triaged test failures from CI
Comments
javanna
added
:Distributed Coordination/Snapshot/Restore
elasticmachine
added
the
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
Will be fixed by #70668 |
ywelsch
added a commit
that referenced
this issue
Mar 24, 2021
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support). This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data (which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty. Closes #70728 Closes #70763
ywelsch
added a commit
to ywelsch/elasticsearch
that referenced
this issue
Mar 24, 2021
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support). This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data (which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty. Closes elastic#70728 Closes elastic#70763
ywelsch
added a commit
that referenced
this issue
Mar 24, 2021
This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support). This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data (which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty. Closes #70728 Closes #70763
ywelsch
added a commit
to ywelsch/elasticsearch
that referenced
this issue
Mar 26, 2021
…lastic#70795) This PR unifies CachedBlobContainerIndexInput and FrozenIndexInput so that they share the same infrastructure for caching metadata blobs as well as header and footer ranges for data blobs. The idea is to always use CacheService for this, which does not evict the metadata, and which efficiently stores the information on disk (using sparse file support). This also allows us to align writes in FrozenCacheService to 4KB block sizes in this PR, which addresses an issue when reusing regions from the shared cache, as writes that are not aligned on page cache boundaries causes the existing data (which we don't care about) to be loaded from disk, which comes with a dramatic performance penalty. Closes elastic#70728 Closes elastic#70763
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Some searchable snapshots tests are failing today:
Build scan:
https://gradle-enterprise.elastic.co/s/rkhnksundfwxy
https://gradle-enterprise.elastic.co/s/qg2pyjp2w5ie4/
Repro line:
./gradlew ':x-pack:plugin:searchable-snapshots:test' --tests "org.elasticsearch.index.store.SearchableSnapshotDirectoryStatsTests.testReadBytesContiguously" -Dtests.seed=BFE4833CEC7197EF -Dtests.security.manager=true -Dtests.locale=es-VE -Dtests.timezone=America/El_Salvador -Druntime.java=8
./gradlew ':x-pack:plugin:searchable-snapshots:test' --tests "org.elasticsearch.index.store.SearchableSnapshotDirectoryStatsTests.testCachedBytesReadsAndWrites" -Dtests.seed=BFE4833CEC7197EF -Dtests.security.manager=true -Dtests.locale=es-VE -Dtests.timezone=America/El_Salvador -Druntime.java=8
Applicable branches: master and 7.x
The text was updated successfully, but these errors were encountered: