Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Segment Replication] Reduce/remove document parsing on replicas #7164

Closed
andrross opened this issue Apr 14, 2023 · 1 comment · Fixed by #7279
Closed

[Segment Replication] Reduce/remove document parsing on replicas #7164

andrross opened this issue Apr 14, 2023 · 1 comment · Fixed by #7279
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request

Comments

@andrross
Copy link
Member

When using segment replication, TransportShardBulkAction.performOpOnReplica() is invoked on replicas in order to persist the document into the translog on the replica and give the same durability guarantees as document replication. Here are two snapshots of CPU profiling for the same workload for both docrep and segrep:

Document Replication

docrep

Segment Replication

segrep

It clearly shows that NRTReplicationEngine does not invoke IndexWriter, whereas InternalEngine does. This is expected since the replicas do not need to index the document. However, a lot of work is spent doing DocumentMapper#parse in both cases. The question is: can this be refactored so that with segment replication the replicas avoid doing some or all of that document parsing?

@anasalkouz
Copy link
Member

@mch2 Can you share new profiler after fixing the issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants