Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lastExecutionTime for autofollow coroutine #508

Merged
merged 1 commit into from
Aug 26, 2022
Merged

Add lastExecutionTime for autofollow coroutine #508

merged 1 commit into from
Aug 26, 2022

Conversation

ankitkala
Copy link
Member

Signed-off-by: Ankit Kala [email protected]

Description

Currently if the autofollow coroutine is not getting executed for any reason, there is no mechanism to get visibility into that.
This change adds a new key last_execution_time in the autofollow stats which user can monitor to ensure that autofollow task is getting executed as expected.

Issues Resolved

485

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov-commenter
Copy link

Codecov Report

Merging #508 (2f29d65) into main (a859596) will decrease coverage by 2.83%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main     #508      +/-   ##
============================================
- Coverage     75.16%   72.33%   -2.84%     
+ Complexity     1007      988      -19     
============================================
  Files           141      141              
  Lines          4579     4591      +12     
  Branches        506      510       +4     
============================================
- Hits           3442     3321     -121     
- Misses          823      951     +128     
- Partials        314      319       +5     
Impacted Files Coverage Δ
...arch/replication/task/autofollow/AutoFollowTask.kt 71.89% <100.00%> (-1.08%) ⬇️
...cation/action/changes/TransportGetChangesAction.kt 63.49% <0.00%> (-25.40%) ⬇️
...rch/replication/action/status/ShardInfoResponse.kt 72.26% <0.00%> (-19.33%) ⬇️
...ch/replication/task/index/IndexReplicationState.kt 51.66% <0.00%> (-15.00%) ⬇️
...ication/action/setup/TransportSetupChecksAction.kt 59.52% <0.00%> (-14.29%) ⬇️
...action/stop/TransportStopIndexReplicationAction.kt 60.75% <0.00%> (-13.91%) ⬇️
...ication/seqno/RemoteClusterRetentionLeaseHelper.kt 80.00% <0.00%> (-11.67%) ⬇️
...tion/repository/RemoteClusterMultiChunkTransfer.kt 90.24% <0.00%> (-9.76%) ⬇️
.../replication/metadata/store/ReplicationMetadata.kt 62.96% <0.00%> (-8.65%) ⬇️
...rch/replication/task/shard/ShardReplicationTask.kt 69.34% <0.00%> (-6.57%) ⬇️
... and 18 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@ankitkala ankitkala marked this pull request as ready for review August 24, 2022 12:46
@ankitkala ankitkala requested a review from a team August 24, 2022 12:46
@@ -45,6 +45,7 @@ import org.opensearch.tasks.TaskId
import org.opensearch.threadpool.Scheduler
import org.opensearch.threadpool.ThreadPool
import java.util.concurrent.ConcurrentSkipListSet
import kotlin.collections.HashMap
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not used, Lets remove the import

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep

@@ -95,6 +95,7 @@ class UpdateAutoFollowPatternIT: MultiClusterRestTestCase() {
var af_stats = stats.get("autofollow_stats")!! as ArrayList<HashMap<String, Any>>
for (key in af_stats) {
assert(key["num_success_start_replication"]!! as Int == 2)
assert(key["last_execution_time"]!! as Long != 0L)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add checks after the poll duration as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

@ankitkala ankitkala merged commit 8f0a55c into opensearch-project:main Aug 26, 2022
@opensearch-trigger-bot
Copy link

The backport to 1.1 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.1 1.1
# Navigate to the new working tree
cd .worktrees/backport-1.1
# Create a new branch
git switch --create backport/backport-508-to-1.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8f0a55c9a7e32f643345a972d28e4829d7e3c530
# Push it to GitHub
git push --set-upstream origin backport/backport-508-to-1.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.1

Then, create a pull request where the base branch is 1.1 and the compare/head branch is backport/backport-508-to-1.1.

@opensearch-trigger-bot
Copy link

The backport to 1.2 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.2 1.2
# Navigate to the new working tree
cd .worktrees/backport-1.2
# Create a new branch
git switch --create backport/backport-508-to-1.2
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8f0a55c9a7e32f643345a972d28e4829d7e3c530
# Push it to GitHub
git push --set-upstream origin backport/backport-508-to-1.2
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.2

Then, create a pull request where the base branch is 1.2 and the compare/head branch is backport/backport-508-to-1.2.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)

Co-authored-by: Ankit Kala <[email protected]>
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)

Co-authored-by: Ankit Kala <[email protected]>
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)

Co-authored-by: Ankit Kala <[email protected]>
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)

Co-authored-by: Ankit Kala <[email protected]>
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)

Co-authored-by: Ankit Kala <[email protected]>
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 26, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
(cherry picked from commit 8f0a55c)
ankitkala added a commit that referenced this pull request Aug 29, 2022
Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>

Signed-off-by: Ankit Kala <[email protected]>
@ankitkala ankitkala deleted the autofollow_bug branch April 5, 2023 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants