Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Legacy executors are executors after all #9304

Merged
merged 2 commits into from
Nov 10, 2020

Conversation

notnoop
Copy link
Contributor

@notnoop notnoop commented Nov 10, 2020

This fixes a bug where pre-0.9 executors fail to recover after an
upgrade.

The bug is that legacyExecutorWrappers didn't get updated with
ExecStreaming function, and thus failed to implement the Executor
function. Sadly, this meant that all recovery attempts fail, as the
runtime check in

raw, err := rpcClient.Dispense("executor")
if err != nil {
return nil, nil, fmt.Errorf("unable to dispense the executor plugin: %v", err)
}
executorPlugin, ok := raw.(Executor)
if !ok {
return nil, nil, fmt.Errorf("unexpected executor rpc type: %T", raw)
}

.

Impact

The bug affects all versions since 0.9.2: if a client upgrades from pre-0.9 to 0.9.2 or later; then all exec based tasks (i.e. exec, raw_exec, java, qemu) will fail to recover, will be leaked, and then nomad will start another task. The leaked pre-0.9 task will run un-interrupted and unmanaged until the client dies or it's killed manually.

This fixes a bug where pre-0.9 executors fail to recover after an
upgrade.

The bug is that legacyExecutorWrappers didn't get updated with
ExecStreaming function, and thus failed to implement the Executor
function. Sadly, this meant that all recovery attempts fail, as the
runtime check in
https://github.com/hashicorp/nomad/blob/b312aacbc961dfd976b6a402d22d80b45176a962/drivers/shared/executor/utils.go#L103-L110
.
@notnoop notnoop requested a review from schmichael November 10, 2020 15:35
@notnoop notnoop self-assigned this Nov 10, 2020
Copy link
Member

@schmichael schmichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof great catch.

[ci skip]
@notnoop notnoop merged commit 669f9cb into master Nov 10, 2020
@notnoop notnoop deleted the b-legacy-executors-are-executors branch November 10, 2020 17:54
schmichael pushed a commit that referenced this pull request Nov 18, 2020
schmichael pushed a commit that referenced this pull request Nov 19, 2020
schmichael pushed a commit that referenced this pull request Nov 19, 2020
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants