Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redirect command Stdout & Stderr for command_runner #2448

Merged
merged 1 commit into from
Feb 5, 2018

Conversation

lichen2013
Copy link

@lichen2013 lichen2013 commented Jan 23, 2018

CombinedOutput waits command complete. When we run a long run command, such as continuously get new log entries, it failed to get run results.

Fixes: #2447

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 23, 2018
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 23, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: lichen2013
We suggest the following additional approver: luxas

Assign the PR to them by writing /assign @luxas in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@dlorenc
Copy link
Contributor

dlorenc commented Jan 23, 2018

@minikube-bot ok to test

@@ -48,11 +48,26 @@ func (*ExecRunner) Run(cmd string) error {
func (*ExecRunner) CombinedOutput(cmd string) (string, error) {
glog.Infoln("Run with output:", cmd)
c := exec.Command("/bin/bash", "-c", cmd)
out, err := c.CombinedOutput()
out, err := c.StdoutPipe()
defer out.Close()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CombinedOutput also gets us Stderr, right? I think if you add that this should be fine.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated with another approach, and redirected both Stdout & Stderr

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 24, 2018
@lichen2013 lichen2013 changed the title Use StdoutPipe instead of CombinedOutput for exec_runner Redirect command Stdout & Stderr for exec_runner Jan 24, 2018
@lichen2013
Copy link
Author

lichen2013 commented Jan 24, 2018

Did a little more check, I have a guess that issue #2447 might not only exists for exec_runner.
Maybe a better fix should be update the function GetClusterLogs in CommandRunner interface ?

  • allow os.Stdout or *bytes.Buffer to be set as an input
  • no need to have the string return

}
return string(out), nil

return "", nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is meant to return the combined output. That's the string parameter in the output.

I think we need to do something like you had before, where we capture both stdout and stderr as variables, then run the command and return the combined strings.

Copy link
Author

@lichen2013 lichen2013 Jan 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a long running command, like continues printing new logs, it is not possible to get the return strings.

I would like to propose the change like this:

func (*ExecRunner) CombinedOutputTo(cmd string, out io.Writer) error {
        glog.Infoln("Run with output:", cmd)
        c := exec.Command("/bin/bash", "-c", cmd)
        c.Stdout = out
        c.Stderr = out
        err := c.Run()
        if err != nil {
                return errors.Wrapf(err, "running command: %s\n.", cmd)
        }
        return nil
}
func (e *ExecRunner) CombinedOutput(cmd string) (string, error) {
        var b bytes.Buffer
        err := e.CombinedOutputTo(cmd,  &b)
        if err != nil {
		return "", errors.Wrapf(err, "running command: %s\n output: %s", cmd, b.Bytes())
        }
        return b.Bytes(), nil
}

For command like minikube logs -f, it should call CombinedOutputTo directly:

CombinedOutputTo(cmd, os.Stdout)

@r2d4
Copy link
Contributor

r2d4 commented Jan 26, 2018

I actually think the issue is with the SSHRunner implementation rather than the ExecRunner. Both implement the CommandRunner interface. Neither support streaming. I agree the interface should probably be changed to accommodate it.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 1, 2018
@lichen2013 lichen2013 changed the title Redirect command Stdout & Stderr for exec_runner Redirect command Stdout & Stderr for command_runner Feb 1, 2018
@lichen2013
Copy link
Author

Tested with vm-driver none & virtualbox.

Copy link
Contributor

@dlorenc dlorenc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for spending the time on this!

@dlorenc
Copy link
Contributor

dlorenc commented Feb 2, 2018

@minikube-bot test this please

@dlorenc
Copy link
Contributor

dlorenc commented Feb 2, 2018

D'oh, looks like a small build error still: pkg/minikube/bootstrapper/localkube/localkube_test.go:209:15: l.GetClusterLogs undefined (type LocalkubeBootstrapper has no field or method GetClusterLogs)

CombinedOutput function return after command complete.
When we run a long run command, such as continuously get
new log entries, it failed to get run results.

Fixes: kubernetes#2447
@dlorenc
Copy link
Contributor

dlorenc commented Feb 5, 2018

@minikube-bot test this please

@dlorenc dlorenc merged commit 82ea016 into kubernetes:master Feb 5, 2018
@lichen2013 lichen2013 deleted the log_f branch February 6, 2018 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants