Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push and Pull send progress steps to supplied block; pushing raises an error if unauthorized. #222

Closed

Conversation

mrflip
Copy link

@mrflip mrflip commented Nov 29, 2014

This patch lets me show feedback during the long slow process of pushing or pulling an image.

  • A block supplied to Image#push or Image.create will be called with the parsed hash of each json message streamed back from the server.
  • create and push now raise errors if an error response is returned from the server
  • The VCR example for creating an image was actually that of a 401 Unauthorized request. I faked data from a successful pull into the 'pushes_the_Image.yml' VCR record, and used the old version as the test case for an unauthorized call.

I'm not quite sure how I should be handling exceptions in all this.

First, I'm assuming it's fair to raise an exception if any response from the server shows an error -- but if there are known-harmless errors out there, they will now be explodey.

Also, if an error condition occurs in the response_block, it is caught and re-thrown as an Excon error. So I'm throwing a generic error inside the block, then catching all excon errors and re-throwing only the ones I recognize (Unauthorized for push, Not Found for pull) as Docker errors. Let me know if you prefer some other strategy.

This obviates the PR in #209, as the last ID is retrieved naturally in the response block

@mrflip mrflip force-pushed the response_blocks_for_push_and_pull branch from 7d1d9d0 to 4dd8ad6 Compare November 29, 2014 13:07
@mrflip
Copy link
Author

mrflip commented Nov 29, 2014

So two code nanny alerts are tripping about a complex method, which I assume is because I'm assembling the response block in the method itself:

  def push(creds = nil, options = {}, &caller_blk)
    # (...setup stuff...)
    resp_blk = lambda do |chunk, *_|
      steps = Docker::Util.fix_json(chunk)
      steps.each do |step|
        raise step['error'] if step['error']
        caller_blk.call(step) if caller_blk
      end
    end
    #
    connection.post("/images/#{repo}/push", opts, :headers => headers, :response_block => resp_blk)
    self
  end

So I could fix that by

  def push(creds = nil, options = {}, &caller_blk)
    # (...setup stuff...)
     resp_blk = get_response_block(caller_blk)
    #
    connection.post("/images/#{repo}/push", opts, :headers => headers, :response_block => resp_blk)
    self
  end

  def get_response_block(caller_blk)
    lambda do |chunk, *_|
      steps = Docker::Util.fix_json(chunk)
      steps.each do |step|
        raise step['error'] if step['error']
        caller_blk.call(step) if caller_blk
      end
    end
  end

But at that point I have a method which takes a block calling a method to make a block; that block, given content by the later call of the first method, will call back to the callback block it was sent on behalf of the caller of the first method.

Which, I argue, is much less clear.
But I'll switch to the second version if you like.

@mrflip
Copy link
Author

mrflip commented Nov 29, 2014

I pushed another commit that passes the cane nanny; I'd argue it's worse but you may find it more readable.

The problem is that the block for create needs to smuggle back the id of the final layer seen, and so .create has to hand off a little bit of scope to do so.. The second version uses an array, which is passed by reference, as a pretense to accomplish this dishonestly. If y'all have suggestions about how to do this with a scalar instead, I would be less offended by the split-out block version.

@mrflip mrflip force-pushed the response_blocks_for_push_and_pull branch from af9f5c6 to 855323a Compare December 1, 2014 12:12
@nahiluhmot
Copy link
Contributor

Supporting progress steps is an interesting feature, however your README.md update is unrelated and should go in a separate pull. It's much easier to review and reason about pulls when they're single-purpose, so please remove that for now. Also, you're going to have to rebase off of master now that #220 is merged.

self
rescue StandardError => err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rescue StandardError => err is redundant, prefer rescue => err

The tests in push took the image tianon/true and did a build with no changes -- giving an image named both `tianon/true` and `<USERNAME>/true`.
Since `tianon/true` came first in the repo tags, the push specs tried to push to tianon/true, who we are not and not, and so the actual recorded response is an unauthorized error.
The next commit will catch such errors, making those tests fail. For cleanliness of diffs, I'm correcting the existing tests first.

My solution is to make the created image non-identical to tianon/true, by adding a gratuitous environment variable to the build script.
That leaves the image at hand named only `<USERNAME>/true`, and so it sails off to your actual account as it should.
@mrflip mrflip force-pushed the response_blocks_for_push_and_pull branch from 4702b1b to 5ed995d Compare December 10, 2014 05:27
raise ArgumentError, "Image does not have a name to push." if repo.nil?

def push(creds = nil, options = {}, &callback)
repo, tag = valid_repo_tag(options)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The push method tripped an ABC complexity flag, and this seemed like a reasonable idea anyway. Pulling the case statement out of the rescue block without screwing up backtraces seemed harder.

@mrflip
Copy link
Author

mrflip commented Dec 10, 2014

OK, rebased off master and code review points addressed. Please give this the same close going-over.


Since this change adds error checking to the .create and #push methods, I had to first fix a problem where the existing tests were recordings of "unauthorized error" responses

The tests in push took the image tianon/true and did a build with no changes -- giving an image named both tianon/true and <USERNAME>/true. Since tianon/true came first in the repo tags, the push specs tried to push to tianon/true, and so error.

My solution (in the first commit) is to make the created image non-identical to tianon/true, by adding a gratuitous environment variable to the build script. That leaves the image at hand named only <USERNAME>/true, and so it sails off to your actual account as it should.

Similarly, the test in spec/vcr/Docker_Image/_push/when_there_are_no_credentials/still_pushes.yml was not actually pushing to the local registry. I could not get the local registry to work at all (http vs https) and so what's there is a dummied-up response, but at least it's to the correct request.

Two other minor things I'll drop into line comments.

@mrflip mrflip force-pushed the response_blocks_for_push_and_pull branch 3 times, most recently from 25f3abb to ddc7621 Compare December 10, 2014 06:20
…n error if unauthorized

* A block supplied to Image#push or Image.create will be called with
  - the parsed hash of each json message streamed back from the server,
  - the image (push) or the creation hash (create)
* create and push now raise errors if an error response is returned from the server.
  - Pushing an image that is in progress of pushing does _not_ cause an error.
@mrflip mrflip force-pushed the response_blocks_for_push_and_pull branch from ddc7621 to c1d46ae Compare December 10, 2014 06:40
# Accept a prefix match if no :tag given; demand exact match otherwise
unless info['RepoTags'].any?{|rt| rt =~ /^#{repo_tag}(:[\w\.\-]*)?$/ }
raise(ArgumentError, "Not named '#{repo_tag}': #{info['RepoTags']}")
end
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't supply a :tag part (meaning "all tags for that root", not ":latest"), the optional regexp capture becomes a prefix match. If I do supply a :tag part, the optional capture group is irrelevant, because none of the repo tags have two post-slash :s.

I would prefer something less magical, but anything straightforward I could come up with got complicated fast for dealing with things like localhost:5000/alice:chapeau and localhost:5000.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants