Add basic checksum

[catsby]

This adds a basic checksum feature to go-getter. It works by checking a checksum supplied as a query parameter:

if err := gets.GetFile("local.jar", "https://someurl/demoapp.jar?checksum=58d6e8130308d32e197c5108edd4f56ddf1417408f743097c2e662df0f0b17c8"); err != nil {
    log.Fatalf("\nFatal: %s\n", err)
}

Alternatively, we could change checksum param to sha256, and add support for md5 et. al

[mitchellh]

This looks good! We're gonna need a couple things though:

• Unit tests
• Support for every other checksum type (md5, sha1, etc.). I know from experience implementing checksums with Packer that this will be requested very early.

[cbednarski]

Can we specify the checksum in some way that doesn't modify the URL, since that might have side effects on the system we're downloading from?

This might be easier to add if GetFile() accepts a struct like:

type GetFileOptions struct {
    Source string
    Target string
    HashType string
    HashValue string
}

We could also add auth to this struct later and not need to change the function signature.

[mitchellh]

@cbednarski We can't do that.

It is idiomatic "go-getter" to use the URL. The URL is never passed as-is to the underlying system. We parse magic query parameters out or URLs for various things (see "ref" with Git URLs). But, you do bring up a good point that we should remove it from the URL so that it isn't part of it anymore.

The reason we can't support extra struct configs is because systems like Otto and Terraform don't have any other configuration other than URL for imports. Query parameters are meant to encode all the config and so go-getter uses the URL in its entirety for configuration.

You can see how we mangle URLs in all sorts of ways for configuration on the Otto docs page as an example (same as Terraform): https://ottoproject.io/docs/appfile/dep-sources.html

[cbednarski]

It is idiomatic "go-getter" to use the URL. The URL is never passed as-is to the underlying system. ... Query parameters are meant to encode all the config and so go-getter uses the URL in its entirety for configuration.

Ah that makes sense; I see how that works from the Otto examples. Thanks for clarifying!

[avnik]

Specifying http://url/to/file#md5=hash looks more sane, than HTTP GET ?checksum=... parameter. And it common practice used by python.org for years

[mitchellh]

@avnik For the same reason as my response to @cbednarski above, I don't think it'll matter. Plus, it is more idiomatic of this library to use query parameters. It is our API, so to speak. :) We don't parse out hash params, although I agree that in practice in links this is idiomatic.

[mitchellh]

I made the changes to support all checksums plus added tests!

@catsby Feel free to use as a reference, since you just get started with the project I wanted to give you some guidance without having you blocked on this.