Add tool for generating licenses for 3rd party deps #1689
Conversation
| for _, dep := range deps.List { | ||
| counter := len(licenses) | ||
| for _, v := range licenses { | ||
| bytes, err := ioutil.ReadFile(filepath.Join(*dir, "vendor", dep.Name, v)) |
There was a problem hiding this comment.
I don't think we can count on the vendor directory since the default for go modules is to not use it. Looks like the default src cache is $GOPATH/pkg/mod: https://golang.org/cmd/go/#hdr-GOPATH_and_Modules
There was a problem hiding this comment.
Yeah, we can't count on it out of the box. But I'm thinking about using go mod vendor to collect all deps into vendor folder. In this case we can rely on content of vendor dir.
|
Can you clarify what the overall plan is to integrate this? I didn't see much more detail in the issue #1664. Also, how will this handle |
hack/licenser/main.go
Outdated
| flag.Parse() | ||
| log.SetFlags(log.LstdFlags | log.Lshortfile) | ||
|
|
||
| deps, err := loadFile(filepath.Join(*dir, *depFile)) |
There was a problem hiding this comment.
I wonder if parsing the output of go list -deps -test -json [1] might be better than parsing go.mod itself because the former is less likely to change with newer Go versions.
[1] probably needs to be tweaked a bit to get the desired output
There was a problem hiding this comment.
I tried to get output from go list but wasn't been able to get something useful from it. Instead I decided to use modules.txt file which is generated when go mod vendor is executed. It will produce easy to parse text file like:
# github.com/davecgh/go-spew v1.1.1
github.com/davecgh/go-spew/spew
# github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c
github.com/docker/spdystream
github.com/docker/spdystream/spdy
....
There was a problem hiding this comment.
I had some luck parsing the output of go list and running licence-detector. Happy to collaborate on this if you are up for it.
|
@anyasabo At the moment, I'm thinking that it can be similar to the tool for generating release notes https://github.com/elastic/cloud-on-k8s/blob/master/hack/release_notes.go. Like release manager or someone else before release will run it to generate
I think it should handle it pretty well. If I understand https://github.com/golang/go/blob/6f7542e4cb6914e8b8d888f16d440feeb8e631d2/src/cmd/go/internal/modcmd/vendor.go#L90 correctly, then |
I didn't know what our legal requirements were. Do we just need to have a notice file only for tagged releases or do we need them for every commit on master?
👍 Yep I think by parsing |
There was a problem hiding this comment.
Overall it LGTM except for a couple of nits. I do have some reservations about using go mod vendor because it is primarily a command available for backward compatibility reasons and generating a vendor directory just for this purpose feels a bit odd to me. That being said, I do not have a good alternative to suggest either so we can probably go with this for now and revisit it later.
| {{range $i,$v := .}} | ||
| Dependency: {{ $v.Name }} | ||
| Version: {{ $v.Version }} | ||
|
|
There was a problem hiding this comment.
Say we have 10 dependencies that are Apache 2 licensed. Aren't we repeating the full text of the Apache licence 10 times in the notice file in that case? Is that the expected behaviour?
|
|
||
| func checkForLicense(deps *Dependencies) int { | ||
| var issues []string | ||
| licenses := []string{"LICENSE", "LICENSE.txt", "LICENCE"} // Used to keep all possible names of license files |
There was a problem hiding this comment.
I wonder if using a library like https://github.com/src-d/go-license-detector might be better here so that we don't have to maintain our own list of different spellings and typos.
|
Closed in favor of #1764 |
This PR adds tool to generate
NOTICE.txtfile with licenses of 3rd party libraries. Result will be pretty similar to https://github.com/elastic/beats/blob/master/NOTICE.txtTool parses
go.modand checks dependencies invendorfolder for license file.