x/pkgsite: support symlinks or LICENSES directory

[oxzi]

On a project I am working on, I recently implemented the Free Software Foundation Europe's REUSE (repository) recommendations for licensing. The used licenses are the GNU GPL for code and CC0 for some other stuff, both listed as supported licenses.

In a nutshell, REUSE stores all licenses named as their SPDX identifier in a LICENSES directory. Furthermore, each file starts with a SPDX header.

Because GitHub does not handled this, I created a LICENSE symlink to the GPL file within the LICENSES directory. GitHub then at least detects the presence of a license. (Edit: GitHub only detects a LICENSE file exists, but does not follow the symlink. Thus, I removed the symlink.)

However, the pkg.go.dev page is currently not able to process this. Furthermore, the documentation is "not displayed due to license restrictions". Compared to this, the old GoDoc has no limitations in this regard. Sorry, this is annoying.

Would it be possible to adjust pkg.go.dev to

• handle a LICENSES directory,
• detect and follow symlinks, or (Edit: see edit above)
• to parse SPDX headers?

Thanks a lot for all the great work!

[jba]

Please see this post to understand why we check licenses and godoc.org doesn't.

For legal reasons, we cannot consider SPDX headers authoritative. We must verify the license text itself.

A LICENSES directory is a reasonable request. We'll look into it.

[oxzi]

Please see [this post](https://groups.google.com/g/golang-dev/c/mfiPCtJ1BGU/m/qtCrqlrEEwAJ) to understand why we check licenses and godoc.org doesn't.

Thanks for the background and clarification. For the record, the following link also works without having a Google account. https://groups.google.com/d/msg/golang-dev/mfiPCtJ1BGU/qtCrqlrEEwAJ

A `LICENSES` directory is a reasonable request. We'll look into it.

That would be very cool, thanks. Alternatively, could a license hint or link to such a file also be included in the go.mod file, for example?

[jba]

I don't think the go.mod file is the right place for that sort of information. We don't add information to go.mod that is obtainable elsewhere, or that is language-agnostic.

[oxzi]

Any updates? After closing down the good old godoc.org and redirecting to the new pkg.go.dev thingy, there is no more documentation for properly licensed code.

A promising alternative seems to be godocs.io. However, for my next release I will add a LICENSE file…

[bernhardreiter]

For legal reasons, we cannot consider SPDX headers authoritative. We must verify the license text itself.

The SPDX-headers list which license texts are to be verified in the first place.

E.g. there are many repositories, where the LICENCES file or COPYING file does not list all licenses used in the repository. So if you are believing this, from a legal point of view, you should also be able to believe the SPDX headers.

A LICENSES directory is a reasonable request. We'll look into it.

Yes, please make it reuse.software compatible. What would have to be done for this?

[bernhardreiter]

One current example of the problem is https://pkg.go.dev/github.com/csaf-poc/csaf_distribution/[email protected]/csaf .

[bernhardreiter]

A LICENSES directory is a reasonable request. We'll look into it.

Yes, please make it reuse.software compatible. What would have to be done for this?

According to https://pkg.go.dev/license-policy the detector https://github.com/google/licensecheck is used and here is the relevant issue: google/licensecheck#53

[jba]

We currently have no cycles to add the LICENSES directory. I would pursue other options.

[bernhardreiter]

We currently have no cycles to add the LICENSES directory.

Would you be willing to accept a well tested patch?

[jba]

Sorry, the coding isn't the hard part. This change would probably require the intervention of Google's legal department, and a re-evaluation of many of the existing modules.

[bernhardreiter]

This change would probably require the intervention of Google's legal department,

Reuse compatibility would not change the legal status the documentation is under, so why would the legal department be a stakeholder? The legal evaluation which documentation can be shown would stay the same.

[jba]

@bernhardreiter that sounds right, but since neither of us are lawyers and this change affects what license(s) apply to to a module, we would have to consult a lawyer to make sure.

In other words, your reasoning is absolutely sound and correct, but it is not legal reasoning, which is often neither.

In any case, it's the evaluation of a large corpus of modules to gauge the effect of this change that is the real time suck. We simply don't have the bandwidth for a change of this magnitude right now.

[bernhardreiter]

@jba I have a bit of working experience with lawyers, and my idea is: If we can technically show that the resulting modules from the software change that are acceptable are exactly the same, then there is no legal change and thus no need for legal review.

A sort of automatic test could make sure that the evaluation result for existing modules would be the same.

Otherwise I would see if the Reuse people have contacts to Google's lawyers to get some support there. In the end the result would be more more legal clarity through better marking and this is something that all people want and should make the legal folks happy as well.