feat: add reuseconn linter

[atzoum]

https://github.com/atzoum/reuseconn

reuseconn is a linter that checks whether the HTTP response body is consumed and closed properly in a single function, so that the underlying TCP connection can be reused.

This linter is a fork of timakin/bodyclose with the additional goal to address the rule that is clearly described in the GoDoc of http.Response and requires that the body should be both read to completion and closed so that the underlying TCP connection can be successfully reused.

It is the caller's responsibility to close Body. The default HTTP client's Transport may not reuse HTTP/1.x "keep-alive" TCP connections if the Body is not read to completion and closed.

[boring-cyborg]

Hey, thank you for opening your first Pull Request !

[CLAassistant]

All committers have signed the CLA.

[ldez]

In order for a pull request adding a linter to be reviewed, the linter and the PR must follow some requirements.

Pull Request Description

• It must have a link to the linter repository.
• It must provide a short description of the linter.

Linter

• It must not be a duplicate of another linter or a rule of a linter. (the team will help to verify that)
• It must have a valid license (AGPL is not allowed) and the file must contain the required information by the license, ex: author, year, etc.
• The linter repository must have a CI and tests.
• It must use go/analysis.
• It must have a valid tag, ex: v1.0.0, v0.1.0.
• It must not contain init().
• It must not contain panic(), log.fatal(), os.exit(), or similar.
• It must not have false positives/negatives. (the team will help to verify that)
• It must have tests inside golangci-lint.

The Linter Tests Inside Golangci-lint

• They must have at least one std lib import.
• They must work with T=<name of the linter test file>.go make test_linters. (the team will help to verify that)

.golangci.reference.yml

• The linter must be added to the list of available linters (alphabetical case-insensitive order).
  • enable and disable options
• If the linter has a configuration, the exhaustive configuration of the linter must be added (alphabetical case-insensitive order)
  • The values must be different from the default ones.
  • The default values must be defined in a comment.
  • The option must have a short description.

Others Requirements

• The files (tests and linter) inside golangci-lint must have the same name as the linter.
• The .golangci.yml of golangci-lint itself must not be edited and the linter must not be added to this file.
• The linters must be sorted in the alphabetical order (case-insensitive) in the Manager.GetAllSupportedLinterConfigs(...) and .golangci.reference.yml.
• The load mode (WithLoadMode(...)):
  • if the linter doesn't use types: goanalysis.LoadModeSyntax
  • goanalysis.LoadModeTypesInfo required WithLoadForGoAnalysis() in the Manager.GetAllSupportedLinterConfigs(...)
• The version in WithSince(...) must be the next minor version (v1.X.0) of golangci-lint.

Recommendations

• The linter should not use SSA. (currently, SSA does not support generics)
• The linter repository should have a readme and linting.
• The linter should be published as a binary. (useful to diagnose bug origins)

---

The golangci-lint team will edit this comment to check the boxes before and during the review.

This checklist does not imply that we will accept the linter.

[Antonboom]

@atzoum, hello!

Cool, but why not contribute in bodyclose?
Due to the its description:

Analyzer: checks whether HTTP response body is closed **and a re-use of TCP connection is not blocked**.

[atzoum]

@atzoum, hello!

Cool, but why not contribute in bodyclose? Due to the its description:

Analyzer: checks whether HTTP response body is closed **and a re-use of TCP connection is not blocked**.

Hi @Antonboom! bodyclose is about closing the body, whereas reuseconn is about both consuming and closing the body and all that happening within a single function.

The reason why reuseconn linter enforces both operations to happen in a single function (separate than the function that performs the HTTP request), is to avoid potential branching logic that can be introduced between the two operations which could result in skipping to consume the body for some of the branches. E.g. it is typical for client code to only attempt to read the response body if the response code is interesting enough (e.g. 200) and skip consuming the body for all other response codes. But, it would be really complex and practically impossible for a linter to verify all such branches consistently through static analysis (or in other words, I am unable to conceive how this could happen!).

That being said, the reuseconn linter promotes the following pattern

func disposeResponseBody(resp *http.Response) {
	if resp != nil && resp.Body != nil {
		// if the body is already read in some scenarios, the below operation becomes a no-op
		_, _ = io.Copy(io.Discard, resp.Body) 
		_ = resp.Body.Close()
	}
}

func main() {
	resp, err := http.Get("http://example.com/")
	defer closeResponseBody(resp) // OK
	if err != nil {
		// handle error
	}
	if resp2.StatusCode == http.StatusOK {
		body, _ := ioutil.ReadAll(resp.Body)
	}
}

With respect to the possibility of extending the original bodyclose linter, indeed this was considered, however, such a change is not backwards compatible, in the sense that it would cause the bodyclose linter to start complaining for most of the code paths that it previously allowed. Thus, IMO such a radically different linting behavior deserves to have a separate linter.

cc @timakin

[atzoum]

Hi team, please let me know if there is anything need to be done from my side in order to move this pull request forward

[bombsimon]

I was also curious why this wasn't PRed upstream to bodyclose since they seem so very closely related. I think backwards compatibility could easily be solved with an opt-in flag (disable by default), is that not an option?

[atzoum]

@timakin do you see this as a possibility, merging the 2 linters into one?

[atzoum]

btw

1. it is not possible to add an external linter as plugin to golangci-lint if you are using an official executable (link) and
2. it is impossible to customize the golangci-lint-action with respect to the executable that it is running (issue).

If (2) was possible I could compile my own binary with my private linter bundled and use it without bothering you 😄

[ldez]

For me, the right way to move on this PR:

• you have to open a PR on bodyclose or at least an issue
• you have to fix your current PR (checklist and CI)

After I have an idea if there is no reaction from @timakin to handle the duplication problem.

[timakin]

I'm on the same side as @bombsimon .
It's sure verifying whether TCP connection is reused is a definitely good feature.
But as already mentioned by others, it looks just a little bit duplicated with my bodyclose.

I'll be appreciated it if @atzoum opened PR to bodyclose.
Or as an alternative, it's better to clarify both packages' roles.
bodyclose checks whether HTTP response body is closed, and your reuseconn should focus on only checking whether it was reused.

[atzoum]

Or as an alternative, it's better to clarify both packages' roles.
bodyclose checks whether HTTP response body is closed, and your reuseconn should focus on only checking whether it was reused.

Checking whether a connection can be reused entails making sure that the body is both consumed and closed. Thus, there will always be some overlapping between reuseconn & bodyclose linters.

I will try to prepare a pull request, however based on my comment here, it is not really straightforward to achieve this for all scenarios that bodyclose covers now, thus my linter was focusing only on enforcing a specific pattern, i.e. expecting both operations to happen in a single function.

@timakin if you have any better idea for implementing this please go ahead

[alexandear]

Please use io instead of deprecated ioutil.

[chrishalbert]

Should this be moved up into the existing block of indirect required modules?