make upstream keepalive work for http

[ElvinEfendi]

What this PR does / why we need it:
When upstream-keepalive-connections is used we don't configure Nginx properly to actually make keepalive effective. Excerpt from docs: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive

For HTTP, the proxy_http_version directive should be set to “1.1” and the “Connection” header field should be cleared:

We do set proxy_http_version, but don't clear Connection header. This PR makes sure the Connection header is cleared when the client is not websocket client (a.k.a it is not setting HTTP Upgrade header).

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

[aledbf]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aledbf, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ElvinEfendi,aledbf]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment