TLS not working with go 1.15

[simitt]

Summary

Compiling e.g. APM Server with go 1.15 and trying to run it via elastic agent results in following error:

{"level":"error","timestamp":"2020-12-03T10:36:56.840+0100","logger":"centralmgmt.fleet","caller":"fleet/manager.go:187","message":"elastic-agent-client got error: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0\""}

The TLS related code is the same accross beats.

Related golang issues: https://github.com/golang/go/issues/39568 and https://github.com/golang/go/issues/40748

How to reproduce

• run go test ./pkg/client with go 1.15 leads to a timeout
  OR
• compile e.g. APM Server with go 1.15 and try running it under elastic agent (currently only possible with manual plumbing) results in above mentioned error

[ph]

@blakerouse Can you take a look at this, master is not on 1.15 yet, I think it's planned for 1.15. cc @urso interesting to know.

[urso]

go1.15 changes Certificate validation. @kvch is already looking into updating master to go1.15. The plan is to reenable host checking via Common Field if possible.

For agent or whoever generates the certificates, should use SANs. Even before go 1.15 comes

[ph]

@urso Can you confirm that 1.15 is planned in 7.12?

[urso]

@urso Can you confirm that 1.15 is planned in 7.12?

Actually we were thinking 7.11

[ph]

@urso Can we hold that one? it's close on the FF? cc @andresrc

[urso]

PR upgrading to go1.15: elastic/beats#22495

Looks like we still have a few issues. Checking the 1.15 release notes it looks like there are potentially a few other gotchas. This one is my favorite (time to scan/update code and dependencies?):

Creating a derived Context using a nil parent is now explicitly disallowed. Any attempt to do so with the WithValue, WithDeadline, or WithCancel functions will cause a panic.

I don't mind if we postpone this after until FF.

[kvch]

@urso I checked our code and dependencies for possible context-related issues, but I haven't found anything which would cause problems. I checked in our deps with go mod vendor && git add vendor locally, and it did not show any incorrect context usage.

[ph]

@urso @kvch If we can postpone the update and do it 7.12, I would appreciate it. I want to make sure we don't introduce issue like this with the basic communication channel.

[urso]

If we can postpone the update and do it 7.12, I would appreciate it. I want to make sure we don't introduce issue like this with the basic communication channel.

Yeah, I'm fine with that.

[michalpristas]

as for elastic agent this affects IPC and agent to fleet communication.
IPC fix is easy and already targeted in PR elastic/beats#22495
other use cases agent to fleet or beat to output is handled using that PR as well and as soon as it defaults to backward compatible way there's nothing we should do agent side.

we could do two things here:

• in case when we generate correct certificates for IPC we can switch to strict for this communication to save time.
• make sure endpoint is OK with cert where we dont use CN but DNSName @ferullo (this change https://github.com/elastic/beats/pull/22495/files#diff-805a3ff341d1a82ad8954a59543bd85f08d296d0791679711675746f737a48b9)

[ph]

I am +1 to move to strict for the IPC, we can create the PR and sync with @ferullo's team for merging it.

[ph]

Removed from iteration, with the workaround in go 1.15.7, we think its sufficient.

[ph]

fixed by elastic/beats#22495