Installed agents gets Unhealthy on setting invalid url as default under Fleet Server hosts and even on setting back to valid, agents remains Unhealthy.

[amolnater-qasource]

Kibana version: 8.6 BC6 kibana cloud environment

Host OS and Browser version: All, All

Build details:

VERSION: 8.6.0 BC6 Kibana cloud environment
BUILD: 58740
COMMIT: f329a77595950244361736dff7208a810299fd69

Preconditions:

1. 8.6 BC6 kibana cloud environment should be available.
2. Windows, Mac and linux agents should be installed with policy having System integration.

Steps to reproduce:

1. Navigate to Fleet>Settings tab.
2. Add invalid url under Fleet Server hosts url: https://10.10.10.10:8220.
3. Observe agents goes to Unhealthy state.
4. Now set valid url(hosted fleet server url) back to default.
5. Observe agents remain UNHEALTHY.
6. Restart agent> observe agent still remains Unhealthy.

Logs:
[Same Version]elastic-agent-diagnostics-2022-12-12T11-50-44Z-00.zip

Screenshot:

Agents.-.Fleet.-.Elastic.-.Google.Chrome.2022-12-12.17-14-13.mp4

Expected Result:
Installed agents gets Unhealthy on setting invalid url as default under Fleet Server hosts[Expected], however on setting back to valid, agents should get back to HEALTHY state.

[manishgupta-qasource]

Secondary review for this ticket is Done

[amolnater-qasource]

Similar issue at #257

[cmacknz]

The diagnostics here don't show the agent as unhealthy so I can't tell what the source of the problem is. Could you reproduce and attach agent diagnostics from when the agents were unhealthy?

[cmacknz]

There was a bug similar to this that should have been resolved in #1329. It looked like files involved were unmodified as part of the V2 merge.

[amolnater-qasource]

Hi @cmacknz @michalpristas
We have revalidated this issue on latest 8.6 Snapshot kibana cloud-staging environment and found it still reproducible.

Observations:

• Installed agents gets Unhealthy on setting invalid url as default under Fleet Server hosts and even on setting back to valid, agents remains Unhealthy.

Build details:
BUILD: 58830
COMMIT: 6a5d6d96a534be75fc58acda8f89f2610309d7ff
Artifact: https://snapshots.elastic.co/8.6.0-f6d7d537/downloads/beats/elastic-agent/elastic-agent-8.6.0-SNAPSHOT-windows-x86_64.zip

Screenshots:

Logs:
[Output]elastic-agent-diagnostics-2022-12-27T12-01-20Z-00.zip

Hence we are reopening this issue.

Thanks

[michalpristas]

this is in fact not related, see different last checkin message
previously it was failed to connect to 10.10.10.10.:8220 now it's 1 or more component/unit in failed state

i created an issue for what we're seeing here: elastic/beats#34137
closing in favor of mentioned issue

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.6 BC10 Kibana cloud environment and found it fixed now.

• Installed agents gets Unhealthy on setting invalid url as default under Fleet Server hosts[Expected], however on setting back to valid, agents gets back to HEALTHY state.

Build details:
BUILD: 58852
COMMIT: d3a625ef4a6e611a5b3233a1ce5cbe8ef429eb47

Screen Recording:

Agents.-.Fleet.-.Elastic.-.Google.Chrome.2023-01-09.13-58-42.mp4

Hence, marking this issue as QA:Validated.

Thanks

[ghost]

Bug Conversion:

We have updated 01 testcase for this scenario in our fleet test suite at:

• https://elastic.testrail.io/index.php?/cases/view/168386

Thanks!