[Elastic-Agent] Use data subfolder as default for process logs

[michalpristas]

What does this PR do?

This PR changes default path for process logs from /var/log to {path.data}/logs

this will conflict with paths CLI PR will need to rebase then

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

[elasticmachine]

Pinging @elastic/ingest-management (Project:fleet)

[michalpristas]

Jenkins test this

[ruflin]

Will this have any effect on where the logs from the processes go? Would be nice if it does as currently it is hard to find the logs for metricbeat for example.

[michalpristas]

@ruflin yes this changes the location and logs will go to {where_agent_is_installed}/data/logs/{pipeline_name}

e.g: /Users/me/agent/data/logs/default

[ph]

Is there any reason why we do not use {where_agent_is_installed}/data/{pipeline_name}/logs_ ?

pipeline_name is the name of the output now.

This would ensure that a single place contains everything?

[michalpristas]

@ph i can change it like this. i did it in this way so we end up with data containing
download install logs
instead of
download install default something-pretty something-else

[ruflin]

pipeline = output name? If yes, sounds confusing to me that we talk in the config about outputs but here about pipline?

[michalpristas]

we call it still PipelineID internally in the code, but what it is it is just and output type out of config e.g default/monitoring/elasticsearch/

[ruflin]

Is it type or the name of the output? https://github.com/elastic/beats/blob/master/x-pack/elastic-agent/elastic-agent.yml#L10 The part that is here default or elasticsearch

[michalpristas]

in the config you posted it would be default. it is the same value you use when specifying output using use_output in streams.
my bad, sorry for confusion i need to slow down window switching

[ph]

Its is the output name that create an internal pipeline, we ccould do some renaming internally so it's more consistant. but I don't think its a priority now.

[ph]

@ph i can change it like this. i did it in this way so we end up with data containing
download install logs
instead of
download install default something-pretty something-else

I see your point and I think it's valid, @ruflin wdyt?

[ruflin]

On your last comment, no preference as long as the logs are easy accessible.

[ph]

@michalpristas let's keep it your way :)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request [Elastic-Agent] Use data subfolder as default for process logs #17960 updated]

• Start Time: 2020-04-29T14:24:29.840+0000

• Duration: 31 min 4 sec (1863935)

• Commit: f67e5ea