Remove service datasets

[jalvz]

Motivation/summary

We need to remove service names from index names, as they need to manage exactly what Fleet expects for data streams to be created.

Checklist

• I have signed the Contributor License Agreement.
• I have updated CHANGELOG.asciidoc

I have considered changes for:

• documentation
• logging (add log lines, choose appropriate log selector, etc.)
• metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)
• automated tests (add tests for the code changes, all unit tests pass locally)
• telemetry
• Elasticsearch Service (https://cloud.elastic.co)
• Elastic Cloud Enterprise (https://www.elastic.co/products/ece)
• Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)

How to test these changes

Related issues

[codecov-io]

Codecov Report

Merging #4491 (8e4d215) into master (6062235) will increase coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4491      +/-   ##
==========================================
+ Coverage   75.91%   75.96%   +0.05%     
==========================================
  Files         161      161              
  Lines        9790     9783       -7     
==========================================
  Hits         7432     7432              
+ Misses       2358     2351       -7     

Impacted Files	Coverage Δ
datastreams/servicename.go	100.00% <ø> (ø)
model/error.go	98.58% <100.00%> (-0.01%)	⬇️
model/metricset.go	94.73% <100.00%> (-0.10%)	⬇️
model/profile.go	100.00% <100.00%> (ø)
model/span.go	93.10% <100.00%> (-0.08%)	⬇️
model/transaction.go	96.15% <100.00%> (-0.08%)	⬇️
...ack/apm-server/aggregation/txmetrics/aggregator.go	93.36% <0.00%> (ø)
processor/otel/consumer.go	93.95% <0.00%> (+0.44%)	⬆️
kibana/connecting_client.go	70.17% <0.00%> (+8.77%)	⬆️

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Branch indexing

• Start Time: 2020-12-08T02:50:09.963+0000

• Duration: 43 min 44 sec

Test stats 🧪

Test	Results
Failed	0
Passed	4621
Skipped	139
Total	4760

Steps errors

Expand to view the steps failures

Compress

• Took 0 min 0 sec . View more details on here
• Description: tar --exclude=coverage-files.tgz -czf coverage-files.tgz coverage

Compress

• Took 0 min 0 sec . View more details on here
• Description: tar --exclude=system-tests-linux-files.tgz -czf system-tests-linux-files.tgz system-tests

Test Sync

• Took 3 min 50 sec . View more details on here
• Description: ./.ci/scripts/sync.sh

[axw]

The main changes looks fine, but what's up with the pipeline removal? If it's not related to this change, can you please separate that into another PR?

[jalvz]

The pipeline removal is a diff artifact, and looks "re-added" here: apmpackage/apm/0.1.0/data_stream/profile_metrics/elasticsearch/ingest_pipeline/default.json. But it is the same with the name fixed.

The reason for the pipeline changes is that the profiles one had the wrong name (s/profiles-apm-0.1.0/metrics-apm.profiling-0.1.0) which I blamed on poor data stream naming, so I took the chance to rename the data streams as well.

I made package and code changes into separated commits, but happy to move one out to another PR if you prefer.

[axw]

The pipeline removal is a diff artifact, and looks "re-added" here: apmpackage/apm/0.1.0/data_stream/profile_metrics/elasticsearch/ingest_pipeline/default.json. But it is the same with the name fixed.

The reason for the pipeline changes is that the profiles one had the wrong name (s/profiles-apm-0.1.0/metrics-apm.profiling-0.1.0) which I blamed on poor data stream naming, so I took the chance to rename the data streams as well.

I made package and code changes into separated commits, but happy to move one out to another PR if you prefer.

I see. Since you're reusing the constant now, it makes sense to have them in the same PR. Thanks for the explanation.

[simitt]

Not sure this needs any manual testing/how to test this manually. I'll leave it there fore now @jalvz can you please leave a comment on what to test here if anything should be done, otherwise add the test_plan_skip label - thank you.

[jalvz]

Probably the test of this is covered already by some other issue. Basically, data streams should be created when data is ingested. It was not the case before this.