Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tail sampling to require data streams #5952

Merged
merged 3 commits into from
Aug 18, 2021

Conversation

axw
Copy link
Member

@axw axw commented Aug 16, 2021

Motivation/summary

Tail-sampling now requires data streams to be enabled, and apm-server will error if tail-sampling is enabled and data streams are not. The code for creating a sampled traces data stream when running in standalone has been removed. The data stream is now expected to be created by installing the integration package.

Rename the sampled traces data stream from traces-sampled-* to traces-apm.sampled-*, to maintain a common "apm" prefix for all datasets. The Elasticsearch role used by apm-server in tests has been updated with access to traces-apm*, metrics-apm*, and logs-apm*; similar to elastic/elasticsearch#68749.

Checklist

How to test these changes

  1. Run apm-server with 50% tail-based sampling enabled and data streams enabled
  2. Send N transactions
  3. Wait for N/2 transactions to be indexed

Related issues

Closes #5125

Tail-sampling now requires data streams to be enabled,
and apm-server will error if tail-sampling is enabled
and data streams are not.

The code for creating a sampled traces data stream when
running in standalone has been removed. The data stream
is now expected to be created by installing the integration
package.

Rename the sampled traces data stream from `traces-sampled-*`
to `traces-apm.sampled-*`, to maintain a common "apm" prefix
for all datasets.

The Elasticsearch role used by apm-server in tests has been
updated with access to `traces-apm*`, `metrics-apm*`, and
`logs-apm*`.
@apmmachine
Copy link
Contributor

apmmachine commented Aug 16, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-08-18T08:51:21.726+0000

  • Duration: 41 min 1 sec

  • Commit: d45e02b

Test stats 🧪

Test Results
Failed 0
Passed 5927
Skipped 14
Total 5941

Trends 🧪

Image of Build Times

Image of Tests

@axw axw marked this pull request as ready for review August 16, 2021 13:02
@axw axw requested a review from a team August 16, 2021 13:02
changelogs/head.asciidoc Outdated Show resolved Hide resolved
@simitt simitt mentioned this pull request Aug 18, 2021
2 tasks
@axw axw merged commit 2c65c70 into elastic:master Aug 18, 2021
@axw axw deleted the tail-sampling-data-streams branch August 18, 2021 09:52
mergify bot pushed a commit that referenced this pull request Aug 18, 2021
* Update tail sampling to require data streams

Tail-sampling now requires data streams to be enabled,
and apm-server will error if tail-sampling is enabled
and data streams are not.

The code for creating a sampled traces data stream when
running in standalone has been removed. The data stream
is now expected to be created by installing the integration
package.

Rename the sampled traces data stream from `traces-sampled-*`
to `traces-apm.sampled-*`, to maintain a common "apm" prefix
for all datasets.

The Elasticsearch role used by apm-server in tests has been
updated with access to `traces-apm*`, `metrics-apm*`, and
`logs-apm*`.

* systemtest: fix TestTailSamplingUnlicensed

* Update changelog

(cherry picked from commit 2c65c70)

# Conflicts:
#	changelogs/head.asciidoc
axw added a commit that referenced this pull request Aug 18, 2021
…5964)

* Update tail sampling to require data streams (#5952)

* Update tail sampling to require data streams

Tail-sampling now requires data streams to be enabled,
and apm-server will error if tail-sampling is enabled
and data streams are not.

The code for creating a sampled traces data stream when
running in standalone has been removed. The data stream
is now expected to be created by installing the integration
package.

Rename the sampled traces data stream from `traces-sampled-*`
to `traces-apm.sampled-*`, to maintain a common "apm" prefix
for all datasets.

The Elasticsearch role used by apm-server in tests has been
updated with access to `traces-apm*`, `metrics-apm*`, and
`logs-apm*`.

* systemtest: fix TestTailSamplingUnlicensed

* Update changelog

(cherry picked from commit 2c65c70)

# Conflicts:
#	changelogs/head.asciidoc

* Delete head.asciidoc

Co-authored-by: Andrew Wilkins <[email protected]>
@stuartnelson3 stuartnelson3 self-assigned this Aug 31, 2021
@stuartnelson3
Copy link
Contributor

confirmed with bc3

@stuartnelson3 stuartnelson3 removed their assignment Aug 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[tail sampling] Review behaviour when Elasticsearch license checks fail
4 participants