feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` #2091

iambriccardo · 2023-05-03T12:25:53Z

This PR adds a new function to sentry-relay called run_dynamic_sampling which will run dynamic sampling on a two SamplingConfigs.

iambriccardo · 2023-05-04T06:16:30Z

relay-cabi/src/processing.rs

+    let sampling_config = serde_json::from_str::<SamplingConfig>((*sampling_config).as_str())?;
+    let root_sampling_config =
+        serde_json::from_str::<SamplingConfig>((*root_sampling_config).as_str())?;


I have tried to use the ? operator for curiosity and I don't understand why the compiler allows me to return a Result when the type is RelayStr.

jan-auer · 2023-05-04T08:50:29Z

CHANGELOG.md

@@ -13,6 +13,7 @@
 - Scrub sensitive keys (`passwd`, `token`, ...) in Replay recording data. ([#2034](https://github.com/getsentry/relay/pull/2034))
 - Add document_uri to csp filter. ([#2059](https://github.com/getsentry/relay/pull/2059))
 - Store `geo.subdivision` of the end user location. ([#2058](https://github.com/getsentry/relay/pull/2058))
+- Add new FFI function for running dynamic sampling. ([#2091](https://github.com/getsentry/relay/pull/2091))


Please move this to py/CHANGELOG.md

jan-auer · 2023-05-04T08:51:41Z

py/sentry_relay/processing.py

@@ -262,3 +263,25 @@ def validate_project_config(config, strict: bool):
    error = decode_str(raw_error, free=True)
    if error:
        raise ValueError(error)
+
+
+def run_dynamic_sampling(sampling_config, root_sampling_config, dsc, event):


Should we take an entire envelope as interface, instead? In Relay's pipeline, DS also reads information from other envelope headers.

I wanted to keep it stupid simple for now, in case we want, we can extend it to have an envelope as well.

jan-auer · 2023-05-04T08:54:31Z

py/sentry_relay/processing.py

+    return json.loads(
+        decode_str(
+            rustcall(
+                lib.run_dynamic_sampling,
+                encode_str(sampling_config),
+                encode_str(root_sampling_config),
+                encode_str(dsc),
+                encode_str(event),
+            )
+        )
+    )


Please add free=True to decode_str, since the owned string returned by FFI needs to be released. You can make this more readable like so:

Suggested change

return json.loads(

decode_str(

rustcall(

lib.run_dynamic_sampling,

encode_str(sampling_config),

encode_str(root_sampling_config),

encode_str(dsc),

encode_str(event),

)

)

)

result_json = rustcall(

lib.run_dynamic_sampling,

encode_str(sampling_config),

encode_str(root_sampling_config),

encode_str(dsc),

encode_str(event),

)

return json.loads(decode_str(result_json, free=True))

jan-auer · 2023-05-04T08:57:07Z

py/tests/test_processing.py

@@ -283,3 +283,268 @@ def test_validate_project_config():
    with pytest.raises(ValueError) as e:
        sentry_relay.validate_project_config(json.dumps(config), strict=True)
    assert str(e.value) == 'json atom at path ".foobar" is missing from rhs'
+
+
+def test_run_dynamic_sampling_with_valid_params_and_match():


These tests are quite elaborate and will have to change every time we change something about sampling internals, particularly because they check the contents of the returned rules. Should we instead make more general checks, such as assert "merged_sampling_configs" in result and leave the rest to the DS unit tests in Rust?

That's a very good idea yes!

jan-auer · 2023-05-04T08:58:09Z

relay-cabi/src/processing.rs

 use std::cmp::Ordering;
 use std::ffi::CStr;
 use std::os::raw::c_char;
 use std::slice;

+use crate::core::{RelayBuf, RelayStr};


Please order imports according to the style guide. This concerns the chrono and crate imports.

jan-auer · 2023-05-04T09:00:48Z

relay-cabi/src/processing.rs

+        serde_json::from_str::<SamplingConfig>((*root_sampling_config).as_str())?;
+    // We can optionally accept a dsc and event.
+    let dsc = serde_json::from_str::<DynamicSamplingContext>((*dsc).as_str());
+    let event = serde_json::from_str::<EphemeralEvent>((*event).as_str());


Instead of parsing some synthetic EphemeralEvent, you should just load an actual Annotated<Event> directly:

Suggested change

let event = serde_json::from_str::<EphemeralEvent>((*event).as_str());

let event = Annotated::<Event>::from_json((*event).as_str())?;

let event = event.value().ok_or(<some error>)?;

jan-auer · 2023-05-04T09:01:52Z

relay-cabi/src/processing.rs

+    sampling_config: *const RelayStr,
+    root_sampling_config: *const RelayStr,
+    dsc: *const RelayStr,
+    event: *const RelayStr,


You can pass all of these as references and then don't have to worry about dereferencing the pointer below. This is something we've not done consistently in the codebase, but we should move towards:

Suggested change

sampling_config: *const RelayStr,

root_sampling_config: *const RelayStr,

dsc: *const RelayStr,

event: *const RelayStr,

sampling_config: &RelayStr,

root_sampling_config: &RelayStr,

dsc: &RelayStr,

event: &RelayStr,

jan-auer · 2023-05-08T08:03:15Z

relay-cabi/src/processing.rs

+use anyhow::Context;
+use chrono::Utc;


Please move these two imports to the next import group (above once_cell).

* master: (178 commits) ref(crons): Allow `_` in monitor slugs (#2100) fix(span-metrics): Scrub subdomains in metric tags (#2103) feat(normalization): Scrub span description URLs (#2095) feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` (#2091) ref(dynamic-config): Remove transaction metrics allowlist from Sentry (#2092) feat(protocol): Support old effective-directory format (#2048) ref(spans): Add more tags to span metrics (#2090) fix(server): Support HTTP half-close connections (#2089) ref(spans): Use transactions namespace with `span` prefix (#2087) ref(spans): Update span metrics feature flag name (#2086) feat(metrics): Extract more metrics from spans (#2080) build: Bump tempfile to remove flagged dependency (#2085) feat(filter): Add document_uri csp filter (#2059) build: Use jemalloc on Linux (#2084) ci(gha): add debugging for "Couldn'\''t write tracker file" issue (#2081) feat(crons): Pass-through minimal `sdk` field (#2073) fix(quota): Parse large limits (#2079) instr(server): More details on unexpected envelope drop (#2077) fix(ci): Skip flaky integration test (#2076) feat(dynamic-sampling): Add support for `trace.replay_id` in matching rules (#2070) ...

iambriccardo added 2 commits May 3, 2023 14:25

feat(dynamic-sampling): Add possibility to run dynamic sampling from

741f585

Fix issues

30431ed

iambriccardo changed the title ~~feat(dynamic-sampling): Add possibility to run dynamic sampling from~~ feat(dynamic-sampling): Add possibility to run dynamic sampling from sentry-relay May 3, 2023

iambriccardo added 10 commits May 3, 2023 14:35

Remove print

d5c6b17

Fix FFI

43cd643

Improve test

f918dd9

Add tests

5bbd706

Add tests

ca1d4ee

Fix angry clippy harassing me

00ec200

Fix lint

13bee6d

Add changelog

5f67eb0

Add support for empty event and dsc

d4aaabe

Add support for empty event and dsc

7637113

iambriccardo requested a review from jjbayer May 4, 2023 06:14

iambriccardo marked this pull request as ready for review May 4, 2023 06:14

iambriccardo requested a review from a team May 4, 2023 06:14

iambriccardo commented May 4, 2023

View reviewed changes

jan-auer requested changes May 4, 2023

View reviewed changes

iambriccardo added 3 commits May 4, 2023 14:28

Fix PR comments

9f83dd0

Fix lint

5fe423b

Fix lint

dd785f2

iambriccardo requested a review from jan-auer May 5, 2023 08:31

jan-auer approved these changes May 8, 2023

View reviewed changes

Fix imports

772af0e

iambriccardo merged commit 6e5bab1 into master May 8, 2023

iambriccardo deleted the riccardo/feat/add-sampling-to-relay-server branch May 8, 2023 08:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` #2091

feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` #2091

iambriccardo commented May 3, 2023 •

edited

Loading

iambriccardo May 4, 2023

jan-auer May 4, 2023

jan-auer May 4, 2023

iambriccardo May 4, 2023

jan-auer May 4, 2023

jan-auer May 4, 2023

iambriccardo May 4, 2023

jan-auer May 4, 2023

jan-auer May 4, 2023

jan-auer May 4, 2023

jan-auer May 8, 2023

	let event = serde_json::from_str::<EphemeralEvent>((*event).as_str());
	let event = Annotated::<Event>::from_json((*event).as_str())?;
	let event = event.value().ok_or(<some error>)?;

feat(dynamic-sampling): Add possibility to run dynamic sampling from sentry-relay #2091

feat(dynamic-sampling): Add possibility to run dynamic sampling from sentry-relay #2091

Conversation

iambriccardo commented May 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` #2091

feat(dynamic-sampling): Add possibility to run dynamic sampling from `sentry-relay` #2091

iambriccardo commented May 3, 2023 •

edited

Loading