[ASM] Fix IAST weak random vulnerability error #6432

NachoEchevarria · 2024-12-12T15:41:48Z

Summary of changes

When running an app out of a request context, we were not closing the scope after creating a vulnerability span in the case of an IAST weak random vulnerability. That would make the tracer to stop sending spans and would store them forever, taking an increasing amount of memory.

Basically, in the method close_span of the class traceContext, the variable _openSpans would never reach 0, so the array _spans would never be empty and would keep storing spans forever.

The problem is that, in the System.Random aspect, we were not closing the scope/span, which we were actually doing in other aspects such as the weak hash aspect.

A little refactor has been done. Now, by default, we close an IAST vulnerability span right after creation in non request scenarios. In the call target instrumentations, we already handle the closing of the span, so the IAST instrumentations that can be launched in non request scenarios will keep the span closing that was already implemented.

A filter has been added to avoid instrumenting the Grpc.Net.Client assembly. We were reporting a vulnerability inside this library that was adding noise.

In the deduplication tests, a new random.Next call has been added. Before the implemented changes, this test would receive no spans from the tracer after adding the Random.Next call.

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging · 2024-12-12T16:10:42Z

Datadog Report

Branch report: nacho/FixIastNonRequestSpans
Commit report: 63b18b3
Test service: dd-trace-dotnet

❌ 3 Failed (0 Known Flaky), 462675 Passed, 3653 Skipped, 32h 59m 32.94s Total Time

❌ Failed Tests (3)

StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details
StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details
StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details

andrewlock · 2024-12-12T16:15:50Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (68ms)  : 65, 71
     .   : milestone, 68,
    master - mean (68ms)  : 66, 71
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (975ms)  : 952, 997
     .   : milestone, 975,
    master - mean (976ms)  : 953, 998
     .   : milestone, 976,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (107ms)  : 105, 109
     .   : milestone, 107,
    master - mean (107ms)  : 104, 110
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (674ms)  : 658, 690
     .   : milestone, 674,
    master - mean (679ms)  : 662, 695
     .   : milestone, 679,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 92
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (626ms)  : 609, 643
     .   : milestone, 626,
    master - mean (635ms)  : 619, 651
     .   : milestone, 635,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (196ms)  : 191, 201
     .   : milestone, 196,
    master - mean (193ms)  : 188, 198
     .   : milestone, 193,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (1,102ms)  : 1067, 1138
     .   : milestone, 1102,
    master - mean (1,096ms)  : 1070, 1122
     .   : milestone, 1096,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (279ms)  : 274, 284
     .   : milestone, 279,
    master - mean (279ms)  : 275, 284
     .   : milestone, 279,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (866ms)  : 830, 902
     .   : milestone, 866,
    master - mean (871ms)  : 845, 896
     .   : milestone, 871,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6432) - mean (267ms)  : 262, 271
     .   : milestone, 267,
    master - mean (266ms)  : 263, 269
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6432) - mean (844ms)  : 809, 879
     .   : milestone, 844,
    master - mean (851ms)  : 818, 884
     .   : milestone, 851,

andrewlock · 2024-12-12T19:11:08Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6432) (11.265M)   : 0, 11264987
    master (11.173M)   : 0, 11172862
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6432) (7.242M)   : 0, 7242243
    master (7.214M)   : 0, 7214402
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.560M)   : 0, 7559850

    section Manual
    master (11.194M)   : 0, 11193707

    section Manual + Automatic
    This PR (6432) (6.780M)   : 0, 6780255
    master (6.718M)   : 0, 6717525

    section DD_TRACE_ENABLED=0
    master (10.296M)   : 0, 10295976

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6432) (9.603M)   : 0, 9602810
    master (9.776M)   : 0, 9776047
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6432) (6.293M)   : 0, 6293265
    master (6.425M)   : 0, 6425271

    section Trace stats
    master (6.758M)   : 0, 6757513

    section Manual
    master (9.661M)   : 0, 9660549

    section Manual + Automatic
    This PR (6432) (5.928M)   : 0, 5927617
    master (6.039M)   : 0, 6038872

    section DD_TRACE_ENABLED=0
    master (8.861M)   : 0, 8860501

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6432) (10.098M)   : 0, 10097920
    master (9.399M)   : 0, 9399211
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6432) (6.468M)   : 0, 6467904
    master (5.619M)   : 0, 5618747
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (6.816M)   : 0, 6816490

    section Manual
    master (9.559M)   : 0, 9558747

    section Manual + Automatic
    This PR (6432) (5.938M)   : 0, 5938007
    master (6.022M)   : 0, 6022498

    section DD_TRACE_ENABLED=0
    master (9.201M)   : 0, 9201064

andrewlock · 2024-12-12T23:00:05Z

Benchmarks Report for tracer 🐌

Benchmarks for #6432 compared to master:

2 benchmarks are slower, with geometric mean 1.143
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.02μs	43.7ns	287ns	0.0157	0.00784	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.1μs	46.8ns	181ns	0.0201	0.0101	0	5.8 KB
master	`StartStopWithChild`	net472	16.4μs	38.9ns	151ns	1.04	0.303	0.0983	6.2 KB
#6432	`StartStopWithChild`	net6.0	8.16μs	41.5ns	245ns	0.0247	0.00824	0	5.6 KB
#6432	`StartStopWithChild`	netcoreapp3.1	10.2μs	52.6ns	263ns	0.0205	0.0103	0	5.8 KB
#6432	`StartStopWithChild`	net472	16.5μs	45.7ns	177ns	1.04	0.305	0.0964	6.19 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	484μs	785ns	3.04μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	666μs	1.74μs	6.75μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	847μs	757ns	2.62μs	0.422	3.3 KB
#6432	`WriteAndFlushEnrichedTraces`	net6.0	481μs	320ns	1.2μs	0	2.7 KB
#6432	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	646μs	778ns	2.81μs	0	2.7 KB
#6432	`WriteAndFlushEnrichedTraces`	net472	848μs	552ns	2.06μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	149μs	869ns	8.24μs	0.153	14.47 KB
master	`SendRequest`	netcoreapp3.1	169μs	976ns	8.4μs	0.17	17.27 KB
master	`SendRequest`	net472	0.000589ns	0.000489ns	0.00183ns	0	0 b
#6432	`SendRequest`	net6.0	150μs	875ns	7.77μs	0.14	14.47 KB
#6432	`SendRequest`	netcoreapp3.1	167μs	970ns	8.4μs	0.158	17.27 KB
#6432	`SendRequest`	net472	0.00092ns	0.000378ns	0.00146ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	588μs	3.25μs	19.2μs	0.561	0	0	41.79 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	653μs	2.92μs	16μs	0.331	0	0	41.87 KB
master	`WriteAndFlushEnrichedTraces`	net472	838μs	4.22μs	17.9μs	8.52	2.44	0.406	53.26 KB
#6432	`WriteAndFlushEnrichedTraces`	net6.0	550μs	1.67μs	6.26μs	0.566	0	0	41.74 KB
#6432	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	663μs	3.27μs	17μs	0.324	0	0	41.95 KB
#6432	`WriteAndFlushEnrichedTraces`	net472	841μs	3.92μs	14.7μs	8.5	2.55	0.425	53.33 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.3μs	1.02ns	3.94ns	0.0146	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.81μs	2.54ns	9.85ns	0.0136	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.14μs	1.84ns	7.12ns	0.157	0.00107	987 B
#6432	`ExecuteNonQuery`	net6.0	1.26μs	1.57ns	5.87ns	0.014	0	1.02 KB
#6432	`ExecuteNonQuery`	netcoreapp3.1	1.78μs	2.46ns	9.51ns	0.0133	0	1.02 KB
#6432	`ExecuteNonQuery`	net472	2.14μs	2.51ns	9.72ns	0.157	0.00105	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.33μs	0.64ns	2.48ns	0.0134	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.55μs	1.39ns	5.2ns	0.013	976 B
master	`CallElasticsearch`	net472	2.59μs	2.4ns	9.28ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.36μs	0.422ns	1.52ns	0.0128	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.64μs	1.02ns	3.93ns	0.014	1.02 KB
master	`CallElasticsearchAsync`	net472	2.58μs	2.13ns	8.26ns	0.166	1.05 KB
#6432	`CallElasticsearch`	net6.0	1.32μs	0.867ns	3.24ns	0.0132	976 B
#6432	`CallElasticsearch`	netcoreapp3.1	1.52μs	0.691ns	2.59ns	0.0129	976 B
#6432	`CallElasticsearch`	net472	2.52μs	1.88ns	7.3ns	0.158	995 B
#6432	`CallElasticsearchAsync`	net6.0	1.37μs	0.679ns	2.63ns	0.0138	952 B
#6432	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	0.669ns	2.59ns	0.0134	1.02 KB
#6432	`CallElasticsearchAsync`	net472	2.59μs	1.66ns	6.22ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.38μs	0.494ns	1.91ns	0.0128	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.6μs	0.436ns	1.63ns	0.0128	952 B
master	`ExecuteAsync`	net472	1.82μs	0.371ns	1.39ns	0.145	915 B
#6432	`ExecuteAsync`	net6.0	1.32μs	0.816ns	3.16ns	0.0132	952 B
#6432	`ExecuteAsync`	netcoreapp3.1	1.65μs	0.732ns	2.74ns	0.0124	952 B
#6432	`ExecuteAsync`	net472	1.8μs	0.391ns	1.46ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.3μs	1.95ns	7.54ns	0.0323	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.33μs	1.59ns	5.97ns	0.0374	2.85 KB
master	`SendAsync`	net472	7.38μs	1.78ns	6.88ns	0.494	3.12 KB
#6432	`SendAsync`	net6.0	4.34μs	1.06ns	4.12ns	0.0327	2.31 KB
#6432	`SendAsync`	netcoreapp3.1	5.43μs	2.25ns	8.41ns	0.0381	2.85 KB
#6432	`SendAsync`	net472	7.3μs	1.88ns	6.79ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.48μs	0.72ns	2.69ns	0.0228	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.43μs	2.01ns	7.79ns	0.0219	1.64 KB
master	`EnrichedLog`	net472	2.8μs	1.47ns	5.68ns	0.249	1.57 KB
#6432	`EnrichedLog`	net6.0	1.59μs	1.39ns	5.2ns	0.0225	1.64 KB
#6432	`EnrichedLog`	netcoreapp3.1	2.22μs	4.29ns	16.6ns	0.0222	1.64 KB
#6432	`EnrichedLog`	net472	2.69μs	1.48ns	5.34ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	118μs	243ns	941ns	0.0591	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	121μs	171ns	662ns	0	0	4.28 KB
master	`EnrichedLog`	net472	154μs	81.5ns	294ns	0.684	0.228	4.46 KB
#6432	`EnrichedLog`	net6.0	119μs	244ns	943ns	0.0592	0	4.28 KB
#6432	`EnrichedLog`	netcoreapp3.1	122μs	164ns	612ns	0	0	4.28 KB
#6432	`EnrichedLog`	net472	153μs	172ns	668ns	0.686	0.229	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.02μs	1.06ns	4.1ns	0.0299	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.25μs	2.98ns	11.5ns	0.0296	2.2 KB
master	`EnrichedLog`	net472	4.89μs	1.4ns	5.42ns	0.32	2.02 KB
#6432	`EnrichedLog`	net6.0	2.84μs	1.27ns	4.91ns	0.0302	2.2 KB
#6432	`EnrichedLog`	netcoreapp3.1	4.25μs	1.57ns	6.07ns	0.0297	2.2 KB
#6432	`EnrichedLog`	net472	4.95μs	1.18ns	4.41ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6432

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0	1.118	1,310.72	1,465.73

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.31μs	0.911ns	3.53ns	0.0157	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.93μs	0.778ns	3.01ns	0.0154	1.14 KB
master	`SendReceive`	net472	2.21μs	0.887ns	3.44ns	0.183	1.16 KB
#6432	`SendReceive`	net6.0	1.47μs	0.799ns	3.09ns	0.0162	1.14 KB
#6432	`SendReceive`	netcoreapp3.1	1.85μs	1.05ns	4.08ns	0.0157	1.14 KB
#6432	`SendReceive`	net472	2.12μs	0.758ns	2.84ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.75μs	0.668ns	2.41ns	0.022	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.9μs	2.14ns	8.29ns	0.0216	1.65 KB
master	`EnrichedLog`	net472	4.42μs	3.49ns	13.5ns	0.322	2.04 KB
#6432	`EnrichedLog`	net6.0	2.79μs	1.44ns	5.57ns	0.0223	1.6 KB
#6432	`EnrichedLog`	netcoreapp3.1	3.96μs	1.66ns	6.43ns	0.0216	1.65 KB
#6432	`EnrichedLog`	net472	4.33μs	3.66ns	14.2ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	402ns	0.238ns	0.921ns	0.00805	576 B
master	`StartFinishSpan`	netcoreapp3.1	616ns	0.76ns	2.95ns	0.00768	576 B
master	`StartFinishSpan`	net472	631ns	0.366ns	1.42ns	0.0916	578 B
master	`StartFinishScope`	net6.0	556ns	0.355ns	1.38ns	0.00982	696 B
master	`StartFinishScope`	netcoreapp3.1	733ns	0.697ns	2.7ns	0.00919	696 B
master	`StartFinishScope`	net472	849ns	0.866ns	3.35ns	0.104	658 B
#6432	`StartFinishSpan`	net6.0	399ns	0.276ns	1.07ns	0.00803	576 B
#6432	`StartFinishSpan`	netcoreapp3.1	572ns	0.467ns	1.75ns	0.00767	576 B
#6432	`StartFinishSpan`	net472	687ns	0.571ns	2.21ns	0.0918	578 B
#6432	`StartFinishScope`	net6.0	545ns	0.327ns	1.27ns	0.00983	696 B
#6432	`StartFinishScope`	netcoreapp3.1	691ns	0.247ns	0.955ns	0.0093	696 B
#6432	`StartFinishScope`	net472	847ns	0.356ns	1.38ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6432

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.168	591.12	690.29

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	591ns	0.394ns	1.47ns	0.00976	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	885ns	0.35ns	1.31ns	0.00973	696 B
master	`RunOnMethodBegin`	net472	1.16μs	1.04ns	4.04ns	0.104	658 B
#6432	`RunOnMethodBegin`	net6.0	690ns	0.611ns	2.37ns	0.00975	696 B
#6432	`RunOnMethodBegin`	netcoreapp3.1	848ns	0.788ns	3.05ns	0.00968	696 B
#6432	`RunOnMethodBegin`	net472	1.22μs	0.727ns	2.52ns	0.104	658 B

…aDog/dd-trace-dotnet into nacho/FixIastNonRequestSpans

andrewlock · 2024-12-13T11:22:03Z

Benchmarks Report for appsec 🐌

Benchmarks for #6432 compared to master:

1 benchmarks are faster, with geometric mean 1.220
1 benchmarks have fewer allocations
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6432

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1	1.220	229.02	187.74

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	187μs	117ns	437ns	2.62	0	188.52 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	281μs	159ns	597ns	2.66	0	195.8 KB
master	`AllCycleSimpleBody`	net472	250μs	153ns	574ns	35.8	2	225.32 KB
master	`AllCycleMoreComplexBody`	net6.0	189μs	80.4ns	290ns	2.74	0	192.02 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	298μs	639ns	2.48μs	2.63	0	199.22 KB
master	`AllCycleMoreComplexBody`	net472	254μs	425ns	1.64μs	36.4	2.02	228.83 KB
master	`ObjectExtractorSimpleBody`	net6.0	138ns	0.184ns	0.69ns	0.00394	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	229ns	0.339ns	1.31ns	0.00366	0	272 B
master	`ObjectExtractorSimpleBody`	net472	163ns	0.264ns	1.02ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	2.89μs	1.4ns	5.22ns	0.0522	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.73μs	3.7ns	13.8ns	0.0508	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.6μs	2.19ns	8.18ns	0.603	0.00543	3.8 KB
#6432	`AllCycleSimpleBody`	net6.0	200μs	190ns	710ns	2.6	0	188.38 KB
#6432	`AllCycleSimpleBody`	netcoreapp3.1	304μs	105ns	406ns	2.58	0	195.66 KB
#6432	`AllCycleSimpleBody`	net472	269μs	121ns	451ns	35.7	2.02	225.16 KB
#6432	`AllCycleMoreComplexBody`	net6.0	208μs	125ns	469ns	2.71	0	191.88 KB
#6432	`AllCycleMoreComplexBody`	netcoreapp3.1	312μs	259ns	1μs	2.64	0	199.07 KB
#6432	`AllCycleMoreComplexBody`	net472	277μs	105ns	363ns	36.2	2.07	228.68 KB
#6432	`ObjectExtractorSimpleBody`	net6.0	140ns	0.155ns	0.599ns	0.00395	0	280 B
#6432	`ObjectExtractorSimpleBody`	netcoreapp3.1	188ns	0.35ns	1.31ns	0.00372	0	272 B
#6432	`ObjectExtractorSimpleBody`	net472	163ns	0.191ns	0.69ns	0.0446	0	281 B
#6432	`ObjectExtractorMoreComplexBody`	net6.0	2.9μs	1.91ns	7.13ns	0.0538	0	3.78 KB
#6432	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.8μs	2.52ns	9.41ns	0.0493	0	3.69 KB
#6432	`ObjectExtractorMoreComplexBody`	net472	3.6μs	2.6ns	9.74ns	0.602	0.00544	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	37.2μs	26.4ns	98.6ns	0.445	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.7μs	56.4ns	219ns	0.432	0	32.4 KB
master	`EncodeArgs`	net472	67.7μs	43.1ns	161ns	5.16	0.067	32.5 KB
master	`EncodeLegacyArgs`	net6.0	74.2μs	107ns	416ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	104μs	163ns	630ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	154μs	128ns	497ns	0.306	0	2.15 KB
#6432	`EncodeArgs`	net6.0	39.2μs	12.3ns	47.6ns	0.449	0	32.4 KB
#6432	`EncodeArgs`	netcoreapp3.1	54.6μs	18.5ns	71.5ns	0.435	0	32.4 KB
#6432	`EncodeArgs`	net472	66.9μs	43.2ns	167ns	5.15	0.0669	32.5 KB
#6432	`EncodeLegacyArgs`	net6.0	74.1μs	15ns	58.1ns	0	0	2.14 KB
#6432	`EncodeLegacyArgs`	netcoreapp3.1	106μs	267ns	1.04μs	0	0	2.14 KB
#6432	`EncodeLegacyArgs`	net472	153μs	118ns	458ns	0.307	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	181μs	84ns	291ns	0	2.44 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	192μs	249ns	964ns	0	2.39 KB
master	`RunWafRealisticBenchmark`	net472	209μs	180ns	696ns	0.309	2.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	119ns	461ns	0	1.47 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	129μs	162ns	583ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	56.6ns	219ns	0.209	1.49 KB
#6432	`RunWafRealisticBenchmark`	net6.0	183μs	279ns	1.08μs	0	2.44 KB
#6432	`RunWafRealisticBenchmark`	netcoreapp3.1	193μs	295ns	1.1μs	0	2.39 KB
#6432	`RunWafRealisticBenchmark`	net472	208μs	110ns	413ns	0.311	2.46 KB
#6432	`RunWafRealisticBenchmarkWithAttack`	net6.0	121μs	71.5ns	267ns	0	1.47 KB
#6432	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	129μs	96.6ns	348ns	0	1.46 KB
#6432	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	100ns	387ns	0.208	1.49 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6432

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	257.94 KB	265.46 KB	7.52 KB	2.92%

Fewer allocations 🎉 in #6432

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	61.66 KB	59.07 KB	-2.59 KB	-4.20%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	52.3μs	249ns	932ns	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	53.4μs	176ns	660ns	42.64 KB
master	`StringConcatBenchmark`	net472	36.4μs	80.4ns	290ns	61.66 KB
master	`StringConcatAspectBenchmark`	net6.0	310μs	1.09μs	3.92μs	257.94 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	333μs	1.74μs	9.37μs	253.05 KB
master	`StringConcatAspectBenchmark`	net472	284μs	6.2μs	59.5μs	278.53 KB
#6432	`StringConcatBenchmark`	net6.0	58.9μs	638ns	6.38μs	43.44 KB
#6432	`StringConcatBenchmark`	netcoreapp3.1	52.8μs	259ns	1.04μs	42.64 KB
#6432	`StringConcatBenchmark`	net472	37.1μs	80.4ns	301ns	59.07 KB
#6432	`StringConcatAspectBenchmark`	net6.0	296μs	4.62μs	43.8μs	265.46 KB
#6432	`StringConcatAspectBenchmark`	netcoreapp3.1	337μs	1.61μs	12.5μs	252.01 KB
#6432	`StringConcatAspectBenchmark`	net472	278μs	5.7μs	54.6μs	278.53 KB

github-actions · 2024-12-13T16:46:18Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

-    TraceId: Id_1,
-    SpanId: Id_4,
+    TraceId: Id_4,
+    SpanId: Id_5,
[...]
-    ParentId: Id_5,
+    ParentId: Id_6,

1 occurrences of :

-    TraceId: Id_1,
-    SpanId: Id_6,
+    TraceId: Id_7,
+    SpanId: Id_8,
[...]
-    ParentId: Id_7,
+    ParentId: Id_9,

1 occurrences of :

-    TraceId: Id_1,
-    SpanId: Id_8,
+    TraceId: Id_10,
+    SpanId: Id_11,
[...]
-    ParentId: Id_9,
+    ParentId: Id_12,

robertpi · 2024-12-16T15:52:47Z

...ClrProfiler/AutoInstrumentation/CryptographyAlgorithm/SymmetricAlgorithmIntegrationCommon.cs

@@ -17,7 +17,7 @@ internal class SymmetricAlgorithmIntegrationCommon
    internal const IntegrationId IntegrationId = Configuration.IntegrationId.SymmetricAlgorithm;
    private static readonly IDatadogLogger Log = DatadogLogging.GetLoggerFor(typeof(SymmetricAlgorithmIntegrationCommon));

-    internal static Scope? CreateScope<TTarget>(TTarget instance)
+    internal static Scope? CreateScope<TTarget>(TTarget instance, bool autoCloseScopeWhenSingleSpan = true)


I don't understand why this isn't used.

You are right. Thanks!

robertpi

Thank you!

NachoEchevarria · 2024-12-16T18:05:14Z

Thanks for your feedback and reviews!

NachoEchevarria added 2 commits December 12, 2024 14:26

New test

9762ce9

Dispose scope for single span weak randomness

bdbad40

NachoEchevarria changed the title ~~Nacho/fix iast non request spans~~ [ASM] Fix IAST random error Dec 12, 2024

github-actions bot added the area:asm label Dec 12, 2024

NachoEchevarria changed the title ~~[ASM] Fix IAST random error~~ [ASM] Fix IAST weak random vulnerability error Dec 12, 2024

Always close scope after creation

d6bf3f3

NachoEchevarria and others added 6 commits December 13, 2024 10:54

Merge branch 'master' into nacho/FixIastNonRequestSpans

e1b664e

Move dispose to random aspect

ab8c4da

Merge branch 'nacho/FixIastNonRequestSpans' of https://github.com/Dat…

d0c1bf5

…aDog/dd-trace-dotnet into nacho/FixIastNonRequestSpans

undo

2f563fc

remove dispose from AddVulnerabilityAsSingleSpan

87af3ba

Remove debug

c945086

NachoEchevarria and others added 5 commits December 13, 2024 12:46

Auto close span by default

561ea9d

exclude auto closing for command injection

554e993

Undo command injection spanClose

0414d16

Merge branch 'master' into nacho/FixIastNonRequestSpans

6a3b4da

Update snapshot

f9c0408

NachoEchevarria added 2 commits December 16, 2024 09:22

Fix grpc snapshot

538ad1c

Disable grpc callsite instrumentation

5421e8e

NachoEchevarria marked this pull request as ready for review December 16, 2024 11:28

NachoEchevarria requested review from a team as code owners December 16, 2024 11:28

robertpi reviewed Dec 16, 2024

View reviewed changes

Add missing parameter

63b18b3

robertpi approved these changes Dec 16, 2024

View reviewed changes

kevingosse approved these changes Dec 16, 2024

View reviewed changes

NachoEchevarria merged commit 7544783 into master Dec 16, 2024
105 of 110 checks passed

NachoEchevarria deleted the nacho/FixIastNonRequestSpans branch December 16, 2024 18:05

github-actions bot added this to the vNext-v3 milestone Dec 16, 2024

andrewlock added type:bug area:asm-iast labels Dec 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Fix IAST weak random vulnerability error #6432

[ASM] Fix IAST weak random vulnerability error #6432

NachoEchevarria commented Dec 12, 2024 •

edited

Loading

datadog-ddstaging bot commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6432

Raw results

Raw results

Raw results

Slower ⚠️ in #6432

Raw results

andrewlock commented Dec 13, 2024

Faster 🎉 in #6432

Raw results

Raw results

Raw results

More allocations ⚠️ in #6432

Fewer allocations 🎉 in #6432

Raw results

github-actions bot commented Dec 13, 2024 •

edited

Loading

robertpi Dec 16, 2024

NachoEchevarria Dec 16, 2024

robertpi left a comment

NachoEchevarria commented Dec 16, 2024

[ASM] Fix IAST weak random vulnerability error #6432

[ASM] Fix IAST weak random vulnerability error #6432

Conversation

NachoEchevarria commented Dec 12, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Dec 12, 2024 • edited Loading

Datadog Report

❌ Failed Tests (3)

andrewlock commented Dec 12, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Dec 12, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Dec 12, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6432

Raw results

Raw results

Raw results

Slower ⚠️ in #6432

Raw results

andrewlock commented Dec 13, 2024

Benchmarks Report for appsec 🐌

Benchmark details

Faster 🎉 in #6432

Raw results

Raw results

Raw results

More allocations ⚠️ in #6432

Fewer allocations 🎉 in #6432

Raw results

github-actions bot commented Dec 13, 2024 • edited Loading

Snapshots difference summary

robertpi Dec 16, 2024

Choose a reason for hiding this comment

NachoEchevarria Dec 16, 2024

Choose a reason for hiding this comment

robertpi left a comment

Choose a reason for hiding this comment

NachoEchevarria commented Dec 16, 2024

NachoEchevarria commented Dec 12, 2024 •

edited

Loading

datadog-ddstaging bot commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

andrewlock commented Dec 12, 2024 •

edited

Loading

github-actions bot commented Dec 13, 2024 •

edited

Loading