Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simmy docs #1883

Merged
merged 38 commits into from
Jan 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
664e4bb
expose simmy apis
vany0114 Nov 11, 2023
3dd5703
add missing docs
vany0114 Nov 11, 2023
928d250
Merge branch 'main' into simmy-expose-apis
vany0114 Nov 16, 2023
6c1eb6f
WIP:
vany0114 Nov 19, 2023
202bddc
WIP:
vany0114 Nov 25, 2023
5331538
Merge branch 'main' into simmy-docs
vany0114 Nov 25, 2023
6db206f
add latency monkey docs
vany0114 Jan 7, 2024
a60a1e0
add latency docs
vany0114 Jan 7, 2024
4b0a1ac
add outcome docs
vany0114 Jan 8, 2024
6bcbd52
Merge branch 'main' into simmy-docs
vany0114 Jan 8, 2024
d3ddc4b
fixes docs styling issues
vany0114 Jan 8, 2024
11b69e1
more docs styling issues
vany0114 Jan 8, 2024
1aaf08a
Merge branch 'App-vNext:main' into simmy-docs
peter-csala Jan 15, 2024
ceb9992
Move code samples to snippets project
peter-csala Jan 15, 2024
966bb95
Fix spellcheck issues
peter-csala Jan 15, 2024
ebf3fa5
Format markdown tables
peter-csala Jan 15, 2024
d8adb49
Apply suggested changes
peter-csala Jan 15, 2024
ae445be
Fix a couple of docfx issues
peter-csala Jan 15, 2024
722bca1
Use unicode emojis to describe scenarios
peter-csala Jan 16, 2024
38b7ce8
Fix docfx link issue
peter-csala Jan 16, 2024
59b4cca
Fix build by using another overload of AddChaosXYZ
peter-csala Jan 16, 2024
f15cb64
Merge branch 'App-vNext:main' into simmy-docs
peter-csala Jan 16, 2024
f33a29f
Apply suggested changes
peter-csala Jan 16, 2024
fe9a973
Rename logo
peter-csala Jan 17, 2024
46b4fe5
Apply suggestions from code review
peter-csala Jan 17, 2024
edf0d9a
Update docs/chaos/index.md
peter-csala Jan 17, 2024
7269a66
Fix chaos strategy registration order
peter-csala Jan 17, 2024
fa7805d
Apply suggested changes to Result chaos
peter-csala Jan 17, 2024
c34305f
Use InternalServerError for status code
peter-csala Jan 17, 2024
e00a12d
Remove default options where it does not make sense
peter-csala Jan 17, 2024
93eed65
Fix spelling
peter-csala Jan 17, 2024
1bc64c3
Merge branch 'main' into simmy-docs
peter-csala Jan 18, 2024
1dcc052
Delete simmy logos
peter-csala Jan 18, 2024
5d78a62
Apply suggested changes
peter-csala Jan 18, 2024
cfbca1a
Replace Fault to FaultGenerator
peter-csala Jan 18, 2024
3239718
Update docs/chaos/index.md
peter-csala Jan 18, 2024
58b4eaf
Prefer chaos over monkey wherever possible
peter-csala Jan 18, 2024
a2f0986
Rearrange Table of Contents
peter-csala Jan 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ nuget
oss
pcl
parallelize
paas
pluralsight
pollydocs
pre
Expand All @@ -47,6 +48,7 @@ rethrow
rethrows
retryable
reusability
saas
sdk
serializers
silverlight
Expand All @@ -60,6 +62,7 @@ telemetrylistener
testability
timingpolicy
ui
unhandled
uwp
waitandretry
wpf
Expand Down
6 changes: 0 additions & 6 deletions docs/advanced/simmy.md

This file was deleted.

130 changes: 130 additions & 0 deletions docs/chaos/behavior.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Behavior chaos strategy

## About

- **Options**: [`BehaviorStrategyOptions`](xref:Polly.Simmy.Behavior.BehaviorStrategyOptions)
- **Extensions**: `AddChaosBehavior`
- **Strategy Type**: Proactive

---

The behavior chaos strategy is designed to inject custom behaviors into system operations right before such an operation is invoked. This strategy is flexible, allowing users to define specific behaviors such as altering the input, simulating resource exhaustion, putting the system in a given state before the actual operation is called, or other operational variations to simulate real-world scenarios.

## Usage

<!-- snippet: chaos-behavior-usage -->
```cs
// To use a custom function to generate the behavior to inject.
var optionsWithBehaviorGenerator = new BehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
Enabled = true,
InjectionRate = 0.05
};

// To get notifications when a behavior is injected
var optionsOnBehaviorInjected = new BehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
Enabled = true,
InjectionRate = 0.05,
OnBehaviorInjected = static args =>
{
Console.WriteLine("OnBehaviorInjected, Operation: {0}.", args.Context.OperationKey);
return default;
}
};

// Add a behavior strategy with a BehaviorStrategyOptions instance to the pipeline
new ResiliencePipelineBuilder().AddChaosBehavior(optionsWithBehaviorGenerator);
new ResiliencePipelineBuilder<HttpResponseMessage>().AddChaosBehavior(optionsOnBehaviorInjected);

// There are also a handy overload to inject the chaos easily.
new ResiliencePipelineBuilder().AddChaosBehavior(0.05, RestartRedisVM);
```
<!-- endSnippet -->

Example execution:

<!-- snippet: chaos-behavior-execution -->
```cs
var pipeline = new ResiliencePipelineBuilder()
.AddRetry(new RetryStrategyOptions
{
ShouldHandle = new PredicateBuilder().Handle<RedisConnectionException>(),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true, // Adds a random factor to the delay
MaxRetryAttempts = 4,
Delay = TimeSpan.FromSeconds(3),
})
.AddChaosBehavior(new BehaviorStrategyOptions // Chaos strategies are usually placed as the last ones in the pipeline
{
BehaviorAction = static args => RestartRedisVM(),
Enabled = true,
InjectionRate = 0.05
})
.Build();
```
<!-- endSnippet -->

## Defaults

| Property | Default Value | Description |
|----------------------|---------------|------------------------------------------------|
| `OnBehaviorInjected` | `null` | Action executed when the behavior is injected. |
| `BehaviorAction` | `null` | Custom behavior to be injected. |

## Diagrams

### Normal 🐵 sequence diagram

```mermaid
sequenceDiagram
actor C as Caller
participant P as Pipeline
participant B as Behavior
participant D as DecoratedUserCallback

C->>P: Calls ExecuteAsync
P->>B: Calls ExecuteCore
activate B
B-->>B: Determines Injection<br/>Decision: 🐵
deactivate B
B->>+D: Invokes
D->>-B: Returns result
B->>P: Returns result
P->>C: Returns result
```

### Chaos 🙈 sequence diagram

```mermaid
sequenceDiagram
actor C as Caller
participant P as Pipeline
participant B as Behavior
participant D as DecoratedUserCallback

C->>P: Calls ExecuteAsync
P->>B: Calls ExecuteCore
activate B
B-->>B: Determines Injection<br/>Decision: 🙈
B-->>B: Injects Behavior
deactivate B
B->>+D: Invokes
D->>-B: Returns result
B->>P: Returns result
P->>C: Returns result
```

## Anti-patterns

### Injecting delay

❌ DON'T

Use behavior strategies to inject delays.

✅ DO

Use the latency chaos instead as the [`LatencyChaosStrategy`](latency.md) already correctly handles synchronous/asynchronous delay executions, cancellations, etc.
135 changes: 135 additions & 0 deletions docs/chaos/fault.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Fault chaos strategy

## About

- **Options**: [`FaultStrategyOptions`](xref:Polly.Simmy.Fault.FaultStrategyOptions)
- **Extensions**: `AddChaosFault`
- **Strategy Type**: Proactive

---

The fault chaos strategy is designed to introduce faults (exceptions) into the system, simulating real-world scenarios where operations might fail unexpectedly. It is configurable to inject specific types of exceptions or use custom logic to generate faults dynamically.

## Usage

<!-- snippet: chaos-fault-usage -->
```cs
// 10% of invocations will be randomly affected.
var optionsBasic = new FaultStrategyOptions
{
FaultGenerator = static args => new ValueTask<Exception?>(new InvalidOperationException("Dummy exception")),
Enabled = true,
InjectionRate = 0.1
};

// To use a custom function to generate the fault to inject.
var optionsWithFaultGenerator = new FaultStrategyOptions
{
FaultGenerator = static args =>
{
Exception? exception = args.Context.OperationKey switch
{
"DataLayer" => new TimeoutException(),
"ApplicationLayer" => new InvalidOperationException(),
_ => null // When the fault generator returns null the strategy won't inject any fault and it will just invoke the user's callback
};

return new ValueTask<Exception?>(exception);
},
Enabled = true,
InjectionRate = 0.1
};

// To get notifications when a fault is injected
var optionsOnFaultInjected = new FaultStrategyOptions
{
FaultGenerator = static args => new ValueTask<Exception?>(new InvalidOperationException("Dummy exception")),
Enabled = true,
InjectionRate = 0.1,
OnFaultInjected = static args =>
{
Console.WriteLine("OnFaultInjected, Exception: {0}, Operation: {1}.", args.Fault.Message, args.Context.OperationKey);
return default;
}
};

// Add a fault strategy with a FaultStrategyOptions instance to the pipeline
new ResiliencePipelineBuilder().AddChaosFault(optionsBasic);
new ResiliencePipelineBuilder<HttpResponseMessage>().AddChaosFault(optionsWithFaultGenerator);

// There are also a couple of handy overloads to inject the chaos easily.
new ResiliencePipelineBuilder().AddChaosFault(0.1, () => new InvalidOperationException("Dummy exception"));
```
<!-- endSnippet -->

Example execution:

<!-- snippet: chaos-fault-execution -->
```cs
var pipeline = new ResiliencePipelineBuilder()
.AddRetry(new RetryStrategyOptions
{
ShouldHandle = new PredicateBuilder().Handle<InvalidOperationException>(),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true, // Adds a random factor to the delay
MaxRetryAttempts = 4,
Delay = TimeSpan.FromSeconds(3),
})
.AddChaosFault(new FaultStrategyOptions // Chaos strategies are usually placed as the last ones in the pipeline
{
FaultGenerator = static args => new ValueTask<Exception?>(new InvalidOperationException("Dummy exception")),
Enabled = true,
InjectionRate = 0.1
})
.Build();
```
<!-- endSnippet -->

## Defaults

| Property | Default Value | Description |
|-------------------|---------------|------------------------------------------------------|
| `OnFaultInjected` | `null` | Action executed when the fault is injected. |
| `FaultGenerator` | `null` | Generates the fault to inject for a given execution. |

## Diagrams

### Normal 🐵 sequence diagram

```mermaid
sequenceDiagram
actor C as Caller
participant P as Pipeline
participant F as Fault
participant D as DecoratedUserCallback

C->>P: Calls ExecuteAsync
P->>F: Calls ExecuteCore
activate F
F-->>F: Determines Injection<br/>Decision: 🐵
deactivate F
F->>+D: Invokes
D->>-F: Returns result
F->>P: Returns result
P->>C: Returns result
```

### Chaos 🙈 sequence diagram

```mermaid
sequenceDiagram
actor C as Caller
participant P as Pipeline
participant F as Fault
participant D as DecoratedUserCallback

C->>P: Calls ExecuteAsync
P->>F: Calls ExecuteCore
activate F
F-->>F: Determines Injection<br/>Decision: 🙈
F-->>F: Injects Fault
deactivate F
Note over D: The user's Callback is not invoked<br/>when a fault is injected
F->>P: Throws injected Fault
P->>C: Propagates Exception
```
53 changes: 53 additions & 0 deletions docs/chaos/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Chaos engineering with Simmy

[Simmy][simmy] is a major new addition to Polly library, adding a chaos engineering and fault-injection dimension to Polly, through the provision of strategies to selectively inject faults, latency, custom behavior or fake results.

![Simmy](../media/simmy-logo.png)

## Motivation

There are a lot of questions when it comes to chaos engineering and making sure that a system is actually ready to face the worst possible scenarios:

* Is my system resilient enough?
* Am I handling the right exceptions/scenarios?
* How will my system behave if X happens?
* How can I test without waiting for a handled (or even unhandled) exception to happen in my production environment?

Using Polly helps introduce resilience to a project, but we don't want to have to wait for expected or unexpected failures to test it out. A resilience could be wrongly implemented; testing the scenarios is not straightforward; and mocking failure of some dependencies (for example a cloud SaaS or PaaS service) is not always straightforward.

### What is needed to simulate chaotic scenarios?

* A way to simulate failures of dependencies (any service dependency for example).
* Define when to fail based on some external factors - maybe global configuration or some rule.
* A way to revert easily, to control the blast radius.
* To be production grade, to run this in a production or near-production system with automation.

## Chaos strategies (a.k.a Monkey strategies)

Chaos strategies (or Monkey strategies as we call them) are in essence a [Resilience strategy](../strategies/index.md#built-in-strategies), which means, as a *Resilience Strategy* is the minimum unit of resilience for Polly, a *Chaos Strategy* is the minimum unit of chaos for Simmy.

### Built-in strategies

| Strategy | Reactive | What does the strategy do? |
|-------------------------|----------|----------------------------------------------------------------------|
| [Fault](fault.md) | No | Injects exceptions in your system. |
| [Result](result.md) | Yes | Substitute results to fake outcomes in your system. |
| [Latency](latency.md) | No | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | No | Allows you to inject *any* extra behaviour, before a call is placed. |

## Usage

It is usual to place the chaos strategy as the last strategy in the resilience pipeline. By placing the chaos strategies as last, they subvert the usual outbound call at the last minute, substituting their fault or adding extra latency, etc. The existing resilience strategies - further out in the `ResiliencePipeline` - still apply, so you can test how the Polly resilience strategies you have configured handle the chaos/faults injected by Simmy.

## Common options across strategies

All the strategies' options implement the [`MonkeyStrategyOptions`](xref:Polly.Simmy.MonkeyStrategyOptions) class as it contains the basic configuration for every chaos strategy.

| Property | Default Value | Description |
|--------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `InjectionRate` | 0.001 ms | A decimal between 0 and 1 inclusive. The strategy will inject the chaos, randomly, that proportion of the time, e.g.: if 0.2, twenty percent of calls will be randomly affected; if 0.01, one percent of calls; if 1, all calls. |
| `InjectionRateGenerator` | `null` | Generates the injection rate for a given execution, which the value should be between [0, 1] (inclusive). |
| `Enabled` | `false` | Determines whether the strategy is enabled or not. |
| `EnabledGenerator` | `null` | The generator that indicates whether the chaos strategy is enabled for a given execution. |

[simmy]: https://github.com/Polly-Contrib/Simmy
Loading
Loading