Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instructions on how to utilize Admin/RemoveSanitizers #8142

Merged
merged 6 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ public class ActionDescription
public string Name;
public string Description;
public CtorDescription ConstructorDetails;
public string SanitizerId;

public ActionDescription() { }

Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,7 @@
using Microsoft.AspNetCore.Mvc.RazorPages;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Xml;
using System.IO;
using Azure.Sdk.Tools.TestProxy.Common;
using System.Collections.Concurrent;
using Microsoft.CodeAnalysis.Operations;
using Azure.Sdk.Tools.TestProxy.Common.Exceptions;

namespace Azure.Sdk.Tools.TestProxy.Models
Expand Down Expand Up @@ -44,14 +38,14 @@ private List<ActionDescription> _populateFromHandler(RecordingHandler handler, s
handler.InMemorySessions
};

var sanitizers = handler.SanitizerRegistry.GetSanitizers();
var sanitizers = handler.SanitizerRegistry.GetRegisteredSanitizers();
var recordingFound = false;
if (!string.IsNullOrWhiteSpace(recordingId)){
foreach (var sessionDict in searchCollections)
{
if (sessionDict.TryGetValue(recordingId, out var session))
{
sanitizers = handler.SanitizerRegistry.GetSanitizers(session);
sanitizers = handler.SanitizerRegistry.GetRegisteredSanitizers(session);
transforms = transforms.Concat(session.AdditionalTransforms);

if (session.CustomMatcher != null)
Expand All @@ -76,9 +70,10 @@ private List<ActionDescription> _populateFromHandler(RecordingHandler handler, s
descriptions.AddRange(sanitizers.Select(x => new ActionDescription()
{
ActionType = MetaDataType.Sanitizer,
Name = x.GetType().Name,
ConstructorDetails = GetInstanceDetails(x),
Description = GetClassDocComment(x.GetType(), docXML)
Name = x.Sanitizer.GetType().Name,
SanitizerId = x.Id,
ConstructorDetails = GetInstanceDetails(x.Sanitizer),
Description = GetClassDocComment(x.Sanitizer.GetType(), docXML)
}));

descriptions.AddRange(handler.Transforms.Select(x => new ActionDescription()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
using System.Reflection;
using System.Xml;
using System.IO;
using Azure.Sdk.Tools.TestProxy.Common;

namespace Azure.Sdk.Tools.TestProxy.Models
{
Expand Down Expand Up @@ -107,7 +106,7 @@ public CtorDescription GetInstanceDetails(object target)
}
}

arguments.Add(new Tuple<string, string>(field.Name, propValue));
arguments.Add(new Tuple<string, string>(GetFriendlyFieldName(field.Name), propValue));
}
}

Expand All @@ -118,6 +117,31 @@ public CtorDescription GetInstanceDetails(object target)
};
}

public static Dictionary<string, string> FieldNameMapping = new Dictionary<string, string>()
{
{ "_jsonPath", "jsonPath" },
{ "_value", "value" },
{ "_regex", "regex" },
{ "_groupForReplace", "groupForReplace" },
{ "_condition", "condition" },
{ "_target", "target" },
{ "_key", "key" },
{ "_method", "method" },
{ "_newValue", "value" },
{ "_resetAfterFirst", "resetAfterFirst" },
{ "_regexValue", "regex" }
};

public string GetFriendlyFieldName(string fieldName)
{
if (!string.IsNullOrWhiteSpace(fieldName) && FieldNameMapping.ContainsKey(fieldName))
{
return FieldNameMapping[fieldName];
}

return fieldName;
}

public string GetClassDocComment(Type type, XmlDocument docCommentXml)
{
var memberSearchString = String.Format(CLASS_FORMAT_STRING, type.FullName);
Expand Down
77 changes: 76 additions & 1 deletion tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,10 @@ Of course, feel free to check any of the [examples](https://github.com/Azure/azu

The test-proxy is a record/playback solution. As a result, there a few concepts that devs will likely recognize from other record/playback solutions:

- A `Sanitizer` is used to remove sensitive information prior to storage. When a request comes in during `playback` mode, `sanitizers` are applied to the request prior to matching to a recording.
- A `Sanitizer` is used to remove sensitive information prior to storage. Sanitizers are applied...
- To the request matching against the recording entries during `playback` mode.
- To the recording entries loaded from disk when starting a `playback` session.
- To the recording entries _before_ they are saved to disk when stopping a `record` session.
- `Matchers` are used to retrieve a request/response pair from a previous recording. By default, it functions by comparing `URI`, `Headers`, and `Body`. As of now, only a single matcher can be used when retrieving an entry during playback.
- A `Transform` is used when a user needs to "transform" a matched recording response with some value from the incoming request. This action is specific to `playback` mode. For instance, the test-proxy has two default `transforms`:
- `x-ms-client-id` is copied from request and applied to response prior to return.
Expand Down Expand Up @@ -551,6 +554,78 @@ In some cases, users need to register a lot (10+) of sanitizers. In this case, g
]
```

#### Knowing what was added

When `AddSanitizer` or `AddSanitizers` is called, check the response `body` for an array containing the ids of sanitizers that have been registered.

Example response body:

```jsonc
// POSTS to Admin/AddSanitizer has individual result
{
"Sanitizer": "3"
}
```

```jsonc
// POSTS to Admin/AddSanitizers has multiple results
{
"Sanitizers": ["3", "4", "9"]
}
```

#### Removing a sanitizer

Following #8120, sanitizers were given identifiers so that they can be removed.

- The default `session` sanitizers list can be found in code here.
- Visiting `http://localhost:5000/Info/Active` on your browser when the proxy is running on your machine will present you with an easy summary of these available sanitizers as well.

When a recording or playback session is begun (`Playback/Start` or `Record/Start`), all the **current** session sanitizers are applied to the recording. Session sanitizers added **after** record or playback has begun do not apply to the prior-started session.

To remove a session-level sanitizer, one only must...

```jsonc
// request method = `POST`
// request URI = <proxyUrl>/Admin/RemoveSanitizers
// request headers =
{
"Content-Type": "application/json",
"Content-Length": 36
}
// request body =
{
Sanitizers: ["AZSDK002", "AZSDK003"]
}
```

On successful request, users will receive a list of the removed identifiers.

```jsonc
// response body
{
Removed: ["ID1"]
}
```

To remove a sanitizer from a specific recording...

```jsonc
// request method = `POST`
// request URI = <proxyUrl>/Admin/RemoveSanitizers
// request headers =
{
"Content-Type": "application/json",
"Content-Length": 36,
"x-recording-id": "your-recording-id-here"
}
// request body =
{
Sanitizers: ["AZSDK002", "AZSDK003"]
}
```


### Set a Matcher

Setting a matcher is just like adding a `sanitizer`. Set the `x-abstraction-identifier` value to the name of the matcher you want to instantiate, and provide the proper constructor arguments in the body! Check `Info/Available/` for available matchers.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
<table style="width:100%">
<tr>
<td>
<h2>@extension.Name</h2>
<h2>"@extension.SanitizerId" - @extension.Name</h2>
</td>
</tr>
<tr>
Expand Down