Skip to content

Commit

Permalink
Instructions on how to utilize Admin/RemoveSanitizers (#8142)
Browse files Browse the repository at this point in the history
* instructions on how to remove sanitizers
* updating the runtime data model to provide a cleaner appearance
* making the data revealed on info/active more apparent
  • Loading branch information
scbedd authored May 28, 2024
1 parent 04333b3 commit 4e38840
Show file tree
Hide file tree
Showing 5 changed files with 110 additions and 15 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ public class ActionDescription
public string Name;
public string Description;
public CtorDescription ConstructorDetails;
public string SanitizerId;

public ActionDescription() { }

Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,7 @@
using Microsoft.AspNetCore.Mvc.RazorPages;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Xml;
using System.IO;
using Azure.Sdk.Tools.TestProxy.Common;
using System.Collections.Concurrent;
using Microsoft.CodeAnalysis.Operations;
using Azure.Sdk.Tools.TestProxy.Common.Exceptions;

namespace Azure.Sdk.Tools.TestProxy.Models
Expand Down Expand Up @@ -44,14 +38,14 @@ private List<ActionDescription> _populateFromHandler(RecordingHandler handler, s
handler.InMemorySessions
};

var sanitizers = handler.SanitizerRegistry.GetSanitizers();
var sanitizers = handler.SanitizerRegistry.GetRegisteredSanitizers();
var recordingFound = false;
if (!string.IsNullOrWhiteSpace(recordingId)){
foreach (var sessionDict in searchCollections)
{
if (sessionDict.TryGetValue(recordingId, out var session))
{
sanitizers = handler.SanitizerRegistry.GetSanitizers(session);
sanitizers = handler.SanitizerRegistry.GetRegisteredSanitizers(session);
transforms = transforms.Concat(session.AdditionalTransforms);

if (session.CustomMatcher != null)
Expand All @@ -76,9 +70,10 @@ private List<ActionDescription> _populateFromHandler(RecordingHandler handler, s
descriptions.AddRange(sanitizers.Select(x => new ActionDescription()
{
ActionType = MetaDataType.Sanitizer,
Name = x.GetType().Name,
ConstructorDetails = GetInstanceDetails(x),
Description = GetClassDocComment(x.GetType(), docXML)
Name = x.Sanitizer.GetType().Name,
SanitizerId = x.Id,
ConstructorDetails = GetInstanceDetails(x.Sanitizer),
Description = GetClassDocComment(x.Sanitizer.GetType(), docXML)
}));

descriptions.AddRange(handler.Transforms.Select(x => new ActionDescription()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
using System.Reflection;
using System.Xml;
using System.IO;
using Azure.Sdk.Tools.TestProxy.Common;

namespace Azure.Sdk.Tools.TestProxy.Models
{
Expand Down Expand Up @@ -107,7 +106,7 @@ public CtorDescription GetInstanceDetails(object target)
}
}

arguments.Add(new Tuple<string, string>(field.Name, propValue));
arguments.Add(new Tuple<string, string>(GetFriendlyFieldName(field.Name), propValue));
}
}

Expand All @@ -118,6 +117,31 @@ public CtorDescription GetInstanceDetails(object target)
};
}

public static Dictionary<string, string> FieldNameMapping = new Dictionary<string, string>()
{
{ "_jsonPath", "jsonPath" },
{ "_value", "value" },
{ "_regex", "regex" },
{ "_groupForReplace", "groupForReplace" },
{ "_condition", "condition" },
{ "_target", "target" },
{ "_key", "key" },
{ "_method", "method" },
{ "_newValue", "value" },
{ "_resetAfterFirst", "resetAfterFirst" },
{ "_regexValue", "regex" }
};

public string GetFriendlyFieldName(string fieldName)
{
if (!string.IsNullOrWhiteSpace(fieldName) && FieldNameMapping.ContainsKey(fieldName))
{
return FieldNameMapping[fieldName];
}

return fieldName;
}

public string GetClassDocComment(Type type, XmlDocument docCommentXml)
{
var memberSearchString = String.Format(CLASS_FORMAT_STRING, type.FullName);
Expand Down
77 changes: 76 additions & 1 deletion tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,10 @@ Of course, feel free to check any of the [examples](https://github.com/Azure/azu

The test-proxy is a record/playback solution. As a result, there a few concepts that devs will likely recognize from other record/playback solutions:

- A `Sanitizer` is used to remove sensitive information prior to storage. When a request comes in during `playback` mode, `sanitizers` are applied to the request prior to matching to a recording.
- A `Sanitizer` is used to remove sensitive information prior to storage. Sanitizers are applied...
- To the request matching against the recording entries during `playback` mode.
- To the recording entries loaded from disk when starting a `playback` session.
- To the recording entries _before_ they are saved to disk when stopping a `record` session.
- `Matchers` are used to retrieve a request/response pair from a previous recording. By default, it functions by comparing `URI`, `Headers`, and `Body`. As of now, only a single matcher can be used when retrieving an entry during playback.
- A `Transform` is used when a user needs to "transform" a matched recording response with some value from the incoming request. This action is specific to `playback` mode. For instance, the test-proxy has two default `transforms`:
- `x-ms-client-id` is copied from request and applied to response prior to return.
Expand Down Expand Up @@ -551,6 +554,78 @@ In some cases, users need to register a lot (10+) of sanitizers. In this case, g
]
```

#### Knowing what was added

When `AddSanitizer` or `AddSanitizers` is called, check the response `body` for an array containing the ids of sanitizers that have been registered.

Example response body:

```jsonc
// POSTS to Admin/AddSanitizer has individual result
{
"Sanitizer": "3"
}
```

```jsonc
// POSTS to Admin/AddSanitizers has multiple results
{
"Sanitizers": ["3", "4", "9"]
}
```

#### Removing a sanitizer

Following #8120, sanitizers were given identifiers so that they can be removed.

- The default `session` sanitizers list can be found in code here.
- Visiting `http://localhost:5000/Info/Active` on your browser when the proxy is running on your machine will present you with an easy summary of these available sanitizers as well.

When a recording or playback session is begun (`Playback/Start` or `Record/Start`), all the **current** session sanitizers are applied to the recording. Session sanitizers added **after** record or playback has begun do not apply to the prior-started session.

To remove a session-level sanitizer, one only must...

```jsonc
// request method = `POST`
// request URI = <proxyUrl>/Admin/RemoveSanitizers
// request headers =
{
"Content-Type": "application/json",
"Content-Length": 36
}
// request body =
{
Sanitizers: ["AZSDK002", "AZSDK003"]
}
```

On successful request, users will receive a list of the removed identifiers.

```jsonc
// response body
{
Removed: ["ID1"]
}
```

To remove a sanitizer from a specific recording...

```jsonc
// request method = `POST`
// request URI = <proxyUrl>/Admin/RemoveSanitizers
// request headers =
{
"Content-Type": "application/json",
"Content-Length": 36,
"x-recording-id": "your-recording-id-here"
}
// request body =
{
Sanitizers: ["AZSDK002", "AZSDK003"]
}
```


### Set a Matcher

Setting a matcher is just like adding a `sanitizer`. Set the `x-abstraction-identifier` value to the name of the matcher you want to instantiate, and provide the proper constructor arguments in the body! Check `Info/Available/` for available matchers.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
<table style="width:100%">
<tr>
<td>
<h2>@extension.Name</h2>
<h2>"@extension.SanitizerId" - @extension.Name</h2>
</td>
</tr>
<tr>
Expand Down

0 comments on commit 4e38840

Please sign in to comment.