Skip to content

Commit

Permalink
Update randomization pattern for Adversarial simulation (#38211)
Browse files Browse the repository at this point in the history
* Update randomization pattern for Adversarial simulation

* update changelog
  • Loading branch information
slister1001 authored Oct 31, 2024
1 parent 7212809 commit 4e931f4
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 11 deletions.
1 change: 1 addition & 0 deletions sdk/evaluation/azure-ai-evaluation/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
### Other Changes
- Refined error messages for serviced-based evaluators and simulators.
- Introduced environment variable `AI_EVALS_DISABLE_EXPERIMENTAL_WARNING` to disable the warning message for experimental features.
- Changed the randomization pattern for `AdversarialSimulator` such that there is an almost equal number of Adversarial harm categories (e.g. Hate + Unfairness, Self-Harm, Violence, Sex) represented in the `AdversarialSimulator` outputs. Previously, for 200 `max_simulation_results` a user might see 140 results belonging to the 'Hate + Unfairness' category and 40 results belonging to the 'Self-Harm' category. Now, user will see 50 results for each of Hate + Unfairness, Self-Harm, Violence, and Sex.

## 1.0.0b5 (2024-10-28)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import logging
import random
from typing import Any, Callable, Dict, List, Literal, Optional, Union, cast
from itertools import zip_longest

from tqdm import tqdm

Expand Down Expand Up @@ -215,17 +216,18 @@ async def __call__(
ncols=100,
unit="simulations",
)
for template in templates:
parameter_order = list(range(len(template.template_parameters)))
if randomize_order:
# The template parameter lists are persistent across sim runs within a session,
# So randomize a the selection instead of the parameter list directly,
# or a potentially large deep copy.
if randomization_seed is not None:
random.seed(randomization_seed)
random.shuffle(parameter_order)
for index in parameter_order:
parameter = template.template_parameters[index].copy()

if randomize_order:
# The template parameter lists are persistent across sim runs within a session,
# So randomize a the selection instead of the parameter list directly,
# or a potentially large deep copy.
if randomization_seed is not None:
random.seed(randomization_seed)
random.shuffle(templates)
parameter_lists = [t.template_parameters for t in templates]
zipped_parameters = list(zip_longest(*parameter_lists))
for param_group in zipped_parameters:
for template, parameter in zip(templates, param_group):
if _jailbreak_type == "upia":
parameter = self._join_conversation_starter(parameter, random.choice(jailbreak_dataset))
tasks.append(
Expand Down

0 comments on commit 4e931f4

Please sign in to comment.