Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azure-ai-evaluation release 1.0.0b5 2024-10-28 #38138

Merged
merged 56 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
7de6367
Update task_query_response.prompty
nagkumar91 Oct 1, 2024
f288b34
Update task_simulate.prompty
nagkumar91 Oct 1, 2024
2a4b6f7
Update task_query_response.prompty
nagkumar91 Oct 2, 2024
c8ce251
Update task_simulate.prompty
nagkumar91 Oct 2, 2024
4522ae4
Merge branch 'Azure:main' into main
nagkumar91 Oct 3, 2024
32e9c1d
Merge branch 'Azure:main' into main
nagkumar91 Oct 7, 2024
76df69d
Merge branch 'Azure:main' into main
nagkumar91 Oct 8, 2024
aeddcb4
Merge branch 'Azure:main' into main
nagkumar91 Oct 8, 2024
65a759c
Merge branch 'Azure:main' into main
nagkumar91 Oct 9, 2024
e4cdd30
Fix the api_key needed
Oct 9, 2024
e3ab026
Merge branch 'Azure:main' into main
nagkumar91 Oct 11, 2024
4fb09c4
Merge branch 'Azure:main' into main
nagkumar91 Oct 15, 2024
e71a52d
Merge branch 'Azure:main' into main
nagkumar91 Oct 15, 2024
87166b3
Merge branch 'Azure:main' into main
nagkumar91 Oct 16, 2024
b478651
Update for release
nagkumar91 Oct 16, 2024
8e5a264
Black fix for file
nagkumar91 Oct 16, 2024
2077d6d
Merge branch 'Azure:main' into main
nagkumar91 Oct 17, 2024
3ab59c8
Merge branch 'Azure:main' into main
nagkumar91 Oct 17, 2024
3a80606
Add original text in global context
Oct 17, 2024
6768f9a
Update test
Oct 17, 2024
f7cc4bb
Update the indirect attack simulator
Oct 18, 2024
07eb466
Black suggested fixes
Oct 18, 2024
942bfd5
Update simulator prompty
Oct 18, 2024
2d4c376
Merge branch 'main' into main
nagkumar91 Oct 18, 2024
98cad97
Update adversarial scenario enum to exclude XPIA
Oct 18, 2024
d510316
Update changelog
Oct 18, 2024
742943e
Black fixes
Oct 18, 2024
12e0615
Remove duplicate import
Oct 18, 2024
de32b50
Fix the mypy error
Oct 19, 2024
4b64132
Mypy please be happy
Oct 21, 2024
1c0b4dd
Updates to non adv simulator
Oct 22, 2024
c4f9111
Merge branch 'Azure:main' into main
nagkumar91 Oct 22, 2024
6de617c
accept context from assistant messages, exclude them when using them …
Oct 23, 2024
1e5d40c
update changelog
Oct 23, 2024
93b29c7
pylint fixes
Oct 23, 2024
8e3ddc3
pylint fixes
Oct 23, 2024
31e0d29
Merge branch 'main' into main
nagkumar91 Oct 23, 2024
4ccc7c8
remove redundant quotes
Oct 23, 2024
bed5196
Fix typo
Oct 23, 2024
0fdd644
pylint fix
Oct 23, 2024
1f695cc
Update broken tests
Oct 23, 2024
3da3a94
Merge branch 'main' into main
nagkumar91 Oct 23, 2024
56c2657
Merge branch 'Azure:main' into main
nagkumar91 Oct 23, 2024
b04b3e6
Merge branch 'Azure:main' into main
nagkumar91 Oct 24, 2024
b9793ca
Merge branch 'Azure:main' into main
nagkumar91 Oct 25, 2024
92c9a6d
Include the grounding json in the manifest
Oct 25, 2024
0673cd5
Fix typo
Oct 25, 2024
7b360fc
Come on package
Oct 25, 2024
e3fd2bb
Merge branch 'Azure:main' into main
nagkumar91 Oct 28, 2024
c9f38c9
Release 1.0.0b5
Oct 28, 2024
bbb78fd
Merge branch 'main' of https://github.com/nagkumar91/azure-sdk-for-py…
Oct 28, 2024
ed7eed1
Notice from Chang
Oct 28, 2024
103f397
Merge branch 'Azure:main' into main
nagkumar91 Oct 28, 2024
3de5b66
Remove adv_conv template parameters from the outputs
Oct 28, 2024
21e3551
Merge branch 'main' of https://github.com/nagkumar91/azure-sdk-for-py…
Oct 28, 2024
78df8c9
Merge branch 'Azure:main' into main
nagkumar91 Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/evaluation/azure-ai-evaluation/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Release History

## 1.0.0b5 (Unreleased)
## 1.0.0b5 (2024-10-28)

### Features Added
- Added `GroundednessProEvaluator`, which is a service-based evaluator for determining response groundedness.
Expand Down
20 changes: 20 additions & 0 deletions sdk/evaluation/azure-ai-evaluation/NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,23 @@ distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.


License notice for [Is GPT-4 a reliable rater? Evaluating consistency in GPT-4's text ratings](https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2023.1272229/full)
------------------------------------------------------------------------------------------------------------------
Copyright © 2023 Hackl, Müller, Granitzer and Sailer. This work is openly licensed via [CC BY 4.0](http://creativecommons.org/licenses/by/4.0/).


License notice for [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://aclanthology.org/2023.newsum-1.1) (Wang et al., NewSum 2023)
------------------------------------------------------------------------------------------------------------------
Copyright © 2023. This work is openly licensed via [CC BY 4.0](http://creativecommons.org/licenses/by/4.0/).


License notice for [SummEval: Re-evaluating Summarization Evaluation.](https://doi.org/10.1162/tacl_a_00373) (Fabbri et al.)
------------------------------------------------------------------------------------------------------------------
© 2021 Association for Computational Linguistics. This work is openly licensed via [CC BY 4.0](http://creativecommons.org/licenses/by/4.0/).


License notice for [Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks](https://aclanthology.org/2023.emnlp-main.543) (Sottana et al., EMNLP 2023)
------------------------------------------------------------------------------------------------------------------
© 2023 Association for Computational Linguistics. This work is openly licensed via [CC BY 4.0](http://creativecommons.org/licenses/by/4.0/).
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,9 @@ def _to_chat_protocol(
"target_population",
"topic",
"ch_template_placeholder",
"chatbot_name",
"name",
"group",
):
template_parameters.pop(key, None)
if conversation_category:
Expand Down