v2.0.0 alpha

Announcing AI Verify 2.x – A New Era of AI Governance!

We are thrilled to announce the upcoming release of AI Verify 2.x, a groundbreaking update that will revolutionize the way you validate and govern AI systems. This new version is packed with major enhancements designed to provide more flexibility, efficiency, and scalability.

What's New in AI Verify 2.0.0 Alpha?

• Modular Architecture: The toolkit has been re-architected to be more modular, allowing for easier integration and customization.
• Independent Test Algorithms: Each test algorithm can now be run as a separate Python module, giving you the freedom to use and develop algorithms independently.
• Redesigned Backend Test Engine Worker: The backend test engine worker has been completely redesigned for improved performance and reliability.

More to Come!

• New Frontend Workflows: Experience a more intuitive and streamlined user interface with our newly designed frontend workflows.
• Veritas Integration: Including Veritas into AI Verify allows financial institutions to meet common safety baseline and financial testing requirements.
• Improved support for Computer Vision: More evaluations to be included to support different types of use cases.

v0.10.3

What's Changed

• Updated the Testing Framework to incorporate controls in ISO/IEC 42001:2023 (#287)
  All controls in ISO/IEC 42001:2023 are mapped to the process checks in AI Verify testing framework. Organisations can use AI Verify toolkit to strengthen their AI governance, and practically demonstrate alignment with ISO/IEC 42001:2023, without onerous cost. Refer to crosswalk here (https://aiverifyfoundation.sg/resources/#crosswalk-with-iso42001)

• Added DevContainer by @berrydenhartog (#258)

• Improve Environment Variable Inheritance by @berrydenhartog (#267)

• Fix CI/CD for forked repo by @berrydenhartog (#282)

Minor Changes

• Omit Dev Dependencies during npm install (#285)
• Change IMDA-BTG references in GHA yml files to align with AI Verify Foundation (#289)

Full Changelog: v0.10.2...v0.10.3

v0.10.2

What's Changed

Updated Package Requirements, Setup Scripts and CI/CD Pipeline Workflows.

Bug Fixes

• Fixed issue in portal where error page is display upon report generation (#270)
• Fixed redis client reconnection issue (#266)
• Fixed image corruption plugin (#255)

Features for Improved Application Security

• Enhanced password security for db (#271)
• Enhanced file upload validation (#268)

Package Updates

• Apigw Packages (#270)
• Python Packages (#256)

Full Changelog:
v0.10.1...v0.10.2

v0.10.1

What's Changed

Bug fixes for the API Connector and implemented new application security features.

Bug Fixes:

• Fixed bug where SHAP and robustness test fails when model is connected via API.
• Fixed bug where test task updated hook does not execute while generating report.
• Updated node setup for Dockerfile, which was previously causing a 60s wait on build.

Features for improved application security:

• Removal of default usernames and passwords for Mongodb setup. Users are now required to define these credentials during setup.
• Upgraded npm packages (#231)
• Improved sanitization of file uploads (#231)
• Improved error messages and disabled stack trace to reduce information exposure (#242)
• Disabled autocompletion of sensitive fields (#231)
• CORS whitelist (#245) (#240)
• Updated the portal graphql client so that all browser requests use the portal rewrites for graphql client requests and subscriptions (#232)
• Other minor code cleanups to remove absolute file paths and default URLs (#243)

Full Changelog:
v0.10.0...v0.10.1

Installation Instructions

Build and run from Dockerfile
Build and run from Source Code

v0.10.0

What's Changed

New API Connector Feature!

We're excited to introduce a powerful enhancement to AI Verify – the API Connector feature!
Now, instead of uploading your AI Model file onto AI Verify, you can seamlessly configure an API Connection to your model server.

Learn more about the modes of accessing AI models here

Key Advantages:

• Bypass Size Limitations: Say goodbye to the constraints imposed by browser upload size limits.
• Test previously unsupported AI frameworks: The API Connector empowers you to work with models of any framework, such as PyTorch. Learn more about compatibility of model uploads here.
• Connection Settings for Batch Requests: Optimise your test runs by configuring connection timeouts, connection retries, max connections, rate limit and rate limit timeouts.

** Important Note: This feature currently supports tabular data only.

Getting Started:

To take advantage of this feature, refer to our How-To Guide. This guide provides step-by-step instructions on setting up the API Connector for seamless integration with your model server.

We are excited to have you try it out and hear what you think about this feature!
Discussion Board

Full Changelog: v0.9.4...v0.10.0

Installation Instructions

Build and run from Dockerfile
Build and run from Source Code

v0.9.4

What's Changed

Updated the Testing Framework to include Organisational Considerations and other minor bug fixes.

• Updated the Testing Framework to include Organisational Considerations when deciding AI deployment. (Updates to aiveriy.stock.reports and aiverify.stock.process-checklist plugins)
• Beyond assessment of individual AI systems, organisations need to consider issues such as the use of AI (versus non-AI options), norms and expectations as well as resources to manage the use of AI. We have added 6 process checks under Organisational Considerations.
• Fixed an issue where report title is not substituted with project name in report generated (#77)
• Fixed save message to only appear upon successful save of a project (#186)
• Other minor documentation, unit test, UI updates

Full Changelog: v0.9.3...v0.9.4

Installation Instructions

Build and run from Dockerfile
Build and run from Source Code

v0.9.3

What's Changed

Added Support for Fairness Testing For Image and Major Bug Fixes

• Fairness Testing for Image (Classification) Documentation here and here
• Fixed an issue where generate report operation executes before model and datasets (#153)
• Fixed an issue where validation fails for single-column CSV files (#165)
• Fixed an issue where Dataset and Model will not be available for upload after deselecting and reselecting the same file (#161)
• Fixed an issue where widget description overlapped with the dependencies status if the widget description is too long (#136)
• Fixed an issue where report generation is stuck when running XGBoost
• Fixed an issue where permissions issue in Docker causes Image Testing to fail (#173)

Full Changelog: v0.9.2...v0.9.3

Installation Instructions

Build and run from Dockerfile
Build and run from Source Code

v0.9.2

What's Changed

Major Bug fixes & Python 3.11 support

• Setup scripts will now run Python 3.11
• Fixed bug where some test runs are stuck (#135)
• Fixed error in process checklist numbering in report (#124)
• Vulnerability fixes

Installation Instructions

Build and run from Dockerfile
Build and run from Source Code

aiverify v0.9.1

What's Changed

Dynamic height widgets & dev updates

New Features:

• Dynamic Height functionality for widgets
• Updated stock plugins and report templates
• New radarchart and treemap packages for shared lib

Dev/code:

• Flake8 & prettier configs
• Code linting
• New unit tests
• Vulnerability fixes

The first AI Verify release

The First AI Verify Release

release v0.9.0