Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(plugins): ai-prompt-guard-plugin #12337

Merged
merged 3 commits into from
Jan 25, 2024
Merged

Conversation

tysoekong
Copy link
Contributor

Summary

This commit offers another plugin that extends the functionality of "AI Proxy" in #12207.

It compares the user's llm/v1/chat or llm/v1/completions request against a series of regular expressions, in two config arrays:

  • Allow
  • Deny

If the request matches any regex pattern in deny, the caller is 400'd.

If any allow is specified, by the request matches none of them, the caller is also 400'd.

Engineering design document is available for this feature, but it is quite simple. Comprehensive tests supplied.

This reason for its development, is that many of our users would like to block specific prompts, words, phrases, or otherwise more tightly control how an AI / LLM model is used, if being called via Kong, and this applies especially with the AI Proxy plugin that will simplify this process.

Checklist

@tysoekong tysoekong requested review from flrgh and hbagdi January 12, 2024 08:57
@github-actions github-actions bot added chore Not part of the core functionality of kong, but still needed schema-change-noteworthy labels Jan 12, 2024
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch from 1733ed4 to 6d0ad98 Compare January 12, 2024 09:29
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch 2 times, most recently from d1c0bd7 to 2e8c766 Compare January 12, 2024 10:58
@RobSerafini RobSerafini requested a review from a team January 17, 2024 16:42
@flrgh flrgh added this to the 3.6.0 milestone Jan 19, 2024
@flrgh flrgh added the cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee label Jan 19, 2024
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch 3 times, most recently from ecccaf3 to bcb617b Compare January 22, 2024 17:51
Copy link
Contributor

@flrgh flrgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple style nitpicks and a testing request remaining, but once those are addressed it's a 👍 from me.

@flrgh
Copy link
Contributor

flrgh commented Jan 22, 2024

@locao do we have another gateway reviewer for this PR?

@locao
Copy link
Contributor

locao commented Jan 22, 2024

@flrgh this is on me, reviewing it now.

Copy link
Contributor

@locao locao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems pretty close to ready to merge.

@tysoekong do you think it's possible to add an integration test?

kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
kong/plugins/ai-prompt-guard/handler.lua Outdated Show resolved Hide resolved
spec/03-plugins/42-ai-prompt-guard/00_config_spec.lua Outdated Show resolved Hide resolved
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch 2 times, most recently from a4f5312 to 549609b Compare January 24, 2024 12:28
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch from 549609b to 00f5e46 Compare January 24, 2024 12:33
@flrgh flrgh requested a review from locao January 24, 2024 17:01
Copy link
Contributor

@locao locao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one last clean up (but that's not a blocker) and I think we're good to go.

kong/plugins/ai-prompt-guard/schema.lua Outdated Show resolved Hide resolved
@tysoekong tysoekong force-pushed the feat/ai_prompt_guard_plugin branch from 138022f to f471cae Compare January 25, 2024 18:42
@locao locao merged commit 93a1887 into master Jan 25, 2024
23 checks passed
@locao locao deleted the feat/ai_prompt_guard_plugin branch January 25, 2024 19:24
@team-gateway-bot
Copy link
Collaborator

Backport failed for release/3.6.x, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release/3.6.x
git worktree add -d .worktree/backport-12337-to-release/3.6.x origin/release/3.6.x
cd .worktree/backport-12337-to-release/3.6.x
git switch --create backport-12337-to-release/3.6.x
git cherry-pick -x 93a18877a42855bddedfd01521e420603fd5544f

@team-gateway-bot
Copy link
Collaborator

Cherry-pick failed for master, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git remote add upstream https://github.com/kong/kong-ee
git fetch upstream master
git worktree add -d .worktree/cherry-pick-12337-to-master-to-upstream upstream/master
cd .worktree/cherry-pick-12337-to-master-to-upstream
git checkout -b cherry-pick-12337-to-master-to-upstream
ancref=$(git merge-base 3ef9235a6c76c76a641933abafcd857c740befe0 f471cae50bb87142a953b5a036e53bf17d38c093)
git cherry-pick -x $ancref..f471cae50bb87142a953b5a036e53bf17d38c093

tysoekong added a commit that referenced this pull request Jan 25, 2024
* feat(plugins): ai-prompt-guard-plugin

* fix(ai-prompt-guard): fixes from code review

* Update kong/plugins/ai-prompt-guard/schema.lua

Co-authored-by: Vinicius Mignot <[email protected]>

---------

Co-authored-by: Jack Tysoe <[email protected]>
Co-authored-by: Vinicius Mignot <[email protected]>
flrgh pushed a commit that referenced this pull request Jan 25, 2024
* feat(plugins): ai-prompt-guard-plugin

* fix(ai-prompt-guard): fixes from code review

* Update kong/plugins/ai-prompt-guard/schema.lua



---------

Co-authored-by: Jack Tysoe <[email protected]>
Co-authored-by: Vinicius Mignot <[email protected]>
This was referenced Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee chore Not part of the core functionality of kong, but still needed schema-change-noteworthy size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants