Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow approved AI players to join games #1996

Open
wants to merge 2 commits into
base: next
Choose a base branch
from

Conversation

yannikkellerde
Copy link

The intention of this PR is not for it to be merged immediately. Instead it shall serve as a platform to discuss if approved AI players should be allowed on https://secrethitler.io/ and how one could make this happen in practice.

Background

During my PhD, I've been studying deceptive AI in social deduction games for a year now and have developed an LLM based Secret-Hitler AI that seems to be reaching human level of play. Now I plan to evaluate that AI against actual human secret-hitler players and I think that secrethitler.io would be the most natural place to do this. I've already brought this up with some moderators and coz and they suggested that I create a PR for open discussion.

I've already written an interface between my AI and the cozuya/secret-hitler interface and am currently working on a private test deployment on which my AIs play. So I think there is not too much work left to make it possible for my AI to join games on secrethitler.io. Primarily I would need a Cloudflare bypass so that my AI can connect to secrethitler.io headless. Otherwise it would make sense to give the players the option to decide if they want to allow AI to join their games and that is what this PR is for.

I am also happy to share videos of my bots playing privately with moderators so you may be able to get a better image of what is to come. In the future, I may also be able to give access to my private test deployment with bots privately to moderators.

Changes

This PR adds an Opt-Out indicator during game creation that indicates if AI players should be allowed to join. The idea is that approved AI operators could be granted a Cloudflare bypass and use the allowBots indicator on each game to decide which games their AI should join.

Screenshots

bots

Tested Locally

  • This PR has been tested locally, and ensured to not cause any breaking changes

Tests

  • This PR does not require tests

Changelog

  • Changelog Section below has been updated with Changelog entry

Changelog Entry (delete this section if this PR does not need a changelog entry)

Check one, delete the other:

  • New Feature

Check one, delete the other:

  • Minor Change

Changelog Headline: Add slider to allow AI players join games

Changelog Details: While creating a game you may now switch "Allow Bots to join the game" on or off.

@Vigasaurus
Copy link
Collaborator

I'm open to this as a concept, would probably want to get some additional community feedback though (even if that is just something informal through what mods are hearing, etc). Further, I'd think having it disabled by default everywhere possible (for now) would be good, until it's impacts and community feedback is received - at which point flipping it to default on would be totally fine, imo.

This is very interesting though - greatly appreciate the effort put in this far!

I do notice at least a couple of technical issues with this PR, but I'll put those in a proper review a little later.

@carlgauss37
Copy link
Collaborator

Adding to what Vig said, i am all for it, as long as no ranked games are affected by this update, personally I don't like the idea of moving elo based on bot games, so practice and casual would be 100% ok with, ranked no. That is on top of making it default no and allowing people to customize it to allow bots instead of disable bots.

@yannikkellerde
Copy link
Author

Totally understand the point about bots only in casual games. Let's make it that way and then collect some community feedback before deciding if bots should ever be in ranked games. Regarding making it Opt-In instead of Opt-Out, I'd be worried that most of the player base would not even notice that option and thus never activate it -> Thus making it impossible for me to collect a lot of data with bot play.

What are you mainly worried about with an Opt-Out setup?
A) That players would be unpleasantly surprised when they have a bot in their games for the first time?
B) Or that players that do not want bots would be annoyed when they have to disable bots each time?
C) something else

A) could be solved with a popup or homepage entry informing the players about the change.
B) could maybe be solved by user-level defaults when creating games. E.g. opt-out of bots once and it will be your default forever.

Let me know what you think.

@carlgauss37
Copy link
Collaborator

So i am worried of both A and B, generally i think the philosophy we should follow is to disrupt players the least, also it's all solved, ours and your problem by giving the same amount of attention to the "opt-in" option. So basically your A) solution reversed. About making it a player's selection that is saved more permanently we can talk about it cause either way could and probably should be set. Cause in any case, opt-in or opt-out people will get tired of toggling it every time eventually. In general i think we can give it a lot of attention once rolled out (I myself as a player want to test it the first), but i am pretty adamant in not imposing a new feature on people that aren't used to it or waiting for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants