Skip to content

ayushkhd/MarvinGPT

Repository files navigation

MarvinGPT

MarvinGPT is a ChatGPT plugin that imbues the conversational AI with emotional intelligence, adding EQ to the existing IQ. It does so by running real-time facial detection and emotional analysis and created an API to allow our plugin to receive this information. With emotional context, ChatGPT tailors its response to better serve the user in their current emotional state.

In order to be truly effective, a personal assistant needs to understand your emotions. Nonverbal communication makes up the majority of communication, so by using nonverbal information, we can bring ChatGPT closer to being a true Personal Assistant.

Inspiration

In order to be truly effective, a personal assistant needs to understand you emotions. Nonverbal communication makes up the majority of communication, so by using nonverbal information, we can bring ChatGPT closer to being a true Personal Assistant.

What it does

We enable ChatGPT to see how you're feeling so that it can tailor it responses to your current emotional state. Our plugins analyzes your face in real time and relays your emotional state to ChatGPT.

How we built it

We created an app that runs real-time facial detection and emotional analysis and created an API to allow our plugin to receive this information. Our text stack includes Flask, Node, and Express.

Challenges we ran into

Getting facial detection and emotional analysis to work reliably and learning the nuances of ChatGPT plugin implementation.

Accomplishments that we're proud of

Our plugins accurately assess the emotional state and relays it to ChatGPT and ChatGPT actually differentiates its response in a meaningful way. We are proud that our thesis (giving ChatGPT emotional context is valuable to quality of response) was proven.

What's next for MarvinGPT

Increasing the number of modalities by which MarvinGPT assesses your emotional state (i.e. text analysis, vocal tone analysis) and increasing the variety of emotions that MarvinGPT can detect.

The text stack includes Flask, Node, and Express.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published