Parenting Multitask Helper with Open CV
.
├── HERMES_APPLICATION
│ └── app
│ └── src
│ ├── main
│ ├── AndroidManifest.xml
│ ├── java
│ │ └── com
│ │ └── example
│ │ └── hermes_application
│ │ ├── BabyVideoActivity.java
│ │ ├── IntroActivity.java
│ │ ├── MainActivity.java
│ │ └── MusicActivity.java
│ └── res
│ └── layout
│ ├── activity_baby_video.xml
│ ├── activity_intro.xml
│ ├── activity_main.xml
│ └── activity_music.xml
│
├── raspi
│ ├── asset
│ ├── baby.py
│ ├── camera.py
│ ├── constant.py
│ ├── led.py
│ ├── main.py
│ ├── mobile.py
│ ├── music.py
│ ├── network.py
│ ├── processor
│ │ ├── face_detector.py
│ │ ├── __init__.py
│ │ ├── model
│ │ │ └── haarcascades
│ │ │ ├── haarcascade_eye.xml
│ │ │ └── haarcascade_frontalface_default.xml
│ │ └── simple_streamer.py
│ │
│ ├── templates
│ │ └── index.html
│ ├── utils.py
│ └── videostream.py
├── README.md
code editor : VScode with ssh connection to pi
model name : Raspberry Pi 4 Model B Rev 1.5
CPU: ARM Cortex-A72 1.5GHz
RAM: 2GB
Linux version 5.15.76-v7l+ (dom@buildbot) (arm-linux-gnueabihf-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1597 SMP Fri Nov 4 12:14:58 GMT 2022
'com.android.application': version '7.3.0'
'com.android.library' : version '7.3.0'
Compile SDK : 32
min SDK : 21
target SDK : 32
Java Version : VERSION_1_8
xml version : 1.0
encoding : utf-8
Open CV
opencv-python : 4.6.0.6
mediapipe-rpi4 : 0.8.8
Python : 3.7.3 etc.
Manifest
INTERNET Permission
ACCESS_NETWORK_STATE Permission
3D printing
Calculate the eye ratio with eye's width and height.
| eye ratio | |
|---|---|
| Close | >=4.5 |
| Open | else |
camera.py put baby's state in queue.
| Open | 0 |
|---|---|
| Close | 1 |
count rate = Open/Queue's size
| count rate | |
|---|---|
| Wake | >=0.8 |
| ASLEEP | >=0.4 and Init stat = Wake |
| AWAKE | >=0.4 and Init stat = Sleep |
| Sleep | else |
| None | No detect above threshold |
Init_Stat only can be Wake/Sleep
If baby is wake, blink count decreasing, it goes Asleep.
If baby is sleep, blink ocunt increasing, it goes Awake.
STATS
Wake, Awake, Asleep, Sleep
No APP Signal
| BABY | GPIO |
|---|---|
| WAKE | LED ON, Mobile On, Music on, Lamp more bright |
| SLEEP | All GPIO is Off |
| ASLEEP | Mobile On, Led off, Music off, Lamp darker |
GPIO operation as manipulated in the app, regardless of auto-run.
When the child state changes, it switches back to automatic mode.
Wake -> Asleep Sleep -> Awake
| State Notification and Control | |
|---|---|
| State of Baby | Asleep / Awake / Sleep / Wake / Detect X |
| Mobile | ON / OFF |
| LED | ON / OFF |
| Music | Play / Stop & choose music |
| Illuminance | Level 0 - 10 |
| Mode | Auto / User |
If click State of baby, you can watch real-time video
Choi Yunseo : camera detecting, realtime status judgement, 3D print
Lee Jieun : Android application, socket
Lee yein : GPIO, socket, overall running of the project
raspi/camera.py : facial detecting with picamera using Mediapipe
python3 camera.py
raspi/baby.py : realtime baby's status judgement linked with camera.py and Gpio operation
python3 main.py
raspi/led.py : led on off, pwm
python3 led.py
raspi/mobile.py : mobile on off
python3 mobile.py
raspi/music.py : music on off and change
python3 music.py
raspi/network.py : socket and app signal handling
python3 network.py
- Camera detecting (camera.py)
(1) MediaPipe facemesh
https://google.github.io/mediapipe/solutions/face_mesh.html
(2) openCV with haarcascade ( less accurate than (1) ) - 3D model printing
[ Cubicon style-plus / Suwon Makerspace ]
(1) Raspberry Pi 4B TURBO case With Integrated Camera Module
https://www.youtube.com/watch?v=kXgKs1Zv43U
https://www.thingiverse.com/thing:4912025
(2) LED lamp
https://www.thingiverse.com/thing:1531729 - WebView
https://1d1cblog.tistory.com/21 - socket
https://developmentdiary.tistory.com/510 - raspberry pi
https://github.com/eleparts/raspi-AdvancedKit