DietPi-Software | Home Assistant: Further install enhancements and add workarounds for external bugs

CHANGELOG.txt

@@ -11,6 +11,7 @@ Changes / Improvements / Optimisations:
 - DietPi-Software | NAA daemon: Latest version is now scraped from base URL and installed automatically. Many thanks to @Heroldgray for doing this suggestion: https://github.com/MichaIng/DietPi/issues/3191
 - DietPi-Software | Home Assistant: Python3.7.5 will be installed for the pyenv from now on due to Python3.6 deprecation. Many thanks to @sdomotica for providing this information: https://github.com/MichaIng/DietPi/issues/3219
 - DietPi-Software | Home Assistant: When selecting for install, one will be informed about possibly long install/build times. Many thanks to @Gill-Bates for doing this suggestion to avoid users assuming that their install process is stuck: https://github.com/MichaIng/DietPi/issues/2897
+- DietPi-Software | Home Assistant: Vastly reduced Home Assistant dependencies and install complexity. It does not run inside a virtualenv of pyenv anymore, since it has already its unique pyenv Python instance. pyenv is still required due to the large amount of specific module versions, required for HA and its integrations. The alternative would be a venv from a globally installed Python instance, but this requires further thoughts and discussion. The amount of development libraries and tools, installed via APT, has been updated and reduced to a minimum to build Python and Home Assistant inside the pyenv without errors and warnings. There is a slightly increased chance that certain HA components need additional development packages. To cover this, two new settings have been added to dietpi.txt: "SOFTWARE_HOMEASSISTANT_APT_DEPS" and "SOFTWARE_HOMEASSISTANT_PIP_DEPS". Add a space-separated list of APT packages and pip modules respectively, to have those installed automatically with Home Assistant. As well activating the pyenv environment from console is now easier possible via "source /home/homeassistant/pyenv-activate.sh" as user "homeassistant".
 
 Bug Fixes:
 - DietPi-PREP | Resolved an issue, where in rare cases a wrong $PATH variable could break command calls. Many thanks to @dtm2001 for reporting this issue: https://github.com/MichaIng/DietPi/issues/3206

dietpi.txt

@@ -200,18 +200,18 @@ CONFIG_NTP_MIRROR=debian.pool.ntp.org
 # - Eg: SOFTWARE_EMONHUB_APIKEY=b4dfmk2o203mmxx93a
 SOFTWARE_EMONHUB_APIKEY=
 
-# VNC Server options
+# VNC Server
 SOFTWARE_VNCSERVER_WIDTH=1280
 SOFTWARE_VNCSERVER_HEIGHT=720
 SOFTWARE_VNCSERVER_DEPTH=16
 SOFTWARE_VNCSERVER_DISPLAY_INDEX=1
 SOFTWARE_VNCSERVER_SHARE_DESKTOP=0
 
-# Optional username for ownCloud/Nextcloud admin account, the default is 'admin'. Applied during install.
+# ownCloud/Nextcloud
+# - Optional username for admin account, the default is 'admin', applied during install
 SOFTWARE_OWNCLOUD_NEXTCLOUD_USERNAME=admin
-
-# Optional data directory for ownCloud/Nextcloud, default is '/mnt/dietpi_userdata/owncloud_data' respectively '/mnt/dietpi_userdata/nextcloud_data'. Applied during install.
-# - This option is for advanced users. For full compatibility, please keep this options defaults, and, use dietpi-drive_manager to move the DietPi user data location.
+# - Optional data directory, default is "/mnt/dietpi_userdata/owncloud_data" respectively "/mnt/dietpi_userdata/nextcloud_data", applied during install
+#	NB: This option is for advanced users. For full compatibility, please keep this options defaults, and, use dietpi-drive_manager to move the DietPi user data location.
 SOFTWARE_OWNCLOUD_DATADIR=/mnt/dietpi_userdata/owncloud_data
 SOFTWARE_NEXTCLOUD_DATADIR=/mnt/dietpi_userdata/nextcloud_data
 
@@ -221,15 +221,22 @@ SOFTWARE_WIFI_HOTSPOT_SSID=DietPi-HotSpot
 SOFTWARE_WIFI_HOTSPOT_KEY=dietpihotspot
 SOFTWARE_WIFI_HOTSPOT_CHANNEL=3
 
-# X.org options
+# X.org
 # - DPI 96(default) 120(+25%) 144(+50%) 168(+75%) 192(+100%)
 SOFTWARE_XORG_DPI=96
 
-# Chromium options
+# Chromium
 SOFTWARE_CHROMIUM_RES_X=1280
 SOFTWARE_CHROMIUM_RES_Y=720
 SOFTWARE_CHROMIUM_AUTOSTART_URL=https://dietpi.com
 
+# Home Assistant
+# - Optional Python build dependencies and modules, possibly required for certain HA components
+#	Space separated list (no quotation!), will be installed together with Home Assistant automatically, if present
+SOFTWARE_HOMEASSISTANT_APT_DEPS=
+#	Add Python modules with version string at best, e.g.: firstModule==1.2.3 secondModule==4.5.6
+SOFTWARE_HOMEASSISTANT_PIP_DEPS=
+
 #------------------------------------------------------------------------------------------------------
 # D I E T - P I
 # Dev settings

dietpi/dietpi-software

@@ -1828,10 +1828,6 @@ DietPi-Software will decrypt and use it for software installs. You can change it
 		aSOFTWARE_TYPE[$software_id]=0
 		aSOFTWARE_CATEGORY_INDEX[$software_id]=18
 		aSOFTWARE_ONLINEDOC_URL[$software_id]='p=70#p70'
-		aSOFTWARE_REQUIRES_GIT[$software_id]=1
-		aSOFTWARE_REQUIRES_BUILDESSENTIAL[$software_id]=1
-		aSOFTWARE_REQUIRES_SQLITE[$software_id]=1
-		aSOFTWARE_REQUIRES_FFMPEG[$software_id]=1
 		#------------------
 		software_id=27
 
@@ -1849,8 +1845,8 @@ DietPi-Software will decrypt and use it for software installs. You can change it
 
 		aSOFTWARE_NAME[$software_id]='CloudPrint'
 		aSOFTWARE_DESC[$software_id]='print server for google cloud print'
-		aSOFTWARE_CATEGORY_INDEX[$software_id]=19
 		aSOFTWARE_TYPE[$software_id]=0
+		aSOFTWARE_CATEGORY_INDEX[$software_id]=19
 		aSOFTWARE_ONLINEDOC_URL[$software_id]='p=6630#p6630'
  		aSOFTWARE_REQUIRES_RSYSLOG[$software_id]=1 # Not required, but comes in as package dep
 		#------------------
@@ -5356,7 +5352,7 @@ Package: wireguard wireguard-dkms wireguard-tools\nPin: release n=bullseye\nPin-
 			[[ -d /etc/couchpotato ]] && rm -R /etc/couchpotato
 			mv CouchPotato* /etc/couchpotato
 
-			pip install --upgrade pyopenssl
+			pip install -U pyopenssl
 
 		fi
 
@@ -5853,7 +5849,7 @@ If you want to update ${aSOFTWARE_NAME[$software_id]}, please use it's internal
 			git clone -b voicekit "$INSTALL_URL_ADDRESS" $G_FP_DIETPI_USERDATA/voice-recognizer-raspi
 			cd $G_FP_DIETPI_USERDATA/voice-recognizer-raspi
 
-			pip3 install --upgrade pip virtualenv
+			pip3 install -U pip virtualenv
 			virtualenv --system-site-packages -p python3 env
 			env/bin/pip install -r requirements.txt
 
@@ -6190,7 +6186,7 @@ Mycroft AI requires at least 2 GiB memory on first start. We will now increase y
 		if (( ${aSOFTWARE_INSTALL_STATE[$software_id]} == 1 )); then
 
 			Banner_Installing
-			G_AGI build-essential make autoconf automake
+			G_AGI build-essential automake
 
 		fi
 
@@ -6549,51 +6545,65 @@ Mycroft AI requires at least 2 GiB memory on first start. We will now increase y
 			Banner_Installing
 
 			local ha_user='homeassistant'
-			local ha_userroot="/home/$ha_user"
-			local ha_srvroot='/srv/homeassistant'
-			local ha_pyenv_activation="export PATH=\"$ha_userroot/.pyenv/bin:\$PATH\"; eval \"\$(pyenv init -)\"; eval \"\$(pyenv virtualenv-init -)\""
-			local ha_python_version='3.7.5'
+			local ha_home="/home/$ha_user"
+			local ha_pyenv_activation="source $ha_home/pyenv-activate.sh"
+			local ha_python_version='3.8.0'
 
 			G_DIETPI-NOTIFY 2 "ha_user: $ha_user"
-			G_DIETPI-NOTIFY 2 "ha_userroot: $ha_userroot"
-			G_DIETPI-NOTIFY 2 "ha_srvroot: $ha_srvroot"
+			G_DIETPI-NOTIFY 2 "ha_home: $ha_home"
 			G_DIETPI-NOTIFY 2 "ha_pyenv_activation: $ha_pyenv_activation"
 			G_DIETPI-NOTIFY 2 "ha_python_version: $ha_python_version"
 
-			# Install needed libraries
-			G_AGI cmake daemon gcc nmap net-tools swig uuid-dev libc-ares-dev libgnutls28-dev libgnutlsxx28 libglib2.0-dev libudev-dev libusb-1.0-0 libssl-dev libffi-dev libbz2-dev zlib1g-dev libreadline-dev libsqlite3-dev libncurses5-dev libncursesw5-dev libmariadbclient-dev
-
-			# Setup the user account information
-			local usercmd='useradd -rmU'
+			# User
+			local usercmd='useradd -rMU'
 			getent passwd $ha_user &> /dev/null && usercmd='usermod -a'
-			$usercmd -G dietpi,dialout -d $ha_userroot -s $(command -v nologin) $ha_user
-
-			mkdir -p $ha_srvroot
-			chown $ha_user:$ha_user $ha_srvroot
-
-			# Install pyenv
-			su -s /bin/dash -c "cd $ha_userroot; curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash" $ha_user
+			$usercmd -G dialout -d $ha_home -s $(command -v nologin) $ha_user
 
-			# Install Python which is needed for HA.
-			su -s /bin/dash -c "cd $ha_userroot; $ha_pyenv_activation; pyenv install $ha_python_version" $ha_user
+			# Start with fresh instance, to allow clean pyenv and Python updates and fix broken instances. All userdata and configs are preserved in: /mnt/dietpi_userdata/homeassistant
+			[[ -d $ha_home/.pyenv ]] && rm -R $ha_home/.pyenv
+			[[ -d '/srv/homeassistant' ]] && rm -R /srv/homeassistant # pre-v6-27
 
-			# Make the virtual environment.
-			su -s /bin/dash -c "cd $ha_srvroot; $ha_pyenv_activation; pyenv virtualenv $ha_python_version homeassistant-$ha_python_version" $ha_user
-			su -s /bin/dash -c "cd $ha_srvroot; $ha_pyenv_activation; pyenv local homeassistant-$ha_python_version" $ha_user
-			su -s /bin/dash -c "cd $ha_srvroot; $ha_pyenv_activation; pyenv local" $ha_user
+			# APT dependencies
+			# - MariaDB support: (( $G_DISTRO < 5 )) && G_AGI libmariadbclient-dev || G_AGI libmariadb-dev
+			# - Read custom build dependencies
+			local custom_apt_deps=$(sed -n '/^[[:blank:]]*SOFTWARE_HOMEASSISTANT_APT_DEPS=/{s/^[^=]*=//p;q}' /DietPi/dietpi.txt)
+			DEPS_LIST="gcc libc6-dev make zlib1g-dev libbz2-dev libreadline-dev libssl-dev libsqlite3-dev libffi-dev $custom_apt_deps"
 
-			# Install Home Assistant and extra modules.
-			su -s /bin/dash -c "cd $ha_srvroot; $ha_pyenv_activation; pyenv activate homeassistant-$ha_python_version; pip3 install colorlog PyMySQL mysqlclient" $ha_user
-			su -s /bin/dash -c "cd $ha_srvroot; $ha_pyenv_activation; pyenv activate homeassistant-$ha_python_version; pip3 install -U homeassistant" $ha_user
+			# Install pyenv to $ha_home
+			mkdir -p $ha_home
+			Download_Install 'https://github.com/pyenv/pyenv/archive/master.tar.gz'
+			mv pyenv-master $ha_home/.pyenv
+			chown -R $ha_user:$ha_user $ha_home
 
-			# Generate the scripts to launch HA using pyenv.
+			# Generate script to activate pyenv: This must be sourced from the originating shell, hence it does not require execute permissions.
 			echo "#!/bin/dash
-cd $ha_srvroot
-$ha_pyenv_activation
-pyenv activate homeassistant-$ha_python_version
-hass -c '$ha_userroot/.homeassistant'" > $ha_srvroot/homeassistant-start.sh
-			#su -s /bin/dash -c "$ha_srvroot/homeassistant-start.sh" $ha_user
-			chmod +x $ha_srvroot/homeassistant-start.sh
+if [ $(whoami) != $ha_user ]; then
+	echo '[FAILED] This pyenv must be activated as user \"homeassistant\". Aborting...'
+	kill -INT \$\$
+fi
+cd $ha_home
+export PATH=\"$ha_home/.pyenv/bin:\$PATH\"
+eval \"\$(pyenv init -)\"" > pyenv-activate.sh
+
+			# Install Python with Home Assistant module into pyenv
+			# - Read custom Python modules
+			local custom_pip_deps=$(sed -n '/^[[:blank:]]*SOFTWARE_HOMEASSISTANT_PIP_DEPS=/{s/^[^=]*=//p;q}' /DietPi/dietpi.txt)
+			# - MariaDB support: pip3 install mysqlclient|PyMySQL
+			# - Workaround: Install home-assistant-frontend: https://github.com/home-assistant/home-assistant/issues/28361#issuecomment-550070511
+			sudo -u $ha_user dash -c "
+. $ha_home/pyenv-activate.sh
+pyenv install $ha_python_version
+pyenv local $ha_python_version
+pip3 install -U pip
+[ -z "$custom_pip_deps" ]] || pip3 install $custom_pip_deps
+pip3 install homeassistant home-assistant-frontend==20191025.1"
+
+			# Generate the script to launch HA using pyenv
+			echo "#!/bin/dash
+. $ha_home/pyenv-activate.sh
+hass -c '$G_FP_DIETPI_USERDATA/homeassistant'" > $ha_home/homeassistant-start.sh
+			chmod +x $ha_home/homeassistant-start.sh
+			#sudo -u $ha_user $ha_home/homeassistant-start.sh
 
 		fi
 
@@ -11732,11 +11742,11 @@ _EOF_
 			cat << _EOF_ > /etc/systemd/system/home-assistant.service
 [Unit]
 Description=Home Assistant (DietPi)
-After=network.target
+After=network.target mariadb.service
 
 [Service]
 User=homeassistant
-ExecStart=/srv/homeassistant/homeassistant-start.sh
+ExecStart=/home/homeassistant/homeassistant-start.sh
 
 [Install]
 WantedBy=multi-user.target
@@ -13930,22 +13940,24 @@ _EOF_
 		if (( ${aSOFTWARE_INSTALL_STATE[$software_id]} == -1 )); then
 
 			Banner_Uninstalling
-			#apt-mark auto cmake daemon gcc nmap net-tools swig uuid-dev libc-ares-dev libgnutls28-dev libgnutlsxx28 libglib2.0-dev libudev-dev libusb-1.0-0 libssl-dev libffi-dev libbz2-dev zlib1g-dev libreadline-dev libsqlite3-dev libncurses5-dev libncursesw5-dev
+			#apt-mark auto gcc libc6-dev make zlib1g-dev libbz2-dev libreadline-dev libssl-dev libsqlite3-dev libffi-dev
+			# Pre-v6.27: cmake daemon gcc nmap net-tools swig uuid-dev libc-ares-dev libgnutls28-dev libgnutlsxx28 libglib2.0-dev libudev-dev libusb-1.0-0 libssl-dev libffi-dev libbz2-dev zlib1g-dev libreadline-dev libsqlite3-dev libncurses5-dev libncursesw5-dev
 
-			# Remove the service.
+			# Remove the service
 			if [[ -f '/etc/systemd/system/home-assistant.service' ]]; then
 
 				systemctl disable --now home-assistant
-				rm /etc/systemd/system/home-assistant.service
+				rm -R /etc/systemd/system/home-assistant.service*
 
 			fi
 
-			# Remove the user and all files. This removed pyenv for this user as well.
+			# Remove the user and all files which removes his home + pyenv incl. HA as well
 			getent passwd homeassistant &> /dev/null && userdel -rf homeassistant
 			getent group homeassistant &> /dev/null && groupdel homeassistant
+			[[ -d '/srv/homeassistant' ]] && rm -R /srv/homeassistant # pre-v6-27
 
-			# Remove installation of HA.
-			[[ -d '/srv/homeassistant' ]] && rm -R /srv/homeassistant
+			G_DIETPI-NOTIFY 2 'Home Assistant data and settings are not removed: /mnt/dietpi_userdata/homeassistant
+ - You might want to do this manually: rm -R /mnt/dietpi_userdata/homeassistant'
 
 		fi