DietPi-Software | Re-allow to install webserver, database and PHP independently from menu

[DarrenSchwartz]

Creating a bug report/issue:

Required Information:

• DietPi version | cat /DietPi/dietpi/.version

G_DIETPI_VERSION_CORE=6
G_DIETPI_VERSION_SUB=19
G_DIETPI_VERSION_RC=7
G_GITBRANCH=master
G_GITOWNER=Fourdee

• Distro version | echo $G_DISTRO_NAME or cat /etc/debian_version
  9.6

• Kernel version | uname -a
  Linux rpi3 4.14.79-v7+ Letsencrypt supports Free Noip.com Dynamic DNS #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l GNU/Linux

• SBC device | echo $G_HW_MODEL_DESCRIPTION or (EG: RPi3)
  RPi 3 Model B (armv7l)

• Power supply used | (EG: 5V 1A RAVpower)
  5V 3.5 A USB power station

• SDcard used | (EG: SanDisk ultra)
  Sandisk Ultra 16 GB

Additional Information (if applicable):

• Software title | (EG: Nextcloud)
  Certbot

• Was the software title installed freshly or updated/migrated?
  Yes, the Dietpi image was freshly installed. Certbot and a few other optimized software titles were installed from dietpi-software.

• Can this issue be replicated on a fresh installation of DietPi?
  Yes.

• dietpi-bugreport ID

Steps to reproduce:

1. Install Certbot, without having installed a web server.
2. Run dietpi-letsencrypt.

Expected behaviour:

• Dietpi-letsencrypt should run through without error
• If a web server is not installed when installing Certbot, it should install one based on the user's selected web server preference.

Actual behaviour:

• dietpi-letsencrypt throws an error when selecting 'apply'
• the error states that a web server is not installed (because it is not).

Extra details:

• This is easy to workaround by installing another package containing a web server, but ideally, Certbot would bring along a web server. Lighttpd is also not available as a standalone install, so the user is forced to install additional packages that they may not need (php, mariadb). It would be nice to have Lighttpd available as a standalone install too.

[MichaIng]

@noGitForYou
Thanks for your suggestion.

However I think it is fine as it is, for the following reason:

• CertBot itself does not depend on a webserver. It even brings it's own internal standalone webserver that can be used to get a certificate. E.g. you have a Java or Node.js application with web interface, so do not need a dedicated webserver, but certificates.
• DietPi-LetsEncrypt is another topic. it uses CertBot to retrieve SSL certificates and installs them to an installed webserver, so it depends on CertBot as well as on a webserver to do something, that's true.
• So what we could do, is offering automated install of CertBot and the chosen webserver via dietpi-software directly from dietpi-letsencrypt, so users don't need to manually do that. But it is not 100% perfect, since currently Minio S3 is also detected as webserver by DietPi-LetsEncrypt, even that it is isn't one, just a standalone web interface application 🤔.

What I have in mind for the future of DietPi-LetsEncrypt is the following:

• Allow to choose whether to use CertBot, acme.sh (https://github.com/Fourdee/DietPi/issues/2363), create a self-singed certificate or enter the path to key+cert from external provider.
• It will download/renew the certificate independently from installed software, optionally use webroot or webserver method, if available, but otherwise use an independent method (standalone webserver or DNS validation).
• Then a separate menu allows to apply the certificate and enable SSL for installed software titles, e.g. webservers but as well other web interface software.

What is actually true and not ideal from my point of view is, that one cannot select a webserver independently currently. You can select a whole webserver stack instead (e.g. LAMP), that includes PHP and a database. In most cases you anyway want/need a whole webserver stack, but there might be cases where the webserver along would be sufficient. But to allow that, we also need to review the install script. since it currently expects at least PHP the be installed, AFAIK.

[DarrenSchwartz]

@MichaIng
Thanks for your detailed reply. I think your plan for the future makes good sense. Your last comment highlights where my real concern lies. First I got the error from dietpi-letsencrypt about not having a web server installed, and I would have happily installed Lighttpd manually from dietpi-software, but didn't want the whole stack. Especially since installing a whole stack gives a somewhat cryptic warning message that makes you worry about screwing up your config.

It seems that if you view the software list using the command line with dietpi-software list instead of dietpi-software, there is an option to just install Lighttpd and PHP together (software ID 84), but it is not visible as an option in dietpi-software. Also no entry for just installing the server without PHP. I guess its not the end of the world to install a whole stack, I'm just a minimalist by nature and I don't need the database or PHP.

[MichaIng]

@noGitForYou
To address your initial suggestion a bid, I added some more info to the DietPi-Software error prompt, if no webserver was found: Fourdee@550c549

• It also turned into a yes/no menu, which allows to directly switch to DietPi-Software.

I hope things are a bid clearer by this.

---

About the webserver topc:

The initial intention was to clean up the menu and make it easier for users to fast find and select the stack they require. If we show the webservers and databases as well in the menu, the whole webserver category will look somehow messy, also due to the chosen install IDs (which we do not want to rearrange, every time out software offer changes). So I think either the stacks or only the individual webservers, databases and PHP (no stacks then) should show up.

What you can always do to install only a single software ID: dietpi-software install 84
But currently, webservers will always install PHP as hard coded dependency.

---

Okay some thoughts:

• DietPi-Software has a search function for some time, which makes it easier to search through the large menu.
• There are indeed cases where one requires a webserver, database or PHP only.
• (Coding) We already separated webserver and PHP requirement arrays for this reason.
• For flexibility and to be consistent with the more flexible dependencies, I think we can switch/revert from webserver stack to individual webserver, database and PHP selection. If anything as dependency is missing, dietpi-software will add it automatically anyway.
• Only thing that we could enhance, although not required urgently:
  • Currently all webservers have PHP as dependency and also from config code it is expected to be installed.
  • We could make it more flexible by configure the webserver to use PHP only, if it's actually installed or marked for install.
  • But this is a larger effort, so something that would be done by times, webserver by webserver.

@Fourdee
What do you think?

[DarrenSchwartz]

@MichaIng
I think your change to the dietpi-letsencrypt error handling is a simple and sufficient solution for the letsencrypt installation issue.

As for the web server dependency issue, I guess it's important not to over complicate the config, so if PHP comes along for the ride, I could get over it. Will leave it to you guys to decide whether the benefit outweighs the extra work.

Some of my discomfort comes from this error message when selecting a stack for installation:

DietPi will automatically install a webserver stack (based on your webserver Preference) when any software that requires a webserver is selected for installation (eg: ownCloud, Pi-hole etc).
It is highly recommended that you allow DietPi to do this for you, ensuring compatibility and stability across DietPi installed programs.
Please only select a webserver stack if you specifically require it, and, no other webserver stack is installed.
TLDR: You do NOT need to select a webserver stack for installation with DietPi. Its all automatic.

So I had to select a stack because dietpi-letsencrypt did not automatically install the required web server (and the web server is not available individually for install in dietpi-software). Then this error message makes me worry that I'm screwing up my config by installing a stack manually. Maybe it's just a confusion with the wording of the error message?

Food for thought: maybe instead of selecting a stack in the "software optimized" list, the user could specify a database preference on the preceeding dietpi-software menu (the main dietpi-software menu where you specify web server preference). Then instead of a stack, dietpi-software could just install the required components when needed based on the user's preference (I think this is along the lines of what you mention above by removing the stacks and giving individual software choices).

[Fourdee]

@MichaIng

So I had to select a stack because dietpi-letsencrypt did not automatically install the required web server (and the web server is not available individually for install in dietpi-software).

Webserver stack should be a pre-req for Certbot, to be consistent with what users expect from our install system.

, I think we can switch/revert from webserver stack to individual webserver, database and PHP selection.

I believe this will open a can of worms and increase our support level. I'd personally prefer to only allow users to install a web-server stack that we pre-define.

Hmm, these should not install PHP/SQL unless they are specified.

		aSOFTWARE_REQUIRES_WEBSERVER[$software_id]=1
			  aSOFTWARE_REQUIRES_PHP[$software_id]=1
		   aSOFTWARE_REQUIRES_SQLITE[$software_id]=1

EDIT: If I remember correctly, we enabled for all, need to go through them and check if we can remove any?

[MichaIng]

@Fourdee
Jep, databases can already be skipped, PHP currently not, if webserver selected. The auto pre-req selection will select PHP with any webserver automatically, doubled, since all webservers have aSOFTWARE_REQUIRES_PHP[$software_id]=1 as well.

Webserver stack should be a pre-req for Certbot, to be consistent with what users expect from our install system.

At least currently Minio S3 object server is handled separately without any webserver. So DietPi-LetsEncrypt will not show any error, if this is installed.

However, issue should be solved with: https://github.com/Fourdee/DietPi/issues/2363#issuecomment-449649984

• So the script would allow to enable/disable HTTPS for various software titles without webserver as well, if installed. It will not depend on any webserver, but, in case of CertBot, using it's standalone module to receive the cert.
• Only downside in case of webserver is, that we need to configure HTTPS manually. But we do that for Lighttpd already and not hard to achieve for Nginx/Apache via drop-in configs, based on: https://mozilla.github.io/server-side-tls/ssl-config-generator/

I believe this will open a can of worms and increase our support level.

Not sure. I mean most users do not only install a webserver for their own manually built website, where they might need/expect PHP. Usually one installs a forum, wordpress, cloud or such a thing, which then installs everything according the dependency arrays anyway. However, currently the webserver installers expect PHP, so some coding required to separate this, more a midterm project, if any.

Another idea is moving all the webserver stuff to the other Linux software menu, making it more consistently one which contains software, that one usually doesn't want/need to install as standalone applications. With webserver choice system, database/PHP dependency arrays, everything is then correctly selected anyway 🤔.