DietPi Derivatives | Ideas/Questions about how to create those

[diveyez]

Basing Image Requested

[Fourdee]

@diveyez
? 😄
http://mediad.publicbroadcasting.net/p/wium/files/styles/x_large/public/201604/IMG_1477.JPG

[MichaIng]

@diveyez
Note that it is and was never the aim to invest much of our unpaid spare time to develop DietPi a way that you can easily create derivatives without DietPi tools/scripts on it. Instead we want to make DietPi as-is better. Also much of which makes DietPi IS indeed the scripts, especially on first boot, that automate much of the otherwise required manual system setup steps. But those again depend on the whole script base.

So as how far I understand your request of a minimal DietPi-based image for derivatives, it is simply hard to define where this basis should end and regular DietPi or the derivative respectively should begin 😉.

As a result, if you want to make a derivative, the first thing is that you decide/evaluate which parts of the regular DietPi you want/need to keep and which not, especially which system configuration/setup steps that are done by our scripts.

Always important to remember that DietPi itself relies on ready-to-boot base images (Raspbian, Debian, TinkerOS, ARMbian, ...), different ones for every SBC/device. They already ship some individual firmware/tool set in most cases, on top of default minimum Debian.

Have a look at: https://github.com/MichaIng/DietPi/blob/dev/PREP_SYSTEM_FOR_DIETPI.sh

• This is the very first thing that we do on the base images, so where "our control" starts and everything that should be required to adjust for a derivative.
• I am willing to help explaining parts if something is not clear and will help to implement PRs, e.g. to allow customising or skipping some steps in regards to derivatives. But I will not do such development myself in the near feature.
• Whether to allow running this on the SBC itself or via qemu+chroot on an external "creation" system is an independent topic I want to start working on. Any help/testing appreciated.

To automate image creation I am working on: #2693

• But this has not much to do with DietPi and can be done on any image/drive. Also it has not yet been tested on every kind of partition layout, so use with caution.

So most start needs to be done by you, e.g. PRs or fork and I am willing to help with answers or implementation.

[diveyez]

Okay. I will continue to get with Minideb developers to pursue my original path, however, I would love to make a derivative of dietpi for the SBC support within https://github.com/Meth0dOS
I decided to go with DietPI deviations because it makes sense to do so for what my vision is.

[MichaIng]

@diveyez
Yeah Minideb indeed looks like a real minimal base OS to start a derivative from. However it is made for docker, or did you manage to run the resulting image on a real machine as well?

Btw perhaps I can a better idea of your needs if you tell a bid about the aim of Meth0dOS, what it is/will be etc?

[FredericGuilbault]

I think what attract people to make derivative from dietpi is the multiple SBC support. Armbian support a lot of SBC it but it does not support RPI for technical and political reasons. Sadly RPI IS the cool kid on the block ATM and that's suicide to not support it.

AFAIK Dietpi is the only project doing it ATM. Untill some others projects pop-up. Like it or not you are our only saviour available XD

it is simply hard to define where this basis should end and regular DietPi or the derivative respectively should begin

For me there should be a layer where Dietpi make all differents OS/SBC look the same ( as much as possible/needed ). Then an other layer where Dietpi's script make dietpi what dietpi is. this is where the line is.

But it's indeed hard to define where this line is in the current codebase.

---

So most start needs to be done by you, e.g. PRs or fork

+1 , Talk is cheap, Show some code ™️

[MichaIng]

@FredericGuilbault

but it does not support RPI for technical and political reasons. Sadly RPI IS the cool kid on the block ATM and that's suicide to not support it.

I always thought the reason is that with Raspbian one already has a working and well maintained Debian based image? And ARMbian is the only Debian solution for most other SBCs, since the manufacturer images often Ubuntu only (and oder non-Debian derivatives) and mostly bad maintained, with a non-updated Linux kernel. ARMbian does a great job there, increasingly with porting 5.X mainline kernel for those SBCs as well. But for RPi it is simply no required, since due to their orders of magnitude higher selling counts they can afford great kernel and firmware development themselves.

With DietPi the aim was and is not to enable Debian on SBCs but this opposite approach of having a minimum purist base OS, still with full Debian core capabilities, systemd, APT on the one side (compare image sizes, installed package counts, initial RAM usage, active processes etc with other server/minimal images) but enable a user friendly way to turn it into any kind of server/media/download/... system without the need to do much manual research>download>build>install>configuration steps. That is something I found unique until today. All other Debian images/derivatives I found, even when called minimal/server images, are still relatively high loaded with things that most user will never need. Those I like to only have/install on demand. Minibian was there, but it is not updated for more than three years and RPi only.

For me there should be a layer where Dietpi make all differents OS/SBC look the same ( as much as possible/needed ). Then an other layer where Dietpi's script make dietpi what dietpi is. this is where the line is.

One could make a cut where it is just about purging down APT packages to the minimum, installing kernel/bootloader/firmware + base packages + WiFi from updated APT sources and doing core cleanup + configuration of systemd, cron, network, device dependant setup/config steps and such. But at least file system expansion and initial network setup needs to be done on first boot, as I believe headless first boot with SSH capability is essential, also because there are some headless-only devices (e.g. NanoPi Fire3 + NEO Air) and not everyone has or even knows what a serial console is 😉. And those configurations are currently done within our boot scripts which depend on each other, RAMdisk initially and especially dietpi-globals which are sourced by most scripts.
DietPi cron jobs, RAMlog and even RAMdisk could be quite easily removed and dependency on the latter worked around (for boot scripts), then there are no active DietPi parts anymore. The remaining ~1M scripts are optional anyway.

Yeah however what my aim is, aside from the derivatives topic, to make all our scripts and settings less intrusive, so that they can be seen as an "offer" which can be used to setup/configure the device but not as a must. This is mostly the case already but currently using our scripts (including first boot steps) in cases override custom configs (/etc/network/interfaces, fstab, ... okay not too much anymore actually) and custom configs can break our scripts. v6.26 will e.g. implement network setup via e.g. /etc/network/interfaces.d/dietpi-eth0|wlan0 etc. so that the basic /etc/network/interfaces does not need to be touched but can be used manually (or other drop-in configs) to configure e.g. VLANs or other custom network setups. The clearer we separate DietPi scripts and configs from base OS/packages ones, the clearer also a line can be defined for where Debian ends and DietPi starts. And it makes it more compatible to other 3rd party installers and such.

[FredericGuilbault]

I always thought the reason is that with Raspbian one already has a working and well maintained Debian based image.

On the technical aspect, From what I understoud, Raspbian have a closed source peice of code they call manifacturer firmware (but it do it's way over what a firmware should do). Some key components for graphical rendering and other cpu call things are hidden secretly in here. Armbian license forbid to use it as it's closed source (And armbian people would not distribute the closed source peice of code blindly). Also there is a guy in the armbian core devs team that HATE RPI with passion and I must say he have some pretty solid, reproductibles and documented arguments about RPI flaws.

All this story is on a 10+ pages tread on armbian forum titled something about support request.

[FredericGuilbault]

Yeah however what my aim is, aside from the derivatives topic, to make all our scripts and settings less intrusive, so that they can be seen as an "offer" which can be used to setup/configure the device but not as a must. This is mostly the case already but currently using our scripts (including first boot steps)

Seperating first boot script as it's own service was a huge step for me in having a clear line, What still blur it for me as someone external to Dietpi project. is the fact that dietpi ship in a broken state and require dietpi's scripts execution to first boot. and this script rely on other dietpi scripts (mainly ramfs parts ) So already you have half of dietpi loaded only to have a working OS.

I understand why it's like that and I don't think it could be done much differently. Without Changing the spirit of dietpi.

I think there have been a confusion from my part on the begining between Dietpi pretending to me a minimal pure non-bloated OS Then foundout a huge stack of scripts that can't be avoid in order to have a working OS. BUT In fact, AFAIK Repackagers like me where not the initial goal of dietpi, You do a lot to accomodate me. And I feel like im pulling on dietpi away from it's original purpus. On the other hand, If im hacking on dietpi scripts. Im better sharing it upstream with the dietpi repository so anyone can profit from it.

Im kind expecting you to have to put a stop on repackagers requests at some point to not loosing your self in everyone else needs.

The more distinct the line will be, The easyest it will be to have repackagers contributing to the minimal image AND not having messing in what/how dietpi features should be.

[MichaIng]

@FredericGuilbault
As said, the RAMdisk requirement can be relatively easily removed, for example via dietpi.txt AUTO_SETUP_RAMDISK setting that is checked within DietPi-FirstBoot and replaces the /DietPi mount point with a symlink to /boot. I am just not 100% sure currently if it can be unmounted as long as a script within it is loaded. Although the script is fully loaded to RAM via curly braces.
Or it is simply checked if RAMdisk is active and in case not, place the symlink. Then it's up to image creator to disable it. Same for RAMlog, which is enabled by default but not not require anything else, not is required by anything.

What is definitely required to run DietPi-FirstBoot is:

• /DietPi/dietpi/func/dietpi-obtain_hw_model to identify hardware and distro details.
• /DietPi/dietpi/func/dietpi-globals since a large number of scripts, called from DietPi-FirstBoot,

PreBoot can also easily be removed:

• Call /DietPi/dietpi/func/dietpi-obtain_hw_model within DietPi-FirstBoot and DietPi-Boot (when first boot has finished)
• Make CPU governor script a separate systemd unit
• The same for LED control, an I am thinking if this can be achieved via udev rules as well, so no call on boot would be required.

DietPi-Boot does now nothing more then some network failsafe steps (as result of found issue on some SBCs), waiting for network and applying time sync. When trusting in networking.service/ifupdown and having a timesync daemon active (e.g. systemd-timesyncd) then this can be skipped as well.

DietPi-PostBoot can be skipped as long as all software services are "enabled", thus started by systemd on boot.

And btw even DietPi-FirstBoot is not required to have a working system. The only thing I currently see is network being not enabled by DietPi-PREP. But this can be easily changed by having eth0 and wlan0 interfaces enabled as allow-hotplug by default. Everything else is just automation if a bunch of setup steps base on dietpi.txt settings, that one otherwise would need to do manually.

---

Less dependencies are good for DietPi in general as well, so I will anyway aim to loosen some knots, of course over time.
The whole network thing, perhaps removing the "obtain_network_details" script, is on the ToDo for v6.26. I want much more flexibility with this.

[FredericGuilbault]

I don't see AUTO_SETUP_RAMDISK in dietpi.txt does it have to be set to zero to be desactivated ?

[MichaIng]

@FredericGuilbault
Adding this was just an idea that is currently not implemented. Sorry I didn't make this clear above 😉. I just meant that it is theoretically possible to skip RAMdisk since the scripts do not care about if /DietPi is a tmpfs mount point, a symlink or regular dir.

[diveyez]

I made a derivative of dietpi, but I discontinued that project due to a death of a parent. I have since shutdown that venture because dietpi does what I need and I just do not need all the shiny toys slowing my boxes down.

[MichaIng]

@diveyez
I'm sad to hear that about your parent. Take your time for family and sorrow, that is more important than all software projects.

I will however not forget about the ideas and goals we collected here. It might take some time, especially since some disturbing current work around Buster and RPi4 release and some major software/repo changes that need to be handled/fixed. But we will go on to step by step to make the underlying system more independent from DietPi scripts, hence DietPi more compatible and flexible for customisation. Thanks for your input about that!

[diveyez]

Tell me about it, I got the hard news that I cannot do a buster to stretch full upgrade today. FML.

[MichaIng]

@diveyez
Possible it is, but Russian roulette if you will face no, minor or major issues, mostly depending on the software that is already installed, especially those outside of the Debian APT repo.

But if some tinkering is acceptable + backup done... I full-upgraded my production server from Buster to Bullseye some days ago 😄. Luckily the difference is not yet large, and I am able find and fix Bullseye issues with DietPi as fast as they appear.