Can't mount ZFS filesystems during boot

[artiomn]

Something creates а directory structure.
It may be fixed by enabling overlay mounts, but it's not an ideal solution.
I suspect, that there are shared folders mentioned in the issue 39

# systemctl status zfs-mount.service 
● zfs-mount.service - Mount ZFS filesystems
   Loaded: loaded (/lib/systemd/system/zfs-mount.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2018-04-01 11:12:24 MSK; 11min ago
     Docs: man:zfs(8)
  Process: 4080 ExecStart=/sbin/zfs mount -a (code=exited, status=1/FAILURE)
 Main PID: 4080 (code=exited, status=1/FAILURE)
      CPU: 205ms

Apr 01 11:12:24 nas systemd[1]: Starting Mount ZFS filesystems...
Apr 01 11:12:24 nas zfs[4080]: cannot mount '/tank0': directory is not empty
Apr 01 11:12:24 nas zfs[4080]: cannot mount '/tank0/docker': directory is not empty
Apr 01 11:12:24 nas systemd[1]: zfs-mount.service: Main process exited, code=exited, status=1/FAILURE
Apr 01 11:12:24 nas systemd[1]: Failed to start Mount ZFS filesystems.
Apr 01 11:12:24 nas systemd[1]: zfs-mount.service: Unit entered failed state.
Apr 01 11:12:24 nas systemd[1]: zfs-mount.service: Failed with result 'exit-code'.

[subzero79]

no, is docker. You cannot use the docker plugin in conjunction with a zfs dataset, due to how the docker dataroot setting was implemented originally (i am not gonna explain this).

This has been corrected in the docker plugin, but it hasn't been released yet. For now don't use the shared folder combo in docker plugin, use a symlink to from /var/lib/docker to whatever point you want.

Also when using docker with zfs driver, the zfs plugin after a a few containers the grid panel will start getting flooded with datasets, is not pretty and you will eventually start having timeout errors

[artiomn]

Is it docker plugin or the docker itself?

For now don't use the shared folder combo in docker plugin, use a symlink to from /var/lib/docker to whatever point you want.

Thank you, I'll try.
Will be a problem, that I have not only pool, but root on ZFS too?
I think, Docker detects ZFS and want use it as a storage backend instead of aufs.

the grid panel will start getting flooded with datasets

I have noticed.

[subzero79]

Is the docker plugin.

I know about the root pool. You already have mentioned all the problems associated with this so far.

This plugin needs to be redone completely, it has only been patched through the years to make it work. I am afraid that this plugin will not move to omv5 if a developer doesn’t come forward to redo this.

[artiomn]

Ok, thank you for the information.

I am afraid that this plugin will not move to omv5 if a developer doesn’t come forward to redo this.

This plugin is really necessary. Many people want to keep their services inside containers and not to pollute the host OS.

[subzero79]

Sorry I mean the zfs plugin. The docker plugin will continue, as it really helps to get rid of some plugins that are just a checkbox (downloaders).

[artiomn]

Yes.
But ZFS plugin too.
ZFS is a one of the killer-features in FreeNAS and OMV is commonly used because it supports ZFS out of the box.

[subzero79]

Careful with the words, OMV doesn't support zfs out of box, neither does Debian. There are many issues ATM with the plugin as you can see and no dev resources.

[artiomn]

It's an out-of-the-box solution for the end user. He can install plugin in a few clicks, and ZFS support will work. If you remove this plugin, many users will go on another products.

[ryecoaaron]

Actually, the number of OMV users using zfs is much lower than users not using zfs.

[artiomn]

Maybe. I can't argue with you, because I don't know the statistical data.
But ZFS is a trend, and I believe, that it will replace "classical chain": partitioning, LVM (and software RAID), filesystem and even LUKS.

[artiomn]

@subzero79 :

This has been corrected in the docker plugin, but it hasn't been released yet. For now don't use the shared folder combo in docker plugin, use a symlink to from /var/lib/docker to whatever point you want.

This workaround helped me, despite system root on ZFS.
Thank you, very much.

[artiomn]

The system boots normally.

[kalleyne]

I've often wondered about that myself. It would seem that based on my non-scientific viewing of forum activity, non-zfs users appear to outnumber zfs users. I guess that history has a role to play in this as zfs is "relatively" new to Linux and also there is the problem with incompatible licenses. Personally, I had to use zfs for a while before I could get accustomed to its features and potential. Given enough time, I hope that more Linux users would use zfs as its benefits outweigh the disadvantages. OMV is one of the few free, non-commercial NAS products out there based on Linux that supports zfs although in an indirect fashion. What's the best way to improve the zfs plugin? Better funding? More developers? Are we in a catch-22 situation? Because the zfs plugin is a little problematic is that the reason why zfs users are fewer in number? And because the zfs users are few in number...is that why the zfs plugin gets limited developer resources? Would a better quality zfs plugin result in more zfs users? Thanks.

…

On 04/07/2018 06:52 AM, Aaron Murray wrote: Actually, the number of OMV users using zfs is much lower than users not using zfs. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIOZ1k_BeWrMIDt-HFFtdKBYrbvnj4oXks5tmKhugaJpZM4TCwg7>.

[ryecoaaron]

I think the reason there are few zfs users on OMV is because most users don't need it (or raid) and ext4 just works. Some use something simpler for bitrot protection like snapraid. Plenty of users don't want to give up any drives for parity. And most arm users aren't going to use zfs.

As for improving the zfs plugin, I don't know who the funding would go to. subzero and myself are the only ones really doing any work on plugins at the moment and that work is usually not directed at the zfs plugin. I don't use zfs. So, that is one reason I don't work on it much. Another reason is that I think it needs to be completely re-written to use more of the OMV framework than work along side it. I don't think improving the zfs plugin would increase users.

[ryecoaaron]

This won't help zfs - openzfs/zfs#7401

[bunder2015]

Which has nothing to do with the issue reported by OP.

[ryecoaaron]

I know that. The issue has gotten off topic (which I did not cause) and it was inline with the last few posts.

[artiomn]

I think the reason there are few zfs users on OMV is because most users don't need it (or raid) and ext4 just works.

It's a low-cost and unreliable variant even for the "home" NAS.

I don't think improving the zfs plugin would increase users.

But removing the zfs plugin, will undoubtedly decrease OMV users count.

[ryecoaaron]

It's a low-cost and unreliable variant even for the "home" NAS.

What is unreliable? ext4?

But removing the zfs plugin, will undoubtedly decrease OMV users count.

No one is going to remove it but if we have no one to fix it for OMV 5.x (or even major 4.x changes), what is the alternative? I don't use zfs so I'm not a good one to work on it.

[artiomn]

What is unreliable? ext4?

Variant with a filesystem, that doesn't control files checksum and doesn't have snapshots (or you need to implement this functionality with other components, but this is redundancy).

[mdziekon]

Another reason is that I think it needs to be completely re-written to use more of the OMV framework than work along side it.

Out of curiosity, what exactly has changed in the OMV framework that makes you say that? I'm a person who has started their journey with OMV 4.x and since I want to use ZFS in OMV, I'm actually interested in helping with this project, but it's unclear to me what is actually wrong with this plugin. The fact that OMV's documentation and blog posts do not point out what has changed in the framework itself does not help either to understand the problem.

[ryecoaaron]

@mdziekon The framework really hasn't changed that much lately. The zfs plugin is just using a minimal amount. Granted it was written before the storage and filesystem classes existed. If you look at how other filesystems are implemented in OMV (https://github.com/openmediavault/openmediavault/tree/4.x/deb/openmediavault/usr/share/php/openmediavault/system) and compare how the zfs plugin has this (https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-zfs/tree/master/usr/share/omvzfs) and (https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-zfs/tree/master/usr/share/php/openmediavault/system). I just think it could be improved greatly. Hard to explain.

Documentation of low level stuff is not documented well because the few people that understand it don't have the time. You will just have to dive into like I have. Porting the luks plugin to the this framework was a good way to learn it for me.

[Tsaukpaetra]

I was wondering why I was having so much trouble. In truth it's probably in combination of using USB-attached storage and docker placed on the pool.

I would love to fixup this plugin, but sadly don't have time myself. I've starred this repo though and added it to my projects (position 91 at the moment) so maybe I'll try and help when I can.

One thing I'm doing at the moment is added a new docker.path file to delay starting docker until it's ready, but that's obviously not a generic solution. 🤔

I also made openmediavault-engined go after zfs.target, but I think that's also a hack.

[ryecoaaron]

Closing due to age.