CPU lock During the ZFS+Zvol #2272
Comments
|
I tried to replicate this on master root@iceisfunzfs: Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) The device presents a logical sector size that is smaller than Command (m for help): p Disk /dev/zvol/tank/Volume_Test: 274.9 GB, 274877906944 bytes Command (m for help): n Command (m for help): t Command (m for help): p Disk /dev/zvol/tank/Volume_Test: 274.9 GB, 274877906944 bytes /dev/zvol/tank/Volume_Test1 2048 536870911 268434432 83 Linux Command (m for help): w Calling ioctl() to re-read partition table. Allocating group tables: done root@iceisfunzfs:~# |
|
Again with block size 4096 root@iceisfunzfs: Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) The device presents a logical sector size that is smaller than Command (m for help): n Command (m for help): t Command (m for help): p Disk /dev/zvol/tank/Volume_Test2: 274.9 GB, 274877906944 bytes /dev/zvol/tank/Volume_Test2p1 2048 536870911 268434432 83 Linux Command (m for help): w Calling ioctl() to re-read partition table. Allocating group tables: done root@iceisfunzfs:~# |
|
Thanks @iceisfun , create ext4.zvol is no problem. When I try some high I/O load test that problem is happen. |
|
I am unable to create the condition you've described even under heavy load. Tomorrow I will try to recreate the issue with a different load test. I did however manage to get my system load numbers up over 85 Under this heavy load I got some different error messages. |
|
I was able to get my test machine into a non-responsive state doing the above testing with a zvol, bonnie++ and ext4 - I believe in my case the issue came down to the machine not having enough memory / zfs allowed to use too much memory |
|
Does anyone know if this is still an issue with the 0.6.3 version or newer? |
|
I am running Debian Wheezy, zfs* 0.6.3-1~wheezy Under heavy load the system becomes stalled and has to be rebooted about once every 2 weeks. This time the system came back after an hour and I found the following stacktraces. Additional information: swap space was configured according to https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-use-a-zvol-as-a-swap-device. [1013751.840885] INFO: task kswapd0:46 blocked for more than 120 seconds. |
|
FYI: Debian Wheezy uses linux-3.2.41 according to It appears that there's a trend with "stable" distributions that run with 2.6* or 3.0_, 3.2_ kernels which suffer from this |
|
FYI: |
|
I updated to a kernel 3.16 last week: This night I experienced the same problem again with the new kernel. Some hints from my monitoring:
Do you have any idea about the time correlation of these locks? |
|
@usrflo I'd suspect that it's rather high i/o load and ARC/memory pressure that leads to (coincidentally the same timepoint) when the system has issues If you're running zfs/master & spl/master also could you - in either case - post the data requested at #3235 (comment) ? Is your configuration and system software (kernel, rsync, etc.) all stock ? if not is using a patched rsync (to lower memory impact) an (temporary) option ? that would add the option to rsync can you run a kernel with http://patchwork.ozlabs.org/patch/433475/ is CONFIG_PARAVIRT_SPINLOCKS and paravirt support an option ? |
|
The condition can be induced by high stress/io load situations, I do not On Sun, Mar 29, 2015 at 5:46 AM, kernelOfTruth aka. kOT, Gentoo user <
|
|
@kernelOfTruth : thanks for so many proposals. I hesitate to test the patchset (#3235) on this productive system. If helpful for debugging I can give some more information about my system? |
|
Update: even with drop-cache the problem happened once again after the 19th of April with v0.6.3. |
|
This should be resolved in 0.6.5.x. |
and others
I am running the CentOS 6.5, ZFS-0.6.2
Here is the output for running ext4 on zvol. Please using 500G zvol and block size =4096 and try to mkfs.ext4, you will see the result. This may related to the slow mkfs.ext4.
I find the #2208 maybe the same problem whit me.
AL
The text was updated successfully, but these errors were encountered: