Provide SWAP over NFS

[rodrigorc]

The Raspberry, particularly model A, is a low memory device, and given that the SD card is also quite slow, the ability to swap over NFS may be valuable to some users.

Proper support for this feature is available since 3.6 1. Just compile the kernel with NFS_SWAP option set. The overhead to people not using it should be minimal.

Thank you in advance.

[popcornmix]

NFS file access is much slower than sdcard, particularly for random access 4K reads/writes which is mostly what swap will be doing.

Have you tried this? I imagine it will be painfully slow.

[rodrigorc]

Have you tried this? I imagine it will be painfully slow.
I guess that depends on your SD and your network so YMMV.

Obviously, I cannot test the swap from my raspberry, because there is not support, but I can do some benchmarchs. The scenario is:

• Raspberry Pi model B, linux 3.10.36+
• cheapo SD 8G
• NFS server over local network (100 Gb link, I think), ArchLinux 3.14.1

$ time dd if=/dev/zero of=nfs_file count=$((1024*32)) bs=4k
32768+0 records in
32768+0 records out
134217728 bytes (134 MB) copied, 18.7157 s, 7.2 MB/s

real    0m18.747s
user    0m0.080s
sys 0m1.220s

$ time dd if=/dev/zero of=sd_file count=$((1024*32)) bs=4k
32768+0 records in
32768+0 records out
134217728 bytes (134 MB) copied, 81.8568 s, 1.6 MB/s

real    1m21.895s
user    0m0.100s
sys 0m1.480s

As you can see, performance is about 4.5 times better on NFS than in SD! Yes, it is linear and not random access, but I think that will be something to be handled in the server side. I mean, if the NFS server has plenty of RAM, the swap file will live in the server buffer cache and the access pattern would not matter so much, will it?

Maybe I could use some oflags=xxx to make the test more meaningful?

Naturally, not everybody will benefit from this feature, but somebody (me!) probably will.

Another advantage of swap-over-nfs is that my SD has little free space, but I have practically unlimited NFS space available.

[ghollingworth]

http://www.raspberrypi.org/documentation/linux/kernel/building.md

Go build the kernel with it enabled and give us an idea of what the performance is like and what the difference is in free memory when enabled (but not used) just to understand what the effect is for other users who don't need it.

Gordon

[rodrigorc]

@ghollingworth: Ok, I did what you suggested and compiled the kernel with and without CONFIG_NFS_SWAP. I couldn't test it yet, because my rpi is currently being used ;-)...

Anyway, a few obvervations I made:

• The compiled code of the kernel and all the modules is exactly the same.
• That is not surprising, given that the CONFIG_NFS_SWAP is used in just 3 small functions, all of them in the nfsd.ko module, and only used when swapping. The difference in code size is probably less than the segment code aligment. This is because the swap code relies on direct access to the volume, and that code is already there.

So, my estimations are:

• Cost when nfsd.ko is not loaded: 0
• Cost when nfsd.ko is loaded but nfs-swap not used: 0

Actually, thinking about my benchmarks above, they make sense: the RPi ethernet is 100 Mbps, that is roughly, 10MB/s. Take some overhead here and there and you get my 7.2 MB/s. Cheapest SD cards are about 2 MB/s [1], hence my 1.6 MB/s.

Yeah, I know, swapping at 7.2 MB/s must be painful... but swapping at 1.6 MB/s is so much worse!

[popcornmix]

A decent sdcard can provide over 20MB/s (50MHz clock with 4-bit bus), and has much lower latency (important for random access 4K reads/writes).
(Try a swag store NOOBs sdcard which has good random access 4K reads/writes).

I agree that the additional code is trivial, so I don't mind including it in next firmware update.

I would be interested in a proper benchmark (e.g. run a few copies of midori on a complex site so swap file is busy, and time how long it takes to refresh).
Try that with nfs swap, and good and bad sdcard swap.

[popcornmix]

This should be in next firmware tree.
Try "sudo BRANCH=next rpi-update"
to test.

[rodrigorc]

I've just updated my RPi to the 'next' branch, but alas, the kernel option is not yet available :-(

$ uname -a
Linux raspberrypi 3.12.18+ #673 PREEMPT Thu Apr 24 18:34:24 BST 2014 armv6l GNU/Linux
$ zgrep NFS_SWAP /proc/config.gz 
# CONFIG_NFS_SWAP is not set

[popcornmix]

Should be in latest master branch firmware now.