forked from koalaman/linuxatemyram.com
-
Notifications
You must be signed in to change notification settings - Fork 0
/
play.html
271 lines (202 loc) · 10.2 KB
/
play.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Experiments and fun with the Linux disk cache</title>
<link rel="stylesheet" href="github-markdown.css">
<link rel="stylesheet" href="normalize.min.css">
<link rel="stylesheet" href="common.css">
</head>
<body bgcolor="white" text="black">
<article class="markdown-body" style="font-size: 1.1em">
<h1>Experiments and fun with the Linux disk cache</h1>
<p>Hopefully you are now convinced that Linux didn't just <a href="index.html">eat your ram</a>. Here are some interesting things you can do to learn how the disk cache works.</p>
<p><i>Note: The examples shown are for and from 2009 era hardware without SSDs. You may have to multiply several of the example numbers here by 10 to see an appreciable effect.</i></p>
<h2>Effects of disk cache on application memory allocation</h2>
<p>Since I've already promised that disk cache doesn't prevent applications from getting the memory they want, let's start with that. Here is a C app (<code>munch.c</code>) that gobbles up as much memory as it can, or to a specified limit:</p>
<pre>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int main(int argc, char** argv) {
int max = -1;
int mb = 0;
char* buffer;
if(argc > 1)
max = atoi(argv[1]);
while((buffer=malloc(1024*1024)) != NULL && mb != max) {
memset(buffer, 0, 1024*1024);
mb++;
printf("Allocated %d MB\n", mb);
}
return 0;
}
</pre>
<p>Running out of memory isn't fun, but the OOM killer should end just this process and hopefully the rest will remain undisturbed. We'll definitely want to disable swap for this, or the app will gobble up that as well.</p>
<pre>
$ <b>sudo swapoff -a</b>
$ <b>free -m</b>
</pre>
<p><i>(note that your <code>free</code> output could be different, and have an 'available' column instead of a '-/+' row)</i></p>
<pre>
total used free shared buffers cached
Mem: 1504 1490 14 0 24 809
-/+ buffers/cache: 656 848
Swap: 0 0 0
$ <b>gcc munch.c -o munch</b>
$ <b>./munch</b>
Allocated 1 MB
Allocated 2 MB
(...)
Allocated 877 MB
Allocated 878 MB
Allocated 879 MB
Killed
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 650 854 0 1 67
-/+ buffers/cache: 581 923
Swap: 0 0 0
</pre>
<p>Even though it said 14MB "free", that didn't stop the application from grabbing 879MB. Afterwards, the cache is pretty empty<sup><a href="#footnote2">2</a></sup>, but it will gradually fill up again as files are read and written. Give it a try. </p>
<h2>Effects of disk cache on swapping</h2>
<p>I also said that disk cache won't cause applications to use swap. Let's try that as well, with the same <code>munch</code> app as in the last experiment. This time we'll run it with swap on, and limit it to a few hundred megabytes:</p>
<pre>
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 1490 14 0 10 874
-/+ buffers/cache: 605 899
Swap: 2047 6 2041
$ <b>./munch 400</b>
Allocated 1 MB
Allocated 2 MB
(...)
Allocated 399 MB
Allocated 400 MB
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 1090 414 0 5 485
-/+ buffers/cache: 598 906
Swap: 2047 6 2041
</pre>
<p><code>munch</code> ate 400MB of ram, which was taken from the disk cache without resorting to swap. Likewise, we can fill the disk cache again and it will not start eating swap either. If you run <code>watch free -m</code> in one terminal, and <code>find . -type f -exec cat {} + > /dev/null</code> in another, you can see that "cached" will rise while "free" falls. After a while, it tapers off but swap is never touched<sup><a href="#footnote1">1</a></sup>.</p>
<h2>Clearing the disk cache</h2>
<p>For experimentation, it's very convenient to be able to drop the disk cache. For this, we can use the special file <code>/proc/sys/vm/drop_caches</code>. By writing <code>3</code> to it, we can clear most of the disk cache:</p>
<pre>
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 1471 33 0 36 801
-/+ buffers/cache: 633 871
Swap: 2047 6 2041
$ <b>echo 3 | sudo tee /proc/sys/vm/drop_caches </b>
3
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 763 741 0 0 134
-/+ buffers/cache: 629 875
Swap: 2047 6 2041
</pre>
<p>Notice how "buffers" and "cached" went down, free mem went up, and free+buffers/cache stayed the same.</p>
<h2>Effects of disk cache on load times</h2>
<p>Let's make two test programs, one in Python and one in Java. Python and Java both come with pretty big runtimes, which have to be loaded in order to run the application. This is a perfect scenario for disk cache to work its magic.</p>
<pre>
$ <b>cat hello.py</b>
print "Hello World! Love, Python"
$ <b>cat Hello.java</b>
class Hello {
public static void main(String[] args) throws Exception {
System.out.println("Hello World! Regards, Java");
}
}
$ <b>javac Hello.java</b>
$ <b>python hello.py</b>
Hello World! Love, Python
$ <b>java Hello</b>
Hello World! Regards, Java
</pre>
<p>Our hello world apps work. Now let's drop the disk cache, and see how long it takes to run them.</p>
<pre>
$ <b>echo 3 | sudo tee /proc/sys/vm/drop_caches</b>
3
$ <b>time python hello.py</b>
Hello World! Love, Python
real 0m1.026s
user 0m0.020s
sys 0m0.020s
$ <b>time java Hello</b>
Hello World! Regards, Java
real 0m2.174s
user 0m0.100s
sys 0m0.056s
$
</pre>
<p>Wow. 1 second for Python, and 2 seconds for Java? That's a lot just to say hello. However, now all the file required to run them will be in the disk cache so they can be fetched straight from memory. Let's try again:</p>
<pre>
$ <b>time python hello.py</b>
Hello World! Love, Python
real 0m0.022s
user 0m0.016s
sys 0m0.008s
$ <b>time java Hello</b>
Hello World! Regards, Java
real 0m0.139s
user 0m0.060s
sys 0m0.028s
$
</pre>
<p>Yay! Python now runs in just 22 milliseconds, while java uses 139ms. That's 45 and 15 times faster! All your apps get this boost automatically!</p>
<h2>Effects of disk cache on file reading</h2>
<p>Let's make a big file and see how disk cache affects how fast we can read it. I'm making a 200MB file, but if you have less free ram, you can adjust it.</p>
<pre>
$ <b>echo 3 | sudo tee /proc/sys/vm/drop_caches</b>
3
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 546 958 0 0 85
-/+ buffers/cache: 461 1043
Swap: 2047 6 2041
$ <b>dd if=/dev/zero of=bigfile bs=1M count=200</b>
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 6.66191 s, 31.5 MB/s
$ <b>ls -lh bigfile</b>
-rw-r--r-- 1 vidar vidar 200M 2009-04-25 12:30 bigfile
$ <b>free -m</b>
total used free shared buffers cached
Mem: 1504 753 750 0 0 285
-/+ buffers/cache: 468 1036
Swap: 2047 6 2041
</pre>
<p>Since the file was just written, it will go in the disk cache. The 200MB file caused a 200MB bump in "cached". Let's read it, clear the cache, and read it again to see how fast it is:</p>
<pre>
$ <b>time cat bigfile > /dev/null</b>
real 0m0.139s
user 0m0.008s
sys 0m0.128s
$ <b>echo 3 | sudo tee /proc/sys/vm/drop_caches</b>
3
$ <b>time cat bigfile > /dev/null</b>
real 0m8.688s
user 0m0.020s
sys 0m0.336s
$
</pre>
<p>That's more than fifty times faster!</p>
<h2>Conclusions</h2>
<p>The Linux disk cache is very unobtrusive. It uses spare memory to greatly increase disk access speeds, and without taking any memory away from applications. A fully used store of ram on Linux is efficient hardware use, not a warning sign.</p>
<small><strong class="text-gray-light">LinuxAteMyRam.com was presented by <a href="http://www.vidarholen.net">VidarHolen.net</a>. This site is available <a href="https://github.com/koalaman/linuxatemyram.com">on GitHub</a> for comments and PRs.</strong></small>
<hr />
<p>These pages do simplify a little:</p>
<ol>
<li id="footnote1">
<p>While newly allocated memory will always (though see point #2) be taken from the disk cache instead of swap, Linux can be configured to preemptively swap out other unused applications in the background to free up memory for cache. The is tunable through the <code>swappiness</code> setting, accessible through <code>/proc/sys/vm/swappiness</code>.</p>
<p>A server might want to swap out unused apps to speed up disk access of running ones (making the system faster), while a desktop system might want to keep apps in memory to prevent lag when the user finally uses them (making the system more responsive). This is the subject of much debate.</p>
</li>
<li id="footnote2">
<p>Some parts of the cache can't be dropped, not even to accomodate new applications. This includes <code>mmap</code>'d pages that have been mlocked by some application, dirty pages that have not yet been written to storage, and data stored in <code>tmpfs</code> (including <code>/dev/shm</code>, used for shared memory). The <code>mmap</code>'d, mlocked pages are stuck in the page cache. Dirty pages will for the most part swiftly be written out. Data in <code>tmpfs</code> will be swapped out if possible.</p>
</li>
</ol>
</article>
</body>
</html>