-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathindex.html
250 lines (242 loc) · 9.69 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
<!doctype html>
<html lang="en">
<head>
<!-- Required meta tags -->
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name=”description” content="Interactive visualization of a neural network. "/>
<!-- Bootstrap CSS -->
<link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css"
integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm"
crossorigin="anonymous">
<link rel="stylesheet" href="main.css">
<link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
<link rel="icon" href="favicon.ico" type="image/x-icon">
<script src="./dist/main.js"></script>
<script
src='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML'
async></script>
<script defer data-domain="nnplayground.com"
src="https://analytics.gifmemes.io/js/plausible.js"></script>
<title>Neural network visualized</title>
</head>
<body>
<div class="container mt-3" style="min-width: 992px;">
<h1>Neural network playground</h1>
<p class="lead">Visualization of a simple neural network for educational purposes.</p>
<hr>
<div class="container-fluid" id="controls">
<div class="row">
<div class="col-4">
<h5>Network size</h5>
<table id="layer-controls">
</table>
</div>
<div class="col-4 pr-5">
<h5>Learning settings</h5>
<div class="row">
<label for="rate-input" class="control-label col-8">Learning rate:</label>
<select class="form-control form-control-sm col-4" id="rate-input">
<option value="0.001">0.001</option>
<option value="0.003">0.003</option>
<option value="0.01">0.01</option>
<option value="0.03">0.03</option>
<option value="0.1">0.1</option>
<option value="0.3">0.3</option>
<option value="1" selected="true">1</option>
<option value="3">3</option>
<option value="10">10</option>
<option value="30">30</option>
</select>
</div>
<div class="row mt-1 mb-1">
<label for="regularization-input" class="col-8">Regularization:</label>
<select class="form-control form-control-sm col-4" id="regularization-type-input">
<option value="none">None</option>
<option value="L1">L1</option>
<option value="L2">L2</option>
</select>
</div>
<div class="row">
<label for="regularization-input" class="col-8">Regularization rate:</label>
<select class="form-control form-control-sm col-4" id="regularization-rate-input">
<option value="0.00001">0.00001</option>
<option value="0.00003">0.00003</option>
<option value="0.0001">0.0001</option>
<option value="0.0003" selected="true">0.0003</option>
<option value="0.001">0.001</option>
<option value="0.003">0.003</option>
<option value="0.01">0.01</option>
<option value="0.03">0.03</option>
<option value="0.1">0.1</option>
<option value="0.3">0.3</option>
<option value="1">1</option>
<option value="3">3</option>
<option value="10">10</option>
<option value="30">30</option>
</select>
</div>
</div>
<div class="col-4 pr-5">
<h5>Train</h5>
<div class="row">
<label for="iters-input" class="col-8">Num. of iterations:</label>
<input class="form-control form-control-sm col-4" id="iters-input" type="number"
name="rate" min="0" max="20000"
value="5000" step="1000">
</div>
<div class="row">
<label for="iter-output" class="col-8">Current iteration:</label>
<b id="iter-output" class="col-4">0</b>
</div>
</div>
</div>
<hr>
<div class="row">
<div class="col-4">
<h5>Training set input</h5>
<textarea class="form-control" id="training-set-input" rows="6">
[
[[0.2,0.1,0.7],[1,0]],
[[0.6,0.1,0.3],[0,1]],
[[0.4,0.3,0.3],[1,0]],
[[0, 0.8, 0.2],[0,1]]
]</textarea>
<button type="button" class="btn btn-outline-primary btn-sm mt-3"
onclick="setTrainingData()">Set training
data
</button>
</div>
<div class="col-8">
<h5>Training set</h5>
<table class="table table-sm table-bordered table-hover table-striped">
<thead class="thead-dark">
<tr id="training-set-neurons-output">
</tr>
</thead>
<tbody id="training-set-data-output">
</tbody>
</table>
<div><i>Hint: Click on an entry in the table to activate an input</i></div>
</div>
</div>
<div class="row mt-3">
<div class="col-6">
<h5>Weights</h5>
<textarea class="form-control" id="training-set-weights" rows="6">
[
[[0.1, 0.2, 0.6, 0.3], [0.4, 0.3, 0.1, 0.2]],
[[0.2, 0.1, 0.5], [0.1, 0.4, 0.8], [0.5, 0.2, 0.3]],
[[0.2, 0.1, 0.5, 0.1], [0.1, 0.4, 0.8, 0.4]]
]</textarea>
<button type="button" class="btn btn-outline-primary btn-sm mt-3"
onclick="setWeights()">Set
Weights
</button>
<button type="button" class="btn btn-outline-secondary btn-sm mt-3"
onclick="randomWeights()">Random Weights
</button>
</div>
</div>
<hr>
<div class="row mb-3">
<div class="col-md-auto">
<label style="margin: 8px">
<input type="checkbox" style="margin-right: 4px" id="train-repeat-chckbx">Repeat
</label>
<button type="button" id="train-btn" class="btn btn-primary mr-2" onclick="train(true)">
Train
</button>
<button type="button" class="btn btn-secondary" onclick="train(false)">Step</button>
<button type="button" class="btn btn-danger" onclick="reset()">Reset</button>
</div>
<div class="col-md-auto" style="padding-top: 5px;">
Cost:
<b id="cost"></b>
</div>
</div>
</div>
<div id="input-controls"></div>
<div style="width: 100%; text-align: center; margin-bottom: 1em;">
<canvas id="content" width="982" height="600"
style="border: 5px solid #2E282A; display: inline">Sorry,
you need
a better browser to see the neural net.
</canvas>
</div>
<div>
<h2>What is this?</h2>
<p>This is implementation of neural network with back-propagation. There aren't any special
tricks, it's as
simple neural
network as it gets.</p>
<h5>Cost function</h5>
<p>The cost is defined as \(C = \frac{1}{2 \times
sampleCnt}\sum^{sampleCnt}_{m=1}(\sum^{outputSize}_{n=1}(neruon_n-target_n)^2)\).
In words: Error is defined as \((value - target)^2\). To get error of neural network for one
training sample,
you
simply add errors of all output neurons. The total cost is then defined as average error of
all training
samples.
</p>
<h5>Forward propagation</h5>
<p>
Let's say that the value of connection is the connection's weight (how wide it is) times the
first connected
neuron. To calculate
the value of some neuron you add the values of all incoming connections and apply the
<a target="_blank" href="https://www.desmos.com/calculator/dw9fmqwlmn">sigmoid</a> function to
that sum. Other
activation functions are possible, but I have not implemented them yet.
</p>
<h5>Back propagation</h5>
<p>The cost function defined above is a function dependend on weights of connections in the same
way as \(f(x, y)
= x^2
+ y^2\) is dependend on x and y. In the beginning, the weights are random. Let's say x = 5 and
y = 3. The cost
at
this point would be 25 + 9 = 34, which we want to get to 0. Now we take the derivate with
respect to each of
these
weights, which tells us how to adjust the wieghts to minimize the function. \(\frac{\partial
f(x, y)}{\partial
x}
= 2x\), \(\frac{\partial f(x, y)}{\partial y} = 2y\). Now that we have the derivatives, we
know the "direction"
in
which to change the weights. \(x_{new} = x_{old} - rate \times 2x = 5 - 0.1 \times 2 \times 5
= 4\) and that's
a
little bit closer to the desired 0 result of f(x, y). The rate is necessary to avoid stepping
over the minimum.</p>
<p>
In practice is the computation of the derivatives is a little bit harder, but all you need to
know is the
<i>chain rule</i>. I highly recommend
<a target="_blank" href="https://www.youtube.com/watch?v=aircAruvnKk">3blue1brown's series</a>
and
<a target="_blank"
href="https://web.archive.org/web/20150317210621/https://www4.rgu.ac.uk/files/chapter3%20-%20bp.pdf">this
paper</a> for better understanding.
</p>
<p>
If you have enjoyed this project, you might also like my <a href="http://gifmemes.io"
target="_blank">tool for creating
gif memes</a> and <a href="http://vocabuo.com"
target="_blank">Vocabuo - an app for learning vocabulary</a>.
</p>
<footer class="futter m-5" style="text-align: center;">
<br> Created by
<a target="_blank"
href="mailto:[email protected]?Subject=Nerual%20network%20-%20suggestion/hate/or%20just%20saying%20hi"
target="_top">Petr Kubeš</a>,
<a target="_blank" href="https://github.com/PetrKubes97/ts-neural-network" target="_top">source
code</a>
</footer>
</div>
</div>
</body>
</html>