Skip to content

Commit

Permalink
Deploying to gh-pages from @ be334fb 🚀
Browse files Browse the repository at this point in the history
  • Loading branch information
@bryce13950
bryce13950 committed Sep 10, 2024
1 parent b4f53bb commit 0510ecf
Show file tree
Hide file tree
Showing 78 changed files with 3,965 additions and 3,973 deletions.
4 changes: 2 additions & 2 deletions _static/coverage/d_37285d613390727b_can_be_used_as_mlp_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_gated_mlp_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -166,7 +166,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_gated_mlp_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_37285d613390727b_gated_mlp_4bit_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_mlp_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -168,7 +168,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_mlp_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_37285d613390727b_gated_mlp_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_gated_mlp_4bit_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -164,7 +164,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_gated_mlp_4bit_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_37285d613390727b_mlp_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_moe_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -140,7 +140,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_37285d613390727b_moe_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_37285d613390727b_moe_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_db46118ef83ad831_pos_embed_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -204,7 +204,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_db46118ef83ad831_pos_embed_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_65d4430f90bfb219_activation_function_factory_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_65d4430f90bfb219_mlp_factory_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -128,7 +128,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_65d4430f90bfb219_mlp_factory_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_65d4430f90bfb219_mlp_factory_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_head_detector_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -112,7 +112,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_head_detector_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_712808f24eb400fe___init___py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_c1ea89878f9b2ac7___init___py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand All @@ -91,7 +91,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_c1ea89878f9b2ac7___init___py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_af97b5493da09a14_ActivationCache_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_FactoredMatrix_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -1192,7 +1192,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_FactoredMatrix_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_af97b5493da09a14_FactoredMatrix_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedEncoder_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -365,7 +365,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedEncoder_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_af97b5493da09a14_HookedEncoderDecoder_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedTransformer_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -507,7 +507,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedTransformer_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
4 changes: 2 additions & 2 deletions _static/coverage/d_af97b5493da09a14_HookedEncoder_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedEncoderDecoder_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -461,7 +461,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_HookedEncoderDecoder_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
8 changes: 4 additions & 4 deletions _static/coverage/d_af97b5493da09a14_HookedTransformerConfig_py.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_SVDInterpreter_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
<aside class="hidden">
<button type="button" class="button_next_chunk" data-shortcut="j"/>
Expand Down Expand Up @@ -162,7 +162,7 @@ <h2>
<p class="pln"><span class="n"><a id="t78" href="#t78">78</a></span><span class="t"><span class="str"> attention</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t79" href="#t79">79</a></span><span class="t"><span class="str"> attn_types (List[str], *optional*): the types of attention to use for</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t80" href="#t80">80</a></span><span class="t"><span class="str"> local attention</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t81" href="#t81">81</a></span><span class="t"><span class="str"> weight_init_mode (str): the initialization mode to use for the</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t81" href="#t81">81</a></span><span class="t"><span class="str"> init_mode (str): the initialization mode to use for the</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t82" href="#t82">82</a></span><span class="t"><span class="str"> weights. Only relevant for custom models, ignored for pre-trained.</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t83" href="#t83">83</a></span><span class="t"><span class="str"> We now support 'gpt2', 'xavier_uniform', 'xavier_normal', 'kaiming_uniform',</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t84" href="#t84">84</a></span><span class="t"><span class="str"> 'kaiming_normal'. MuP support to come. Defaults to 'gpt2'.</span>&nbsp;</span><span class="r"></span></p>
Expand All @@ -184,7 +184,7 @@ <h2>
<p class="pln"><span class="n"><a id="t100" href="#t100">100</a></span><span class="t"><span class="str"> Used to set sources of randomness (Python, PyTorch and NumPy) and to initialize weights.</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t101" href="#t101">101</a></span><span class="t"><span class="str"> Defaults to None. We recommend setting a seed, so your experiments are reproducible.</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t102" href="#t102">102</a></span><span class="t"><span class="str"> initializer_range (float): The standard deviation of the normal used to</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t103" href="#t103">103</a></span><span class="t"><span class="str"> initialise the weights, initialized to 0.8 / sqrt(d_model). If weight_init_mode is</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t103" href="#t103">103</a></span><span class="t"><span class="str"> initialise the weights, initialized to 0.8 / sqrt(d_model). If init_mode is</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t104" href="#t104">104</a></span><span class="t"><span class="str"> 'xavier_uniform' or 'xavier_normal', this value is instead treated as the `gain` parameter for the weight</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t105" href="#t105">105</a></span><span class="t"><span class="str"> initialisation (a constant factor to scale the weights by). Defaults to -1.0, which means not set.</span>&nbsp;</span><span class="r"></span></p>
<p class="pln"><span class="n"><a id="t106" href="#t106">106</a></span><span class="t"><span class="str"> init_weights (bool): Whether to initialize the weights. Defaults to</span>&nbsp;</span><span class="r"></span></p>
Expand Down Expand Up @@ -444,7 +444,7 @@ <h2>
<a id="nextFileLink" class="nav" href="d_af97b5493da09a14_SVDInterpreter_py.html">&#xbb; next</a>
&nbsp; &nbsp; &nbsp;
<a class="nav" href="https://coverage.readthedocs.io/en/7.4.4">coverage.py v7.4.4</a>,
created at 2024-09-05 17:34 +0000
created at 2024-09-10 17:09 +0000
</p>
</div>
</footer>
Expand Down
Loading

0 comments on commit 0510ecf

Please sign in to comment.