Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update core.py: fix docstring "y = Wx + b" -> "y = xW + b" #1784

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update core.py: fix docstring "y = Wx + b" -> "y = xW + b" #1784

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions trax/layers/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ class Dense(base.Layer):
with trainable weights. Each node in a dense layer computes a weighted sum of
all node values from the preceding layer and adds to that sum a node-specific
bias term. The full layer computation is expressed compactly in linear
algebra as an affine map `y = Wx + b`, where `W` is a matrix and `y`, `x`,
algebra as an affine map `y = xW + b`, where `W` is a matrix and `y`, `x`,
and `b` are vectors. The layer is trained, or "learns", by updating the
values in `W` and `b`.

Less commonly, a dense layer can omit the bias term and be a pure linear map:
`y = Wx`.
`y = xW`.
"""

def __init__(self,
Expand All @@ -64,8 +64,8 @@ def __init__(self,
connection weights `W` for the layer.
bias_initializer: Function that creates a vector of (random) initial
bias weights `b` for the layer.
use_bias: If `True`, compute an affine map `y = Wx + b`; else compute
a linear map `y = Wx`.
use_bias: If `True`, compute an affine map `y = xW + b`; else compute
a linear map `y = xW`.
use_bfloat16: If `True`, use bfloat16 weights instead of the default
float32; this can save memory but may (rarely) lead to numerical issues.
"""
Expand Down