Skip to content

Commit

Permalink
Keras guides snapshot: Thu, 30 Sep 2021 07:02:00 GMT
Browse files Browse the repository at this point in the history
  • Loading branch information
@tfdocsbot
tfdocsbot committed Sep 30, 2021
1 parent 5677c3d commit e9aee83
Showing 1 changed file with 56 additions and 46 deletions.
102 changes: 56 additions & 46 deletions site/en/guide/keras/custom_layers_and_models.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -356,45 +356,55 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fb08b1a45d22"
"id": "0697afb97bc1"
},
"outputs": [],
"source": [
"# At instantiation, we don't know on what inputs this is going to get called\n",
"linear_layer = Linear(32)\n",
"\n",
"# The layer's weights are created dynamically the first time the layer is called\n",
"y = linear_layer(x)"
"y = linear_layer(x)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ddd7e8b22641"
"id": "51b81f42b466"
},
"source": [
"Implementing `build()` separately as shown above nicely separates creating weights\n",
"only once from using weights in every call. However, for some advanced custom\n",
"layers, it can become impractical to separate the state creation and computation.\n",
"Layer implementers are allowed to defer weight creation to the first `__call__()`,\n",
"but need to take care that later calls use the same weights. In addition, since\n",
"`__call__()` is likely to be executed for the first time inside a `tf.function`,\n",
"any variable creation that takes place in `__call__()` should be wrapped in a`tf.init_scope`."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0b7a45f57610"
},
"source": [
"## Layers are recursively composable\n",
"\n",
"If you assign a Layer instance as an attribute of another Layer, the outer layer\n",
"will start tracking the weights of the inner layer.\n",
"will start tracking the weights created by the inner layer.\n",
"\n",
"We recommend creating such sublayers in the `__init__()` method (since the\n",
"sublayers will typically have a build method, they will be built when the\n",
"outer layer gets built)."
"We recommend creating such sublayers in the `__init__()` method and leave it to\n",
"the first `__call__()` to trigger building their weights."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9561cbf2fc60"
"id": "1aaaf82ab8ce"
},
"outputs": [],
"source": [
"# Let's assume we are reusing the Linear class\n",
"# with a `build` method that we defined above.\n",
"\n",
"\n",
"class MLPBlock(keras.layers.Layer):\n",
" def __init__(self):\n",
" super(MLPBlock, self).__init__()\n",
Expand All @@ -419,7 +429,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "496736d98a62"
"id": "2bf11b296bd2"
},
"source": [
"## The `add_loss()` method\n",
Expand All @@ -433,7 +443,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "084d56602ca4"
"id": "ba2782dc0879"
},
"outputs": [],
"source": [
Expand All @@ -451,7 +461,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "5009ff0d1feb"
"id": "a883b230a9e9"
},
"source": [
"These losses (including those created by any inner layer) can be retrieved via\n",
Expand All @@ -464,7 +474,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5bff6d59aea7"
"id": "b56d223a30cd"
},
"outputs": [],
"source": [
Expand All @@ -491,7 +501,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "36751ebe3363"
"id": "0809dec680ff"
},
"source": [
"In addition, the `loss` property also contains regularization losses created\n",
Expand All @@ -502,7 +512,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9327d3b581f8"
"id": "41016153e983"
},
"outputs": [],
"source": [
Expand All @@ -528,7 +538,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "99d502b8899c"
"id": "589465e06e4f"
},
"source": [
"These losses are meant to be taken into account when writing training loops,\n",
Expand Down Expand Up @@ -556,7 +566,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "7fa2db4a631d"
"id": "7fb41ca8c3b0"
},
"source": [
"For a detailed guide about writing training loops, see the\n",
Expand All @@ -570,7 +580,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5c767ccdfc43"
"id": "769bc6612ebf"
},
"outputs": [],
"source": [
Expand All @@ -595,7 +605,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "c753fcbc1818"
"id": "149c71e442bb"
},
"source": [
"## The `add_metric()` method\n",
Expand All @@ -613,7 +623,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "85dad61dc160"
"id": "bfb2df515096"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -641,7 +651,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "cd8807cb9cbc"
"id": "e68f88373800"
},
"source": [
"Metrics tracked in this way are accessible via `layer.metrics`:"
Expand All @@ -651,7 +661,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "475df7270265"
"id": "1834d74450b6"
},
"outputs": [],
"source": [
Expand All @@ -668,7 +678,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "9eda5113fd18"
"id": "546cfbd4ea05"
},
"source": [
"Just like for `add_loss()`, these metrics are tracked by `fit()`:"
Expand All @@ -678,7 +688,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "97f767613953"
"id": "f5e74cb4da34"
},
"outputs": [],
"source": [
Expand All @@ -700,7 +710,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "4bdbac3f6c85"
"id": "4012fa8683e5"
},
"source": [
"## You can optionally enable serialization on your layers\n",
Expand All @@ -714,7 +724,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b359ed5289a8"
"id": "0a720cbd5f54"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -750,7 +760,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "78b207f7acbc"
"id": "1b43aad6c145"
},
"source": [
"Note that the `__init__()` method of the base `Layer` class takes some keyword\n",
Expand All @@ -763,7 +773,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "00a3432cd28c"
"id": "0cbad8a6e6cd"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -800,7 +810,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "ad5d100cc969"
"id": "2421f80b5b86"
},
"source": [
"If you need more flexibility when deserializing the layer from its config, you\n",
Expand All @@ -819,7 +829,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "741c6d134d65"
"id": "3d7e2304a047"
},
"source": [
"## Privileged `training` argument in the `call()` method\n",
Expand All @@ -838,7 +848,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "67ca741d0cfb"
"id": "a169812c2c00"
},
"outputs": [],
"source": [
Expand All @@ -856,7 +866,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "5284e22677da"
"id": "9e1482c9f010"
},
"source": [
"## Privileged `mask` argument in the `call()` method\n",
Expand All @@ -880,7 +890,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "bf87358118de"
"id": "344110f9e134"
},
"source": [
"## The `Model` class\n",
Expand Down Expand Up @@ -922,7 +932,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "cd8cfcb1bb2b"
"id": "09caa642b72e"
},
"source": [
"```python\n",
Expand Down Expand Up @@ -952,7 +962,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "b817a4de8c5d"
"id": "a2e32d225a1b"
},
"source": [
"## Putting it all together: an end-to-end example\n",
Expand All @@ -978,7 +988,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "18842173f875"
"id": "56aaae7af872"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -1057,7 +1067,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "40384d934b3c"
"id": "2f8ae035a7c9"
},
"source": [
"Let's write a simple training loop on MNIST:"
Expand All @@ -1067,7 +1077,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "c37fef01d4bc"
"id": "40f11d1ef3bc"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -1111,7 +1121,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "65e5faeb0029"
"id": "f0d65fae5d3d"
},
"source": [
"Note that since the VAE is subclassing `Model`, it features built-in training\n",
Expand All @@ -1122,7 +1132,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1e98ba7ebdb8"
"id": "5af13f70d528"
},
"outputs": [],
"source": [
Expand All @@ -1137,7 +1147,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "42ee3169e70c"
"id": "d34b7ba21662"
},
"source": [
"## Beyond object-oriented development: the Functional API\n",
Expand All @@ -1155,7 +1165,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b8fe39f892c7"
"id": "be77fc8f9b26"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -1194,7 +1204,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "0f4db7df7eb5"
"id": "e2f135ea7cf5"
},
"source": [
"For more information, make sure to read the [Functional API guide](https://www.tensorflow.org/guide/keras/functional/)."
Expand Down

0 comments on commit e9aee83

Please sign in to comment.