From ffafcdf2946842d345e2d2a8f7a6ec3ca0923ab4 Mon Sep 17 00:00:00 2001 From: bingwang Date: Thu, 14 Apr 2022 00:56:00 -0700 Subject: [PATCH 1/5] Add FLAGS table Signed-off-by: bingwang --- ...ontrol-sonic-behaviors-with-sonic-flags.md | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md diff --git a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md new file mode 100644 index 0000000000..5422f8eb3c --- /dev/null +++ b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md @@ -0,0 +1,108 @@ +# Control SONiC behaviors with FLAGS table + +## 1 Table of Content ### + +- [Revision](#11-revision) +- [Scope](#2-scope) +- [Definitions/Abbreviations](#3-definitionsabbreviations) +- [Overview](#4-overview) +- [Design](#5-design) +- [Change required](#6-change-required) +- [Test requirement](#7-test-requirement) + + +### 1.1 Revision ### +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | | Bing Wang | Initial version | + + +## 2 Scope ## + +This document covers high level design of `FLAGS` table in SONiC. + +## 3 Definitions/Abbreviations ## + + +| Term | Meaning | +|:--------:|:---------------------------------------------:| +| | | + + +## 4 Overview + +A number of flags are required to turn on/off certain feature or control the behaviors of various features in SONiC. Currently, these flags are put into `DEVICE_METADATA` table. + +``` + "DEVICE_METADATA": { + "localhost": { + "default_bgp_status": "down", + "default_pfcwd_status": "enable", + "synchronous_mode": "enable", + "dhcp_server": "enable" + } + } +``` +As a result, the `DEVICE_METADATA` table is inflating rapidly as we are having more and more flags, although these flags seem not to be categorized into `DEVICE_METADATA` + +To have a better management of the flags, a new table `FLAGS` is introduced in this design. + +## 5 Design ## + +### 5.1 DB Schema + +A new table `FLAGS` is added into config_db. +``` + key = FLAGS + + ;field = value + FLAG_NAME = 1*255VCHAR ; FLAG_NAME must be unique, the value is a string, which can be 'enable'/'disable', 'down'/'up' or any string. +``` +Below is a sample of `FLAGS` table + +``` +"FLAGS": { + "default_bgp_status": "down", + "default_pfcwd_status": "enable", + "synchronous_mode": "enable", + "dhcp_server": "enable" + } +``` + +### 5.2 How to update flags in `FLAGS` table + +#### 5.2.1 Set default value with `init_cfg.json` + +The default vaule of flags in `FLAGS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. + +#### 5.2.2 Parse from `minigraph.xml` when loading minigraph + +For the flags that can be changed by reconfiguration, we can update entries in `minigraph.xml`, and parse the new values in to config_db with minigraph parser at reloading minigraph. + +#### 5.2.3 Update value directly in db memory +For some behavior change, we may don't have to interrupt dataplane. To support controlling SONiC behavior on-the-fly, we can update the value of flags in memory with tools like `sonic-cfggen`, `configlet` or `config apply-patch`. + +### 5.3 How to consume flags in `FLAGS` table + +#### 5.3.1 Consume at service startup or reload +All of the flags in `FLAGS` table can be consumed at service startup or reload as we do now. We can use the flags to render templates or control the running path of code. + +#### 5.3.2 Consume on-the-fly without interrupting traffic +The `FLAGS` table can be subscribed by components that are interested on the flags. Hence, the in-memory change of flags will be consumed by running service, and take effect without reloading if possible. + +## 6 Change required ## +### 6.1 Template update +1. Templates that generate default values in `DEVICE_METADATA|localhost` table are required to be updated. The generated flags will be put into `FLAGS` table now. +2. Templates that depend on `DEVICE_METADATA|localhost` table are required to be updated. + +### 6.2 Yang model update +A new Yang model is to be added to restrict the valid flags in `FLAGS` table. The existing entries for flags in current [sonic-device_metadata.yang](https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/yang-models/sonic-device_metadata.yang) are to be removed. + +### 6.3 Code change +1. Update `db_migrator.py` to migrate flags from `DEVICE_METADATA|localhost` table to `FLAGS` table. +2. Update the code of component to subscribe `FLAGS` table to watch the update. + +## 7 Test requirement +TBA + + \ No newline at end of file From 8f25113ea7d8b26793eb5eec7df40ce7551176d9 Mon Sep 17 00:00:00 2001 From: bingwang Date: Mon, 18 Apr 2022 02:24:37 -0700 Subject: [PATCH 2/5] Add clarification for init_cfg Signed-off-by: bingwang --- .../control-sonic-behaviors-with-sonic-flags.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md index 5422f8eb3c..30fd5c4702 100644 --- a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md +++ b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md @@ -73,8 +73,20 @@ Below is a sample of `FLAGS` table #### 5.2.1 Set default value with `init_cfg.json` -The default vaule of flags in `FLAGS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. +The default value of flags in `FLAGS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. +If the values in `config_db.json` is changed by user, it will not be rewritten back by `init_cfg.json` as `config_db.json` is loaded after `init_cfg.json` in [docker_image_ctl.j2](https://github.com/Azure/sonic-buildimage/blob/master/files/build_templates/docker_image_ctl.j2) + +``` +if [ -r /etc/sonic/config_db$DEV.json ]; then + if [ -r /etc/sonic/init_cfg.json ]; then + $SONIC_CFGGEN -j /etc/sonic/init_cfg.json -j /etc/sonic/config_db$DEV.json --write-to-db + else + $SONIC_CFGGEN -j /etc/sonic/config_db$DEV.json --write-to-db + fi +fi +``` +For example, the value of `default_bgp_status` is down in `init_cfg.json` if `shutdown_bgp_on_start` is set to `y` when image is being built. If we modify the value of `default_bgp_status` in `config_db.json` to `up`, it will keep `up`. #### 5.2.2 Parse from `minigraph.xml` when loading minigraph For the flags that can be changed by reconfiguration, we can update entries in `minigraph.xml`, and parse the new values in to config_db with minigraph parser at reloading minigraph. From 12f4c2a81cb020f9a68bb835763ab2ce40ea19aa Mon Sep 17 00:00:00 2001 From: bingwang Date: Mon, 18 Apr 2022 23:54:22 -0700 Subject: [PATCH 3/5] List existing flags Signed-off-by: bingwang --- .../control-sonic-behaviors-with-sonic-flags.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md index 30fd5c4702..616fbdd796 100644 --- a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md +++ b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md @@ -74,7 +74,6 @@ Below is a sample of `FLAGS` table #### 5.2.1 Set default value with `init_cfg.json` The default value of flags in `FLAGS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. - If the values in `config_db.json` is changed by user, it will not be rewritten back by `init_cfg.json` as `config_db.json` is loaded after `init_cfg.json` in [docker_image_ctl.j2](https://github.com/Azure/sonic-buildimage/blob/master/files/build_templates/docker_image_ctl.j2) ``` @@ -111,8 +110,18 @@ The `FLAGS` table can be subscribed by components that are interested on the fla A new Yang model is to be added to restrict the valid flags in `FLAGS` table. The existing entries for flags in current [sonic-device_metadata.yang](https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/yang-models/sonic-device_metadata.yang) are to be removed. ### 6.3 Code change -1. Update `db_migrator.py` to migrate flags from `DEVICE_METADATA|localhost` table to `FLAGS` table. -2. Update the code of component to subscribe `FLAGS` table to watch the update. +1. Update `db_migrator.py` to migrate flags from `DEVICE_METADATA|localhost` table to `FLAGS` table. Current flags include + +|Flag|Source| +|--|--| +|default_bgp_status| From init_cfg.json, the value is determined by option shutdown_bgp_on_start when image being built| +|default_pfcwd_status|From init_cfg.json, the value is determined by option enable_pfcwd_on_start when image being built| +|synchronous_mode|From init_cfg.json, the value is determined by option include_p4rt when image being built| +|buffer_model|From init_cfg.json, the value is determined by option default_buffer_model when image being built| +|dhcp_server| Parse from minigraph.xml| + + +2. Add code to subscribe `FLAGS` table to get the update notification for components that is interested in the `FLAGS` change. Currently, all orchs and daemons don't support changing flag controlled behaviors without restarting service, so no code change is required for existing components. ## 7 Test requirement TBA From 0e81b38dd604fb7e410497bcee30138aaba2dae3 Mon Sep 17 00:00:00 2001 From: bingwang-ms <66248323+bingwang-ms@users.noreply.github.com> Date: Fri, 13 May 2022 16:30:32 +0800 Subject: [PATCH 4/5] Update control-sonic-behaviors-with-sonic-flags.md --- ...ontrol-sonic-behaviors-with-sonic-flags.md | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md index 616fbdd796..b0ecbaf9b0 100644 --- a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md +++ b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md @@ -1,4 +1,4 @@ -# Control SONiC behaviors with FLAGS table +# Control SONiC behaviors with SYSTEM_DEFAULTS table ## 1 Table of Content ### @@ -19,7 +19,7 @@ ## 2 Scope ## -This document covers high level design of `FLAGS` table in SONiC. +This document covers high level design of `SYSTEM_DEFAULTS` table in SONiC. ## 3 Definitions/Abbreviations ## @@ -45,23 +45,23 @@ A number of flags are required to turn on/off certain feature or control the beh ``` As a result, the `DEVICE_METADATA` table is inflating rapidly as we are having more and more flags, although these flags seem not to be categorized into `DEVICE_METADATA` -To have a better management of the flags, a new table `FLAGS` is introduced in this design. +To have a better management of the flags, a new table `SYSTEM_DEFAULTS` is introduced in this design. ## 5 Design ## ### 5.1 DB Schema -A new table `FLAGS` is added into config_db. +A new table `SYSTEM_DEFAULTS` is added into config_db. ``` - key = FLAGS + key = SYSTEM_DEFAULTS ;field = value FLAG_NAME = 1*255VCHAR ; FLAG_NAME must be unique, the value is a string, which can be 'enable'/'disable', 'down'/'up' or any string. ``` -Below is a sample of `FLAGS` table +Below is a sample of `SYSTEM_DEFAULTS` table ``` -"FLAGS": { +"SYSTEM_DEFAULTS": { "default_bgp_status": "down", "default_pfcwd_status": "enable", "synchronous_mode": "enable", @@ -69,11 +69,11 @@ Below is a sample of `FLAGS` table } ``` -### 5.2 How to update flags in `FLAGS` table +### 5.2 How to update flags in `SYSTEM_DEFAULTS` table #### 5.2.1 Set default value with `init_cfg.json` -The default value of flags in `FLAGS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. +The default value of flags in `SYSTEM_DEFAULTS` table can be set in `init_cfg.json` and loaded into db at system startup. These flags are usually set at image being build, and are unlikely to change at runtime. If the values in `config_db.json` is changed by user, it will not be rewritten back by `init_cfg.json` as `config_db.json` is loaded after `init_cfg.json` in [docker_image_ctl.j2](https://github.com/Azure/sonic-buildimage/blob/master/files/build_templates/docker_image_ctl.j2) ``` @@ -93,24 +93,24 @@ For the flags that can be changed by reconfiguration, we can update entries in ` #### 5.2.3 Update value directly in db memory For some behavior change, we may don't have to interrupt dataplane. To support controlling SONiC behavior on-the-fly, we can update the value of flags in memory with tools like `sonic-cfggen`, `configlet` or `config apply-patch`. -### 5.3 How to consume flags in `FLAGS` table +### 5.3 How to consume flags in `SYSTEM_DEFAULTS` table #### 5.3.1 Consume at service startup or reload -All of the flags in `FLAGS` table can be consumed at service startup or reload as we do now. We can use the flags to render templates or control the running path of code. +All of the flags in `SYSTEM_DEFAULTS` table can be consumed at service startup or reload as we do now. We can use the flags to render templates or control the running path of code. #### 5.3.2 Consume on-the-fly without interrupting traffic -The `FLAGS` table can be subscribed by components that are interested on the flags. Hence, the in-memory change of flags will be consumed by running service, and take effect without reloading if possible. +The `SYSTEM_DEFAULTS` table can be subscribed by components that are interested on the flags. Hence, the in-memory change of flags will be consumed by running service, and take effect without reloading if possible. ## 6 Change required ## ### 6.1 Template update -1. Templates that generate default values in `DEVICE_METADATA|localhost` table are required to be updated. The generated flags will be put into `FLAGS` table now. +1. Templates that generate default values in `DEVICE_METADATA|localhost` table are required to be updated. The generated flags will be put into `SYSTEM_DEFAULTS` table now. 2. Templates that depend on `DEVICE_METADATA|localhost` table are required to be updated. ### 6.2 Yang model update -A new Yang model is to be added to restrict the valid flags in `FLAGS` table. The existing entries for flags in current [sonic-device_metadata.yang](https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/yang-models/sonic-device_metadata.yang) are to be removed. +A new Yang model is to be added to restrict the valid flags in `SYSTEM_DEFAULTS` table. The existing entries for flags in current [sonic-device_metadata.yang](https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/yang-models/sonic-device_metadata.yang) are to be removed. ### 6.3 Code change -1. Update `db_migrator.py` to migrate flags from `DEVICE_METADATA|localhost` table to `FLAGS` table. Current flags include +1. Update `db_migrator.py` to migrate flags from `DEVICE_METADATA|localhost` table to `SYSTEM_DEFAULTS` table. Current flags include |Flag|Source| |--|--| @@ -121,9 +121,9 @@ A new Yang model is to be added to restrict the valid flags in `FLAGS` table. Th |dhcp_server| Parse from minigraph.xml| -2. Add code to subscribe `FLAGS` table to get the update notification for components that is interested in the `FLAGS` change. Currently, all orchs and daemons don't support changing flag controlled behaviors without restarting service, so no code change is required for existing components. +2. Add code to subscribe `SYSTEM_DEFAULTS` table to get the update notification for components that is interested in the `SYSTEM_DEFAULTS` change. Currently, all orchs and daemons don't support changing flag controlled behaviors without restarting service, so no code change is required for existing components. ## 7 Test requirement TBA - \ No newline at end of file + From 7def9117f8ec8ba4330519729700fc5962ece5c9 Mon Sep 17 00:00:00 2001 From: bingwang Date: Wed, 18 May 2022 15:10:46 +0000 Subject: [PATCH 5/5] Add an example Signed-off-by: bingwang --- ...ontrol-sonic-behaviors-with-sonic-flags.md | 33 +++++++++++++++---- 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md index b0ecbaf9b0..51015a5fe8 100644 --- a/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md +++ b/doc/sonic-flags/control-sonic-behaviors-with-sonic-flags.md @@ -53,19 +53,28 @@ To have a better management of the flags, a new table `SYSTEM_DEFAULTS` is intro A new table `SYSTEM_DEFAULTS` is added into config_db. ``` - key = SYSTEM_DEFAULTS + key = SYSTEM_DEFAULTS|feature_name; feature name must bt unique ;field = value - FLAG_NAME = 1*255VCHAR ; FLAG_NAME must be unique, the value is a string, which can be 'enable'/'disable', 'down'/'up' or any string. + status = 1*255VCHAR ; The value is a string, which can be 'enable'/'disable', 'down'/'up' or any string. + custom_field = 1*255VCHAR ; The name of custom_field can be any custom string. ``` Below is a sample of `SYSTEM_DEFAULTS` table ``` "SYSTEM_DEFAULTS": { - "default_bgp_status": "down", - "default_pfcwd_status": "enable", - "synchronous_mode": "enable", - "dhcp_server": "enable" + "tunnel_qos_remap": { + "status": "enabled" + } + "default_bgp_status": { + "status": "down" + } + "synchronous_mode": { + "status": "enable" + } + "dhcp_server": { + "status": "enable" + } } ``` @@ -89,7 +98,19 @@ For example, the value of `default_bgp_status` is down in `init_cfg.json` if `sh #### 5.2.2 Parse from `minigraph.xml` when loading minigraph For the flags that can be changed by reconfiguration, we can update entries in `minigraph.xml`, and parse the new values in to config_db with minigraph parser at reloading minigraph. +For example, to turn on/off the `tunnel_qos_remap` feature, a new section will be defined in `minigraph.xml` +``` + + + + TunnelQosRemapEnabled + True + + + +``` +The new section will be parsed by `minigraph.py`, and the parsed value will be merged with the values defined in `init_cfg.json`, and finally written into `config_db`. If there are duplicated entries in `init_cfg.json` and `minigraph.xml`, the values in `minigraph.xml` will overwritten the values defined in `init_cfg.json`. #### 5.2.3 Update value directly in db memory For some behavior change, we may don't have to interrupt dataplane. To support controlling SONiC behavior on-the-fly, we can update the value of flags in memory with tools like `sonic-cfggen`, `configlet` or `config apply-patch`.