From 56bfff2a5083bc084ca86175bddd05aaec800a27 Mon Sep 17 00:00:00 2001 From: Nazarii Hnydyn Date: Wed, 14 Sep 2022 20:54:42 +0300 Subject: [PATCH 1/3] [hash]: Add GH HLD. Signed-off-by: Nazarii Hnydyn --- doc/hash/hash-design.md | 584 +++++++++++++++++++++++++++++ doc/hash/images/gh_design.svg | 262 +++++++++++++ doc/hash/images/gh_swss_design.svg | 115 ++++++ doc/hash/images/gh_update_flow.svg | 248 ++++++++++++ 4 files changed, 1209 insertions(+) create mode 100644 doc/hash/hash-design.md create mode 100644 doc/hash/images/gh_design.svg create mode 100644 doc/hash/images/gh_swss_design.svg create mode 100644 doc/hash/images/gh_update_flow.svg diff --git a/doc/hash/hash-design.md b/doc/hash/hash-design.md new file mode 100644 index 0000000000..cd527f30b2 --- /dev/null +++ b/doc/hash/hash-design.md @@ -0,0 +1,584 @@ +# SONiC Generic Hash + +## High Level Design document + +## Table of contents + +- [Revision](#revision) +- [About this manual](#about-this-manual) +- [Scope](#scope) +- [Abbreviations](#abbreviations) +- [1 Introduction](#1-introduction) + - [1.1 Feature overview](#11-feature-overview) + - [1.2 Requirements](#12-requirements) + - [1.2.1 Functionality](#121-functionality) + - [1.2.2 Command interface](#122-command-interface) + - [1.2.3 Error handling](#123-error-handling) + - [1.2.4 Event logging](#124-event-logging) +- [2 Design](#2-design) + - [2.1 Overview](#21-overview) + - [2.2 SAI API](#22-sai-api) + - [2.3 Orchestration agent](#23-orchestration-agent) + - [2.3.1 Overview](#231-overview) + - [2.3.2 Switch orch](#232-switch-orch) + - [2.4 DB schema](#24-db-schema) + - [2.4.1 Config DB](#241-config-db) + - [2.4.1.1 Switch hash](#2411-switch-hash) + - [2.4.2 Data sample](#242-data-sample) + - [2.4.3 Configuration sample](#243-configuration-sample) + - [2.4.4 Initial configuration](#244-initial-configuration) + - [2.5 Flows](#25-flows) + - [2.5.1 GH update](#251-gh-update) + - [2.6 CLI](#26-cli) + - [2.6.1 Command structure](#261-command-structure) + - [2.6.2 Usage examples](#262-usage-examples) + - [2.6.2.1 Config command group](#2621-config-command-group) + - [2.6.2.2 Show command group](#2622-show-command-group) + - [2.7 YANG model](#27-yang-model) + - [2.8 Warm/Fast boot](#28-warmfast-boot) +- [3 Test plan](#3-test-plan) + - [3.1 Unit tests via VS](#31-unit-tests-via-vs) + - [3.2 Data plane tests via PTF](#32-data-plane-tests-via-ptf) + +## Revision + +| Rev | Date | Author | Description | +|:---:|:----------:|:--------------:|:------------------------------------------------| +| 0.1 | 12/09/2021 | Nazarii Hnydyn | Initial version | + +## About this manual + +This document provides general information about GH implementation in SONiC + +## Scope + +This document describes the high level design of GH feature in SONiC + +**In scope:** +1. ECMP/LAG switch hash configuration + +**Out of scope:** +1. ECMP/LAG switch hash seed configuration +2. ECMP/LAG switch hash algorithm configuration + +## Abbreviations + +| Term | Meaning | +|:-------|:------------------------------------------| +| SONiC | Software for Open Networking in the Cloud | +| GH | Generic Hash | +| ECMP | Equal-Cost Multi-Path | +| LAG | Link Aggregation Group | +| IP | Internet Protocol | +| SAI | Switch Abstraction Interface | +| API | Application Programming Interface | +| ASIC | Application-Specific Integrated Circuit | +| OA | Orchestration agent | +| DB | Database | +| CLI | Сommand-line Interface | +| YANG | Yet Another Next Generation | +| VS | Virtual Switch | +| PTF | Packet Test Framework | + +## List of figures + +[Figure 1: GH design](#figure-1-gh-design) +[Figure 2: GH OA design](#figure-2-gh-oa-design) +[Figure 3: GH update flow](#figure-3-gh-update-flow) + +## List of tables + +[Table 1: Event logging](#table-1-event-logging) + +# 1 Introduction + +## 1.1 Feature overview + +The hashing algorithm is used to make traffic-forwarding decisions for traffic exiting the switch. +It makes hashing decisions based on values in various packet fields, as well as on the hash seed value. +The packet fields used by the hashing algorithm varies by the configuration on the switch. + +For ECMP, the hashing algorithm determines how incoming traffic is forwarded to the next-hop device. +For LAG, the hashing algorithm determines how traffic is placed onto the LAG member links to manage +bandwidth by evenly load-balancing traffic across the the outgoing links. + +GH is a feature which allows user to configure which hash fields suppose to be used by hashing algorithm. +GH provides global switch hash configuration for ECMP and LAG. + +## 1.2 Requirements + +### 1.2.1 Functionality + +**This feature will support the following functionality:** +1. Ethernet packet hashing with inner/outer IP frames +2. Global switch hash configuration for ECMP and LAG +3. Warm/Fast reboot + +### 1.2.2 Command interface + +**This feature will support the following commands:** +1. config: set switch hash global configuration +2. show: display switch hash global configuration + +### 1.2.3 Error handling + +**This feature will provide error handling for the next situations:** +1. Incompatible options/parameters + +### 1.2.4 Event logging + +**This feature will provide event logging for the next situations:** +1. Switch hash update + +###### Table 1: Event logging + +| Event | Severity | +|:----------------------------|:---------| +| Switch hash update: success | NOTICE | +| Switch hash update: error | ERROR | + +# 2 Design + +## 2.1 Overview + +![GH design](images/gh_design.svg "Figure 1: GH design") + +###### Figure 1: GH design + +GH will use SAI Hash API to configure user-defined list of hash fields to ASIC. +Hashing policy can be set independently for ECMP and LAG. + +## 2.2 SAI API + +**SAI native hash fields which shall be used for GH:** + +| Field | Comment | +|:----------------------------------------|:----------------------------------------| +| SAI_NATIVE_HASH_FIELD_DST_MAC | SWITCH_HASH\|GLOBAL\|ecmp_hash/lag_hash | +| SAI_NATIVE_HASH_FIELD_SRC_MAC | | +| SAI_NATIVE_HASH_FIELD_ETHERTYPE | | +| SAI_NATIVE_HASH_FIELD_VLAN_ID | | +| SAI_NATIVE_HASH_FIELD_IP_PROTOCOL | | +| SAI_NATIVE_HASH_FIELD_DST_IP | | +| SAI_NATIVE_HASH_FIELD_SRC_IP | | +| SAI_NATIVE_HASH_FIELD_L4_DST_PORT | | +| SAI_NATIVE_HASH_FIELD_L4_SRC_PORT | | +| SAI_NATIVE_HASH_FIELD_INNER_DST_MAC | | +| SAI_NATIVE_HASH_FIELD_INNER_SRC_MAC | | +| SAI_NATIVE_HASH_FIELD_INNER_ETHERTYPE | | +| SAI_NATIVE_HASH_FIELD_INNER_IP_PROTOCOL | | +| SAI_NATIVE_HASH_FIELD_INNER_DST_IP | | +| SAI_NATIVE_HASH_FIELD_INNER_SRC_IP | | +| SAI_NATIVE_HASH_FIELD_INNER_L4_DST_PORT | | +| SAI_NATIVE_HASH_FIELD_INNER_L4_SRC_PORT | | + +**SAI attributes which shall be used for GH:** + +| API | Function | Attribute | +|:-------|:---------------------|:-------------------------------------| +| SWITCH | get_switch_attribute | SAI_SWITCH_ATTR_ECMP_HASH | +| | | SAI_SWITCH_ATTR_LAG_HASH | +| HASH | get_hash_attribute | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | +| | set_hash_attribute | | + +## 2.3 Orchestration agent + +### 2.3.1 Overview + +![GH OA design](images/gh_swss_design.svg "Figure 2: GH OA design") + +###### Figure 2: GH OA design + +The existing `SwitchOrch` class will be extended with a new APIs to implement GH feature. +OA will be extended with a new GH Config DB schema and SAI Hash API support. +Switch hash updates will be processed by OA based on Config DB changes. +Some updates will be handled and some will be considered as invalid. + +### 2.3.2 Switch orch + +Class `SwitchOrch` holds a set of methods matching generic `Orch` class pattern to handle Config DB updates. +For that purpose a producer-consumer mechanism (implemented in `sonic-swss-common`) is used. +Method `SwitchOrch::doTask()` will be called on switch hash update. It will distribute handling +of DB updates between other handlers based on the table key updated (Redis Keyspace Notifications). + +This class is responsible for: +1. Processing updates of switch hash +2. Partial input data validation +3. Replicating data from Config DB to SAI DB via SAI Redis +4. Caching objects in order to handle updates + +Switch hash object is stored under `SWITCH_HASH|GLOBAL` key in Config DB. On `SWITCH_HASH` update, +method `PbhOrch::doCfgSwitchHashTableTask()` will be called to process the change. +Regular switch hash update will refresh the internal class structures and appropriate SAI objects. +When global configuration is removed - fallback to vendor defaults is done. + +**Skeleton code:** +```cpp +class SwitchOrch : public Orch +{ + ... + +private: + void doCfgSwitchHashTableTask(Consumer &consumer); + void doTask(Consumer &consumer); + + // Switch hash + bool setSwitchHashFieldListSai(sai_object_id_t oid, std::vector &hfList) const; + bool setSwitchHashEcmpHashDefault() const; + bool setSwitchHashLagHashDefault() const; + bool setSwitchHashEcmpHash(const SwitchHash &hash) const; + bool setSwitchHashLagHash(const SwitchHash &hash) const; + bool setSwitchHash(const SwitchHash &hash); + + bool getSwitchHashOidSai(sai_object_id_t &oid, bool isEcmpHashOid) const; + bool getSwitchHashFieldListSai(std::set &hfSet, sai_object_id_t oid) const; + void getSwitchHashEcmpOid(); + void getSwitchHashLagOid(); + void getSwitchHashEcmpHashFields(); + void getSwitchHashLagHashFields(); + void querySwitchHashDefaults(); + + // Switch hash SAI defaults + struct { + struct { + sai_object_id_t oid = SAI_NULL_OBJECT_ID; + std::set hashFieldSet; + } ecmpHash; + struct { + sai_object_id_t oid = SAI_NULL_OBJECT_ID; + std::set hashFieldSet; + } lagHash; + } m_switchHashDefaults; + + // Switch OA helper + SwitchHelper swHlpr; + + ... +}; +``` + +## 2.4 DB schema + +### 2.4.1 Config DB + +#### 2.4.1.1 Switch hash +```abnf +; defines schema for switch hash configuration attributes +key = SWITCH_HASH|GLOBAL ; switch hash global. Must be unique + +; field = value +ecmp_hash = hash-field-list ; hash fields for hashing packets going through ECMP +lag_hash = hash-field-list ; hash fields for hashing packets going through LAG + +; value annotations +hash-field = "DST_MAC" + / "SRC_MAC" + / "ETHERTYPE" + / "IP_PROTOCOL" + / "DST_IP" + / "SRC_IP" + / "L4_DST_PORT" + / "L4_SRC_PORT" + / "INNER_DST_MAC" + / "INNER_SRC_MAC" + / "INNER_ETHERTYPE" + / "INNER_IP_PROTOCOL" + / "INNER_DST_IP" + / "INNER_SRC_IP" + / "INNER_L4_DST_PORT" + / "INNER_L4_SRC_PORT" +hash-field-list = hash-field [ 1*( "," hash-field ) ] +``` + +### 2.4.2 Data sample + +**Config DB:** +```bash +redis-cli -n 4 HGETALL 'SWITCH_HASH|GLOBAL' +1) "ecmp_hash@" +2) "DST_MAC,SRC_MAC,ETHERTYPE,IP_PROTOCOL,DST_IP,SRC_IP,L4_DST_PORT,L4_SRC_PORT, \ +INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER_SRC_IP,INNER_L4_DST_PORT,INNER_L4_SRC_PORT" +3) "lag_hash@" +4) "DST_MAC,SRC_MAC,ETHERTYPE,IP_PROTOCOL,DST_IP,SRC_IP,L4_DST_PORT,L4_SRC_PORT, \ +INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER_SRC_IP,INNER_L4_DST_PORT,INNER_L4_SRC_PORT" +``` + +### 2.4.3 Configuration sample + +**Outer/Inner frame hashing:** +```json +{ + "SWITCH_HASH": { + "GLOBAL": { + "ecmp_hash": [ + "DST_MAC", + "SRC_MAC", + "ETHERTYPE", + "IP_PROTOCOL", + "DST_IP", + "SRC_IP", + "L4_DST_PORT", + "L4_SRC_PORT", + "INNER_DST_MAC", + "INNER_SRC_MAC", + "INNER_ETHERTYPE", + "INNER_IP_PROTOCOL", + "INNER_DST_IP", + "INNER_SRC_IP", + "INNER_L4_DST_PORT", + "INNER_L4_SRC_PORT" + ], + "lag_hash": [ + "DST_MAC", + "SRC_MAC", + "ETHERTYPE", + "IP_PROTOCOL", + "DST_IP", + "SRC_IP", + "L4_DST_PORT", + "L4_SRC_PORT", + "INNER_DST_MAC", + "INNER_SRC_MAC", + "INNER_ETHERTYPE", + "INNER_IP_PROTOCOL", + "INNER_DST_IP", + "INNER_SRC_IP", + "INNER_L4_DST_PORT", + "INNER_L4_SRC_PORT" + ] + } + } +} +``` + +### 2.4.4 Initial configuration + +GH initial configuration will be updated at `sonic-buildimage/files/build_templates/init_cfg.json.j2` +in order to match vendor specific requirements. + +**Skeleton code:** +```jinja +{ + ... + +{%- if sonic_asic_platform == "mellanox" %} + "SWITCH_HASH": { + "GLOBAL": { + "ecmp_hash": [ + "DST_IP", + "SRC_IP", + "IP_PROTOCOL", + "L4_DST_PORT", + "L4_SRC_PORT", + "INNER_DST_IP", + "INNER_SRC_IP" + ], + "lag_hash": [ + "DST_IP", + "SRC_IP", + "IP_PROTOCOL", + "L4_DST_PORT", + "L4_SRC_PORT", + "INNER_DST_IP", + "INNER_SRC_IP" + ] + } + }, +{%- endif %} + + ... +} +``` + +## 2.5 Flows + +### 2.5.1 GH update + +![GH update flow](images/gh_update_flow.svg "Figure 3: GH update flow") + +###### Figure 3: GH update flow + +## 2.6 CLI + +### 2.6.1 Command structure + +**User interface**: +``` +config +|--- switch-hash + |--- global + |--- ecmp-hash ARGS + |--- lag-hash ARGS + +show +|--- switch-hash + |--- global +``` + +### 2.6.2 Usage examples + +#### 2.6.2.1 Config command group + +**The following command updates switch hash global:** +```bash +config switch-hash global ecmp-hash \ +'DST_MAC' \ +'SRC_MAC' \ +'ETHERTYPE' \ +'IP_PROTOCOL' \ +'DST_IP' \ +'SRC_IP' \ +'L4_DST_PORT' \ +'L4_SRC_PORT' \ +'INNER_DST_MAC' \ +'INNER_SRC_MAC' \ +'INNER_ETHERTYPE' \ +'INNER_IP_PROTOCOL' \ +'INNER_DST_IP' \ +'INNER_SRC_IP' \ +'INNER_L4_DST_PORT' \ +'INNER_L4_SRC_PORT' +config switch-hash global lag-hash \ +'DST_MAC' \ +'SRC_MAC' \ +'ETHERTYPE' \ +'IP_PROTOCOL' \ +'DST_IP' \ +'SRC_IP' \ +'L4_DST_PORT' \ +'L4_SRC_PORT' \ +'INNER_DST_MAC' \ +'INNER_SRC_MAC' \ +'INNER_ETHERTYPE' \ +'INNER_IP_PROTOCOL' \ +'INNER_DST_IP' \ +'INNER_SRC_IP' \ +'INNER_L4_DST_PORT' \ +'INNER_L4_SRC_PORT' +``` + +#### 2.6.2.2 Show command group + +**The following command shows switch hash global configuration:** +```bash +root@sonic:/home/admin# show switch-hash global +ECMP HASH LAG HASH +----------------- ----------------- +DST_MAC DST_MAC +SRC_MAC SRC_MAC +ETHERTYPE ETHERTYPE +IP_PROTOCOL IP_PROTOCOL +DST_IP DST_IP +SRC_IP SRC_IP +L4_DST_PORT L4_DST_PORT +L4_SRC_PORT L4_SRC_PORT +INNER_DST_MAC INNER_DST_MAC +INNER_SRC_MAC INNER_SRC_MAC +INNER_ETHERTYPE INNER_ETHERTYPE +INNER_IP_PROTOCOL INNER_IP_PROTOCOL +INNER_DST_IP INNER_DST_IP +INNER_SRC_IP INNER_SRC_IP +INNER_L4_DST_PORT INNER_L4_DST_PORT +INNER_L4_SRC_PORT INNER_L4_SRC_PORT +``` + +## 2.7 YANG model + +Existing YANG model template `sonic-types.yang.j2` at `sonic-buildimage/src/sonic-yang-models/yang-templates` +will be extended with a new common type. + +**Skeleton code:** +```yang + typedef hash-field { + description "Represents native hash field"; + type enumeration { + enum DST_MAC; + enum SRC_MAC; + enum ETHERTYPE; + enum VLAN_ID; + enum IP_PROTOCOL; + enum DST_IP; + enum SRC_IP; + enum L4_DST_PORT; + enum L4_SRC_PORT; + enum INNER_DST_MAC; + enum INNER_SRC_MAC; + enum INNER_ETHERTYPE; + enum INNER_IP_PROTOCOL; + enum INNER_DST_IP; + enum INNER_DST_IPV4; + enum INNER_DST_IPV6; + enum INNER_SRC_IP; + enum INNER_SRC_IPV4; + enum INNER_SRC_IPV6; + enum INNER_L4_DST_PORT; + enum INNER_L4_SRC_PORT; + } + } +``` + +New YANG model `sonic-hash.yang` will be added to `sonic-buildimage/src/sonic-yang-models/yang-models` +in order to extend existing data schema and provide support for GH. + +**Skeleton code:** +```yang +module sonic-hash { + + yang-version 1.1; + + namespace "http://github.com/Azure/sonic-hash"; + prefix hash; + + import sonic-types { + prefix stypes; + } + + description "HASH YANG Module for SONiC OS"; + + revision 2022-09-05 { + description "First Revision"; + } + + container sonic-hash { + + container SWITCH_HASH { + + description "SWITCH_HASH part of config_db.json"; + + container GLOBAL { + + leaf-list ecmp_hash { + description "Hash fields for hashing packets going through ECMP"; + type stypes:hash-field; + } + + leaf-list lag_hash { + description "Hash fields for hashing packets going through LAG"; + type stypes:hash-field; + } + + } + /* end of container GLOBAL */ + } + /* end of container SWITCH_HASH */ + } + /* end of container sonic-hash */ +} +/* end of module sonic-hash */ +``` + +## 2.8 Warm/Fast boot + +No special handling is required + +# 3 Test plan + +## 3.1 Unit tests via VS + +GH basic configuration test: +1. Verify ASIC DB object state after switch ECMP hash update +2. Verify ASIC DB object state after switch LAG hash update + +## 3.2 Data plane tests via PTF + +TBD diff --git a/doc/hash/images/gh_design.svg b/doc/hash/images/gh_design.svg new file mode 100644 index 0000000000..40f4ca4b56 --- /dev/null +++ b/doc/hash/images/gh_design.svg @@ -0,0 +1,262 @@ + + + + + gh design + + + + + Page-5 + + Right Brace.1112 + + + + Rounded Rectangle.1113 + SWITCH + + SWITCH + + Sheet.1002 + ECMP + + ECMP + + Right Brace.1115 + + + + Sheet.1004 + LAG + + LAG + + Rectangle.1117 + + + + 1-D single.1118 + + Sheet.1007 + + + + + Sheet.1008 + + + + Sheet.1009 + + + + + Rectangle.1122 + + + + Rectangle.1123 + + + + 1-D single.1129 + + Sheet.1013 + + + + + Sheet.1014 + + + + Sheet.1015 + + + + + 1-D single.1133 + + Sheet.1017 + + + + + Sheet.1018 + + + + Sheet.1019 + + + + + 1-D single.1137 + + Sheet.1021 + + + + + Sheet.1022 + + + + Sheet.1023 + + + + + 1-D single.1141 + + Sheet.1025 + + + + + Sheet.1026 + + + + Sheet.1027 + + + + + 1-D single.1145 + + Sheet.1029 + + + + + Sheet.1030 + + + + Sheet.1031 + + + + + 1-D single.1149 + + Sheet.1033 + + + + + Sheet.1034 + + + + Sheet.1035 + + + + + Sheet.1036 + Ethernet Packet + + Ethernet Packet + + Sheet.1037 + Outer Frame + + Outer Frame + + Rectangle.1158 + INNER_DST_IP + + INNER_DST_IP + + Rectangle.1159 + INNER_ETHERTYPE + + INNER_ETHERTYPE + + Rectangle.1162 + DST_MAC + + DST_MAC + + Rectangle.1163 + SRC_MAC + + SRC_MAC + + Rectangle.1164 + ETHERTYPE + + ETHERTYPE + + Rectangle.1165 + DST_IP + + DST_IP + + Sheet.1044 + Inner Frame + + Inner Frame + + Rectangle.1167 + INNER_DST_MAC + + INNER_DST_MAC + + Rectangle.1168 + INNER_SRC_MAC + + INNER_SRC_MAC + + Rectangle.1169 + SRC_IP + + SRC_IP + + Rectangle.1170 + INNER_SRC_IP + + INNER_SRC_IP + + diff --git a/doc/hash/images/gh_swss_design.svg b/doc/hash/images/gh_swss_design.svg new file mode 100644 index 0000000000..8611f96a32 --- /dev/null +++ b/doc/hash/images/gh_swss_design.svg @@ -0,0 +1,115 @@ + + + + + gh swss design + + + + + + + + + + + + + + + + + + + Page-6 + + Can.1070 + CONFIG_DB + + Sheet.1001 + + + + + CONFIG_DB + + + Rounded Rectangle.1072 + Orchagent + + ORCHAGENT + + Can.1079 + ASIC_DB + + Sheet.1004 + + + + + ASIC_DB + + + Rounded Rectangle.1081 + switchorch + + switchorch + + Dynamic connector.1083 + + + + Rounded Rectangle.1085 + SYNCD + + SYNCD + + Dynamic connector.1086 + + + + Rounded Rectangle.1087 + ASIC + + ASIC + + Dynamic connector.1088 + + + + Rounded Rectangle.1090 + SWITCH_HASH + + SWITCH_HASH + + Dynamic connector.1091 + + + + diff --git a/doc/hash/images/gh_update_flow.svg b/doc/hash/images/gh_update_flow.svg new file mode 100644 index 0000000000..b2a5e73cf8 --- /dev/null +++ b/doc/hash/images/gh_update_flow.svg @@ -0,0 +1,248 @@ + + + + + gh update flow + + + + + + + + + + + + + + + + + + + Page-8 + + Actor lifeline.1198 + ConfigDB + + Sheet.1001 + + + + Sheet.1002 + + + + Sheet.1003 + + + Sheet.1004 + + + + + ConfigDB + + + Object lifeline.1203 + SwitchOrch + + Sheet.1006 + + + + Sheet.1007 + + + + Sheet.1008 + + + Sheet.1009 + + + + + SwitchOrch + + + Object lifeline.1208 + SAI + + Sheet.1011 + + + + Sheet.1012 + + + + Sheet.1013 + + + Sheet.1014 + + + + + SAI + + + Actor lifeline.1213 + CLI + + Sheet.1016 + + + + Sheet.1017 + + + + Sheet.1018 + + + Sheet.1019 + + + + + CLI + + + Activation.1098 + + + + Self Message.1099 + process data + + + process data + + Activation.1102 + + + + Self Message.1103 + set switch hash + + + set switch hash + + Activation.1104 + + + + Message.1105 + HMSET SWITCH_HASH|GLOBAL + + + HMSET SWITCH_HASH|GLOBAL + + Activation.1106 + + + + Return Message.1107 + + + + Activation.1188 + + + + Self Message.1189 + set switch ecmp/lag hash field list + + + set switch ecmp/lag hash field list + + Activation.1190 + + + + Asynchronous Message.1240 + SWITCH_HASH|GLOBAL + + + SWITCH_HASH|GLOBAL + + Activation.1241 + + + + Self Message.1242 + process data + + + process data + + Message.1244 + set_hash_attribute + + + set_hash_attribute + + Activation.1245 + + + + Return Message.1246 + return <status> + + + return <status> + + Activation.1247 + + + + Self Message.1248 + get switch ecmp/lag hash oid + + + get switch ecmp/lag hash oid + + Activation.1249 + + + + Message.1250 + get_switch_attribute + + + get_switch_attribute + + Activation.1251 + + + + Return Message.1252 + return <status> + + + return <status> + + From cd680c6a024e5312844ef707998dbe81a22547e2 Mon Sep 17 00:00:00 2001 From: Nazarii Hnydyn Date: Tue, 11 Oct 2022 16:30:22 +0300 Subject: [PATCH 2/3] [hash]: GH HLD rev01. Signed-off-by: Nazarii Hnydyn --- doc/hash/hash-design.md | 66 ++++++++++++++++++++++++++++++----------- 1 file changed, 48 insertions(+), 18 deletions(-) diff --git a/doc/hash/hash-design.md b/doc/hash/hash-design.md index cd527f30b2..005184a0db 100644 --- a/doc/hash/hash-design.md +++ b/doc/hash/hash-design.md @@ -88,7 +88,8 @@ This document describes the high level design of GH feature in SONiC ## List of tables -[Table 1: Event logging](#table-1-event-logging) +[Table 1: Frontend event logging](#table-1-frontend-event-logging) +[Table 2: Backend event logging](#table-2-backend-event-logging) # 1 Introduction @@ -100,7 +101,7 @@ The packet fields used by the hashing algorithm varies by the configuration on t For ECMP, the hashing algorithm determines how incoming traffic is forwarded to the next-hop device. For LAG, the hashing algorithm determines how traffic is placed onto the LAG member links to manage -bandwidth by evenly load-balancing traffic across the the outgoing links. +bandwidth by evenly load-balancing traffic across the outgoing links. GH is a feature which allows user to configure which hash fields suppose to be used by hashing algorithm. GH provides global switch hash configuration for ECMP and LAG. @@ -110,7 +111,7 @@ GH provides global switch hash configuration for ECMP and LAG. ### 1.2.1 Functionality **This feature will support the following functionality:** -1. Ethernet packet hashing with inner/outer IP frames +1. Ethernet packet hashing configuration with inner/outer IP frames 2. Global switch hash configuration for ECMP and LAG 3. Warm/Fast reboot @@ -122,21 +123,55 @@ GH provides global switch hash configuration for ECMP and LAG. ### 1.2.3 Error handling +#### 1.2.3.1 Frontend + +**This feature will provide error handling for the next situations:** +1. Invalid parameter value + +#### 1.2.3.2 Backend + **This feature will provide error handling for the next situations:** -1. Incompatible options/parameters +1. Missing parameters +2. Invalid parameter value +3. Parameter removal +4. Configuration removal ### 1.2.4 Event logging +#### 1.2.4.1 Frontend + **This feature will provide event logging for the next situations:** 1. Switch hash update -###### Table 1: Event logging +###### Table 1: Frontend event logging | Event | Severity | |:----------------------------|:---------| | Switch hash update: success | NOTICE | | Switch hash update: error | ERROR | +#### 1.2.4.2 Backend + +**This feature will provide event logging for the next situations:** +1. Missing parameters +2. Invalid parameter value +3. Parameter value with duplicate items +4. Parameter removal +5. Configuration removal +6. Switch hash update + +###### Table 2: Backend event logging + +| Event | Severity | +|:-------------------------------------|:---------| +| Missing parameters | ERROR | +| Invalid parameter value | ERROR | +| Parameter value with duplicate items | WARNING | +| Parameter removal | ERROR | +| Configuration removal | ERROR | +| Switch hash update: success | NOTICE | +| Switch hash update: error | ERROR | + # 2 Design ## 2.1 Overview @@ -154,7 +189,8 @@ Hashing policy can be set independently for ECMP and LAG. | Field | Comment | |:----------------------------------------|:----------------------------------------| -| SAI_NATIVE_HASH_FIELD_DST_MAC | SWITCH_HASH\|GLOBAL\|ecmp_hash/lag_hash | +| SAI_NATIVE_HASH_FIELD_IN_PORT | SWITCH_HASH\|GLOBAL\|ecmp_hash/lag_hash | +| SAI_NATIVE_HASH_FIELD_DST_MAC | | | SAI_NATIVE_HASH_FIELD_SRC_MAC | | | SAI_NATIVE_HASH_FIELD_ETHERTYPE | | | SAI_NATIVE_HASH_FIELD_VLAN_ID | | @@ -178,8 +214,7 @@ Hashing policy can be set independently for ECMP and LAG. |:-------|:---------------------|:-------------------------------------| | SWITCH | get_switch_attribute | SAI_SWITCH_ATTR_ECMP_HASH | | | | SAI_SWITCH_ATTR_LAG_HASH | -| HASH | get_hash_attribute | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | -| | set_hash_attribute | | +| HASH | set_hash_attribute | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | ## 2.3 Orchestration agent @@ -209,8 +244,7 @@ This class is responsible for: Switch hash object is stored under `SWITCH_HASH|GLOBAL` key in Config DB. On `SWITCH_HASH` update, method `PbhOrch::doCfgSwitchHashTableTask()` will be called to process the change. -Regular switch hash update will refresh the internal class structures and appropriate SAI objects. -When global configuration is removed - fallback to vendor defaults is done. +Regular switch hash update will refresh the internal class structures and appropriate SAI objects. **Skeleton code:** ```cpp @@ -224,29 +258,22 @@ private: // Switch hash bool setSwitchHashFieldListSai(sai_object_id_t oid, std::vector &hfList) const; - bool setSwitchHashEcmpHashDefault() const; - bool setSwitchHashLagHashDefault() const; bool setSwitchHashEcmpHash(const SwitchHash &hash) const; bool setSwitchHashLagHash(const SwitchHash &hash) const; bool setSwitchHash(const SwitchHash &hash); bool getSwitchHashOidSai(sai_object_id_t &oid, bool isEcmpHashOid) const; - bool getSwitchHashFieldListSai(std::set &hfSet, sai_object_id_t oid) const; void getSwitchHashEcmpOid(); void getSwitchHashLagOid(); - void getSwitchHashEcmpHashFields(); - void getSwitchHashLagHashFields(); void querySwitchHashDefaults(); // Switch hash SAI defaults struct { struct { sai_object_id_t oid = SAI_NULL_OBJECT_ID; - std::set hashFieldSet; } ecmpHash; struct { sai_object_id_t oid = SAI_NULL_OBJECT_ID; - std::set hashFieldSet; } lagHash; } m_switchHashDefaults; @@ -271,9 +298,11 @@ ecmp_hash = hash-field-list ; hash fields for hashing packets going through ECMP lag_hash = hash-field-list ; hash fields for hashing packets going through LAG ; value annotations -hash-field = "DST_MAC" +hash-field = "IN_PORT" + / "DST_MAC" / "SRC_MAC" / "ETHERTYPE" + / "VLAN_ID" / "IP_PROTOCOL" / "DST_IP" / "SRC_IP" @@ -492,6 +521,7 @@ will be extended with a new common type. typedef hash-field { description "Represents native hash field"; type enumeration { + enum IN_PORT; enum DST_MAC; enum SRC_MAC; enum ETHERTYPE; From 1235e84d925308eaf80b926fc802c832e7fb688b Mon Sep 17 00:00:00 2001 From: Nazarii Hnydyn Date: Mon, 5 Dec 2022 16:09:17 +0200 Subject: [PATCH 3/3] [hash]: GH HLD rev02. Signed-off-by: Nazarii Hnydyn --- doc/hash/hash-design.md | 152 +++++++++-- doc/hash/images/gh_show_cap_flow.svg | 137 ++++++++++ doc/hash/images/gh_show_flow.svg | 136 ++++++++++ doc/hash/images/gh_update_flow.svg | 370 +++++++++++++++++++-------- 4 files changed, 669 insertions(+), 126 deletions(-) create mode 100644 doc/hash/images/gh_show_cap_flow.svg create mode 100644 doc/hash/images/gh_show_flow.svg diff --git a/doc/hash/hash-design.md b/doc/hash/hash-design.md index 005184a0db..0b3ac0803c 100644 --- a/doc/hash/hash-design.md +++ b/doc/hash/hash-design.md @@ -24,11 +24,17 @@ - [2.4 DB schema](#24-db-schema) - [2.4.1 Config DB](#241-config-db) - [2.4.1.1 Switch hash](#2411-switch-hash) - - [2.4.2 Data sample](#242-data-sample) - - [2.4.3 Configuration sample](#243-configuration-sample) - - [2.4.4 Initial configuration](#244-initial-configuration) + - [2.4.2 State DB](#242-state-db) + - [2.4.2.1 Switch hash capabilities](#2421-switch-hash-capabilities) + - [2.4.3 Data sample](#243-data-sample) + - [2.4.4 Configuration sample](#244-configuration-sample) + - [2.4.5 Initial configuration](#245-initial-configuration) - [2.5 Flows](#25-flows) - - [2.5.1 GH update](#251-gh-update) + - [2.5.1 Config section](#251-config-section) + - [2.5.1.1 GH update](#2511-gh-update) + - [2.5.2 Show section](#252-show-section) + - [2.5.2.1 GH show](#2521-gh-show) + - [2.5.2.2 GH show capabilities](#2522-gh-show-capabilities) - [2.6 CLI](#26-cli) - [2.6.1 Command structure](#261-command-structure) - [2.6.2 Usage examples](#262-usage-examples) @@ -44,7 +50,8 @@ | Rev | Date | Author | Description | |:---:|:----------:|:--------------:|:------------------------------------------------| -| 0.1 | 12/09/2021 | Nazarii Hnydyn | Initial version | +| 0.1 | 12/09/2022 | Nazarii Hnydyn | Initial version | +| 0.2 | 05/12/2022 | Nazarii Hnydyn | Capabilities validation | ## About this manual @@ -84,7 +91,9 @@ This document describes the high level design of GH feature in SONiC [Figure 1: GH design](#figure-1-gh-design) [Figure 2: GH OA design](#figure-2-gh-oa-design) -[Figure 3: GH update flow](#figure-3-gh-update-flow) +[Figure 3: GH update flow](#figure-3-gh-update-flow) +[Figure 4: GH show flow](#figure-4-gh-show-flow) +[Figure 5: GH show capabilities flow](#figure-5-gh-show-capabilities-flow) ## List of tables @@ -183,6 +192,12 @@ GH provides global switch hash configuration for ECMP and LAG. GH will use SAI Hash API to configure user-defined list of hash fields to ASIC. Hashing policy can be set independently for ECMP and LAG. +**GH important notes:** +1. According to the SAI Behavioral Model, the hash is calculated on ingress to pipeline +2. SAI configuration of hash fields is applicable to an original packet before any DECAP/ENCAP, +i.e. configuration is tunnel-agnostic +3. If some configured field is not present in an incoming packet, then zero is assumed for hash calculation + ## 2.2 SAI API **SAI native hash fields which shall be used for GH:** @@ -210,11 +225,15 @@ Hashing policy can be set independently for ECMP and LAG. **SAI attributes which shall be used for GH:** -| API | Function | Attribute | -|:-------|:---------------------|:-------------------------------------| -| SWITCH | get_switch_attribute | SAI_SWITCH_ATTR_ECMP_HASH | -| | | SAI_SWITCH_ATTR_LAG_HASH | -| HASH | set_hash_attribute | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | +| API | Function | Attribute | +|:-------|:-------------------------------------------|:-------------------------------------| +| OBJECT | sai_query_attribute_capability | SAI_SWITCH_ATTR_ECMP_HASH | +| | | SAI_SWITCH_ATTR_LAG_HASH | +| | | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | +| | sai_query_attribute_enum_values_capability | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | +| SWITCH | get_switch_attribute | SAI_SWITCH_ATTR_ECMP_HASH | +| | | SAI_SWITCH_ATTR_LAG_HASH | +| HASH | set_hash_attribute | SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST | ## 2.3 Orchestration agent @@ -243,9 +262,13 @@ This class is responsible for: 4. Caching objects in order to handle updates Switch hash object is stored under `SWITCH_HASH|GLOBAL` key in Config DB. On `SWITCH_HASH` update, -method `PbhOrch::doCfgSwitchHashTableTask()` will be called to process the change. +method `SwitchOrch::doCfgSwitchHashTableTask()` will be called to process the change. Regular switch hash update will refresh the internal class structures and appropriate SAI objects. +Switch hash capabilities are stored under `SWITCH_CAPABILITY|switch` key in State DB. +The vendor specific data is being queried by switch OA on init and pushed to both internal cache and DB. +Any further switch hash update is being validated using vendor specific hash capabilities. + **Skeleton code:** ```cpp class SwitchOrch : public Orch @@ -319,7 +342,43 @@ hash-field = "IN_PORT" hash-field-list = hash-field [ 1*( "," hash-field ) ] ``` -### 2.4.2 Data sample +### 2.4.2 State DB + +#### 2.4.2.1 Switch hash capabilities +```abnf +; defines schema for switch hash configuration capabilities +key = SWITCH_CAPABILITY|switch ; must be unique + +; field = value +ECMP_HASH_CAPABLE = capability-knob ; specifies whether switch is ECMP hash capable +LAG_HASH_CAPABLE = capability-knob ; specifies whether switch is LAG hash capable +HASH|NATIVE_HASH_FIELD_LIST = hash-field-list ; hash field capabilities for hashing packets going through switch + +; value annotations +capability-knob = "true" / "false" +hash-field = "" + / "IN_PORT" + / "DST_MAC" + / "SRC_MAC" + / "ETHERTYPE" + / "VLAN_ID" + / "IP_PROTOCOL" + / "DST_IP" + / "SRC_IP" + / "L4_DST_PORT" + / "L4_SRC_PORT" + / "INNER_DST_MAC" + / "INNER_SRC_MAC" + / "INNER_ETHERTYPE" + / "INNER_IP_PROTOCOL" + / "INNER_DST_IP" + / "INNER_SRC_IP" + / "INNER_L4_DST_PORT" + / "INNER_L4_SRC_PORT" +hash-field-list = hash-field [ 1*( "," hash-field ) ] +``` + +### 2.4.3 Data sample **Config DB:** ```bash @@ -332,7 +391,20 @@ INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER_SRC_IP,INNER_L4_DST_PORT,INNER_L4_SRC_PORT" ``` -### 2.4.3 Configuration sample +**State DB:** +```bash +redis-cli -n 6 HGETALL 'SWITCH_CAPABILITY|switch' + 1) "ECMP_HASH_CAPABLE" + 2) "true" + 3) "LAG_HASH_CAPABLE" + 4) "true" + 5) "HASH|NATIVE_HASH_FIELD_LIST" + 6) "IN_PORT,DST_MAC,SRC_MAC,ETHERTYPE,VLAN_ID,IP_PROTOCOL,DST_IP,SRC_IP,L4_DST_PORT,L4_SRC_PORT, \ +INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER_SRC_IP, \ +INNER_L4_DST_PORT,INNER_L4_SRC_PORT" +``` + +### 2.4.4 Configuration sample **Outer/Inner frame hashing:** ```json @@ -380,7 +452,7 @@ INNER_DST_MAC,INNER_SRC_MAC,INNER_ETHERTYPE,INNER_IP_PROTOCOL,INNER_DST_IP,INNER } ``` -### 2.4.4 Initial configuration +### 2.4.5 Initial configuration GH initial configuration will be updated at `sonic-buildimage/files/build_templates/init_cfg.json.j2` in order to match vendor specific requirements. @@ -421,12 +493,34 @@ in order to match vendor specific requirements. ## 2.5 Flows -### 2.5.1 GH update +### 2.5.1 Config section + +### 2.5.1.1 GH update ![GH update flow](images/gh_update_flow.svg "Figure 3: GH update flow") ###### Figure 3: GH update flow +**Note:** + +The list of available hash fields will be queried by `sai_query_attribute_enum_values_capability` in two steps. +The first attempt is used to accommodate the target list size after getting `SAI_STATUS_BUFFER_OVERFLOW` return code. +And the second one is for getting the actual data. + +### 2.5.2 Show section + +#### 2.5.2.1 GH show + +![GH show flow](images/gh_show_flow.svg "Figure 4: GH show flow") + +###### Figure 4: GH show flow + +#### 2.5.2.2 GH show capabilities + +![GH show capabilities flow](images/gh_show_cap_flow.svg "Figure 5: GH show capabilities flow") + +###### Figure 5: GH show capabilities flow + ## 2.6 CLI ### 2.6.1 Command structure @@ -442,6 +536,7 @@ config show |--- switch-hash |--- global + |--- capabilities ``` ### 2.6.2 Usage examples @@ -511,6 +606,31 @@ INNER_L4_DST_PORT INNER_L4_DST_PORT INNER_L4_SRC_PORT INNER_L4_SRC_PORT ``` +**The following command shows switch hash capabilities:** +```bash +root@sonic:/home/admin# show switch-hash capabilities +ECMP HASH LAG HASH +----------------- ----------------- +IN_PORT IN_PORT +DST_MAC DST_MAC +SRC_MAC SRC_MAC +ETHERTYPE ETHERTYPE +VLAN_ID VLAN_ID +IP_PROTOCOL IP_PROTOCOL +DST_IP DST_IP +SRC_IP SRC_IP +L4_DST_PORT L4_DST_PORT +L4_SRC_PORT L4_SRC_PORT +INNER_DST_MAC INNER_DST_MAC +INNER_SRC_MAC INNER_SRC_MAC +INNER_ETHERTYPE INNER_ETHERTYPE +INNER_IP_PROTOCOL INNER_IP_PROTOCOL +INNER_DST_IP INNER_DST_IP +INNER_SRC_IP INNER_SRC_IP +INNER_L4_DST_PORT INNER_L4_DST_PORT +INNER_L4_SRC_PORT INNER_L4_SRC_PORT +``` + ## 2.7 YANG model Existing YANG model template `sonic-types.yang.j2` at `sonic-buildimage/src/sonic-yang-models/yang-templates` diff --git a/doc/hash/images/gh_show_cap_flow.svg b/doc/hash/images/gh_show_cap_flow.svg new file mode 100644 index 0000000000..0a5fa549e8 --- /dev/null +++ b/doc/hash/images/gh_show_cap_flow.svg @@ -0,0 +1,137 @@ + + + + + gh show cap flow + + + + + + + + + + + + + + + + + + + Page-10 + + Actor lifeline.1198 + StateDB + + Sheet.1001 + + + + Sheet.1002 + + + + Sheet.1003 + + + Sheet.1004 + + + + + StateDB + + + Actor lifeline.1213 + CLI + + Sheet.1006 + + + + Sheet.1007 + + + + Sheet.1008 + + + Sheet.1009 + + + + + CLI + + + Activation.1098 + + + + Self Message.1099 + process data + + + process data + + Activation.1102 + + + + Self Message.1103 + get switch capabilities + + + get switch capabilities + + Activation.1104 + + + + Message.1105 + HGETALL SWITCH_CAPABILITY|switch + + + HGETALL SWITCH_CAPABILITY|switch + + Activation.1106 + + + + Return Message.1107 + + + + Activation.1044 + + + + Self Message.1045 + display data + + + display data + + diff --git a/doc/hash/images/gh_show_flow.svg b/doc/hash/images/gh_show_flow.svg new file mode 100644 index 0000000000..f31e79dc94 --- /dev/null +++ b/doc/hash/images/gh_show_flow.svg @@ -0,0 +1,136 @@ + + + + + gh show flow + + + + + + + + + + + + + + + + + + + Page-11 + + Actor lifeline.1198 + ConfigDB + + Sheet.1001 + + + + Sheet.1002 + + + + Sheet.1003 + + + Sheet.1004 + + + + + ConfigDB + + + Actor lifeline.1213 + CLI + + Sheet.1006 + + + + Sheet.1007 + + + + Sheet.1008 + + + Sheet.1009 + + + + + CLI + + + Activation.1098 + + + + Self Message.1099 + process data + + + process data + + Activation.1102 + + + + Self Message.1103 + get switch hash + + + get switch hash + + Activation.1104 + + + + Message.1105 + HGETALL SWITCH_HASH|GLOBAL + + + HGETALL SWITCH_HASH|GLOBAL + + Activation.1106 + + + + Return Message.1107 + + + + Activation.1044 + + + + Self Message.1045 + display data + + + display data + + diff --git a/doc/hash/images/gh_update_flow.svg b/doc/hash/images/gh_update_flow.svg index b2a5e73cf8..b5eae557e0 100644 --- a/doc/hash/images/gh_update_flow.svg +++ b/doc/hash/images/gh_update_flow.svg @@ -1,9 +1,9 @@ - + + width="19.1163in" height="19.5663in" viewBox="0 0 1376.37 1408.77" xml:space="preserve" color-interpolation-filters="sRGB" + class="st18"> gh update flow @@ -39,210 +44,355 @@ - Page-8 - + Page-12 + Actor lifeline.1198 ConfigDB Sheet.1001 - + Sheet.1002 - + - + Sheet.1003 - + Sheet.1004 - + - - ConfigDB + + ConfigDB - + Object lifeline.1203 SwitchOrch Sheet.1006 - + Sheet.1007 - + - + Sheet.1008 - + Sheet.1009 - + - - SwitchOrch + + SwitchOrch - + Object lifeline.1208 SAI Sheet.1011 - + Sheet.1012 - + - + Sheet.1013 - + Sheet.1014 - + - - SAI + + SAI - + Actor lifeline.1213 CLI Sheet.1016 - + Sheet.1017 - + - + Sheet.1018 - + Sheet.1019 - + - - CLI + + CLI - + Activation.1098 - + - + Self Message.1099 process data - - - process data - + + + process data + Activation.1102 - + - + Self Message.1103 set switch hash - - - set switch hash - + + + set switch hash + Activation.1104 - + - + Message.1105 HMSET SWITCH_HASH|GLOBAL - - - HMSET SWITCH_HASH|GLOBAL - + + + HMSET SWITCH_HASH|GLOBAL + Activation.1106 - + - + Return Message.1107 - + - + Activation.1188 - + - + Self Message.1189 set switch ecmp/lag hash field list - - - set switch ecmp/lag hash field list - + + + set switch ecmp/lag hash field list + Activation.1190 - + - + Asynchronous Message.1240 SWITCH_HASH|GLOBAL - - - SWITCH_HASH|GLOBAL - + + + SWITCH_HASH|GLOBAL + Activation.1241 - + - + Self Message.1242 process data - - - process data - + + + process data + Message.1244 set_hash_attribute - - - set_hash_attribute - + + + set_hash_attribute + Activation.1245 - + - + Return Message.1246 return <status> - - - return <status> - + + + return <status> + Activation.1247 - + - + Self Message.1248 get switch ecmp/lag hash oid - - - get switch ecmp/lag hash oid - + + + get switch ecmp/lag hash oid + Activation.1249 - + - + Message.1250 get_switch_attribute - - - get_switch_attribute - + + + get_switch_attribute + Activation.1251 - + - + Return Message.1252 return <status> - - - return <status> + + + return <status> + + Activation.1044 + + + + Self Message.1045 + get switch ecmp/lag hash capabilities + + + get switch ecmp/lag hash capabilities + + Activation.1046 + + + + Message.1047 + sai_query_attribute_capability + + + sai_query_attribute_capability + + Activation.1048 + + + + Return Message.1049 + return <status> + + + return <status> + + Message.1050 + sai_query_attribute_enum_values_capability + + + sai_query_attribute_enum_values_capability + + Activation.1051 + + + + Return Message.1052 + return <status> + + + return <status> + + Optional fragment.1059 + + + + + Sheet.1053 + opt + + opt + + Sheet.1054 + capability is validated + capability is validated + + + Loop fragment.1063 + + + + + Sheet.1056 + query + + query + + + Loop fragment.1066 + + + + + Sheet.1059 + query + + query + + + Actor lifeline.1069 + StateDB + + Sheet.1062 + + + + Sheet.1063 + + + + Sheet.1064 + + + Sheet.1065 + + + + + StateDB + + + Activation.1066 + + + + Self Message.1075 + push switch ecmp/lag hash capabilities to DB + + + push switch ecmp/lag hash capabilities to DB + + Activation.1076 + + + + Message.1077 + HMSET SWITCH_CAPABILITY|switch + + + HMSET SWITCH_CAPABILITY|switch + + Activation.1078 + + + + Return Message.1079 + + + + Sheet.1074 + SAI_SWITCH_ATTR_ECMP_HASH SAI_SWITCH_ATTR_LAG_HASH SAI_HASH_A... + + SAI_SWITCH_ATTR_ECMP_HASH SAI_SWITCH_ATTR_LAG_HASH SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST + + Sheet.1076 + SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST + + SAI_HASH_ATTR_NATIVE_HASH_FIELD_LIST