Introduce a SONiC-specific communication channel between FRR and SONiC

[cscarpitta]

Why I did it

Introduce a SONiC-specific communication channel between FRR and SONiC.
HLD: sonic-net/SONiC#1620

How I did it

Introduce a SONiC-specific communication channel between FRR and SONiC (FPM SONiC module).

How to verify it

By switching to the new FPM SONiC module, all existing test cases for all features are successfully passed.

Description for the changelog

• Add new FPM SONiC module
• Add a patch file "build-dplane-fpm-sonic-module.patch" to compile FRR with the new FPM SONiC module
• Change supervisord.conf.j2 of the docker-fpm-frr container to load the new FPM SONiC module on FRR startup

[mssonicbld]

@StormLiangMS,@liushilongbuaa PR: #18715 is conflict with MS internal repo
Please push fix commit to sonicbld/precheck/head/18715 and approve
https://msazure.visualstudio.com/One/_git/Networking-acs-buildimage/pullrequest/9922845
After ms PR is merged, comment "/azpw ms_conflict" to rerun PR checker.

[liushilongbuaa]

/azpw ms_conflict

[liushilongbuaa]

/azpw ms_conflict

[lguohan]

it is better if we do not change it by default, it is better to first introduce this as option and then wait till it becomes stable.

[cscarpitta]

Hi @lguohan

Many thanks for the review.

I added a compile-time option ENABLE_DPLANE_FPM_SONIC.

When this option is set, SONiC uses the new FPM module.
When this option is not set, SONiC uses the old FPM module and behaves exactly as it does today.

By default, this option is not set.

[lguohan]

i am fine that we build them into image by default, but do not enable it by default. if you can do it at runtime, it will be easier for you, and i am fine with that.

does this dplane_fpm_sonic support all features as dplane_fpm_nl?

[cscarpitta]

@lguohan

Yes, the dplane_fpm_sonic supports all features as dplane_fpm_nl. It is a copy of the old dplane_fpm_nl.

Indeed, it is easier to compile this by default and keep the old dplane_fpm_nl running by default.
I reverted the compile option and changed the supervisord config file to keep dplane_fpm_nl as default.

[lguohan]

@cscarpitta , have we sync with frr community to see if we can add this plugin into frr repo and get maintainer there? I think this can be facilitated by the routing group? @kperumalbfn , and @eddieruan-alibaba ?

if we cannot get into frr repo, for this plugin, can we maintain as a patch format to fpm_nl? so that we can still keep track of upstream changes. What if there is some new changes to dplane_fpm_nl and we need those changes into dplane_fpm_sonic?

[eddieruan-alibaba]

@cscarpitta , have we sync with frr community to see if we can add this plugin into frr repo and get maintainer there? I think this can be facilitated by the routing group? @kperumalbfn , and @eddieruan-alibaba ?

if we cannot get into frr repo, for this plugin, can we maintain as a patch format to fpm_nl? so that we can still keep track of upstream changes. What if there is some new changes to dplane_fpm_nl and we need those changes into dplane_fpm_sonic?

Discussed this with Donald Sharp in Routing WG. The conclusion was that Donald didn't want to take it into FRR community since it is SONiC specific. This proposal was proposed via Mark Stepp from Cisco who worked in FRR fpm before and supported by Donald who owns FRR. The discussion meeting minutes could be found in https://lists.sonicfoundation.dev/g/sonic-wg-routing/wiki/36598.

The basic idea is to let SONiC team own sonic fpm. The zebra/fpm apis have already been changed with versioning by Mark S. Since we own sonic fpm, I feel it is simple to keep dplane_fpm_sonic.c as a file instead of as a diff.

[cscarpitta]

@cscarpitta , have we sync with frr community to see if we can add this plugin into frr repo and get maintainer there? I think this can be facilitated by the routing group? @kperumalbfn , and @eddieruan-alibaba ?

if we cannot get into frr repo, for this plugin, can we maintain as a patch format to fpm_nl? so that we can still keep track of upstream changes. What if there is some new changes to dplane_fpm_nl and we need those changes into dplane_fpm_sonic?

@lguohan As Eddie mentioned, the option of adding the new changes as a patch was discussed in the Routing WG. However there were some concerns on this approach and the consensus was that this will be maintained as a separate module under the sonic-buildimage repository

Regarding the second point, in case of new features or bug fixes or improvements of dplane_fpm_nl, as part of the FRR upgrade process we will make sure that all the changes will be reflected in the dplane_fpm_sonic.

[eddieruan-alibaba]

For the platform which wants to use dplane_fpm_sonic.so, how does this .so get picked up? Do you want to add a README file for that?

[cscarpitta]

For the platform which wants to use dplane_fpm_sonic.so, how does this .so get picked up? Do you want to add a README file for that?

Hi @eddieruan-alibaba Many thanks for the review.

I added a README file to explain how to enable the dplane_fpm_sonic module.

[cscarpitta]

/azpw ms_conflict

[cscarpitta]

@liushilongbuaa @StormLiangMS Can you please help fix the ms_conflict?

[cscarpitta]

@lguohan

The HLD for this feature has been merged: sonic-net/SONiC#1620

We addressed all comments on this PR. Can you please take a look again and help merge?

[liushilongbuaa]

/azpw ms_conflict -f

[eddieruan-alibaba]

For the platform which wants to use dplane_fpm_sonic.so, how does this .so get picked up? Do you want to add a README file for that?

Hi @eddieruan-alibaba Many thanks for the review.

I added a README file to explain how to enable the dplane_fpm_sonic module.

Thanks Carmine. The readme explains the changes needed. As part of Phoenix wing project, we could change the .j2 based on the supported platform type.

[eddieruan-alibaba]

@lguohan currently, he just commit the changes needed as infra. It would not be enabled on the current platforms. As part of phoenix wing project, we could enable them on the platforms verified in that project.

[ysj-alibaba]

@cscarpitta I don't understand what does DPLAINE_OP_STARTUP_STAGE mean and it seems that there is no definition about this flag. Thanks.

[eddieruan-alibaba]

This PR's changes are cherry picked to Phoenixwing Repro and get validated.

[kperumalbfn]

@cscarpitta could you check the buildimage failures and VS test failurs?

[kperumalbfn]

/azpw ms_conflict -f

[kperumalbfn]

LGTM. Will merge after resolving the conflict.

[kperumalbfn]

@liushilongbuaa Could you check the conflict and trigger ms_conflict check

[liushilongbuaa]

/azpw ms_checker

[liushilongbuaa]

/azpw ms_conflict

[ahsalam]

@kperumalbfn could you please check again? the PR passed all tests. Thanks!

[sudhanshukumar22]

Hi, is it possible to know that which of dplane_fpm_sonic or dplane_fpm_nl is running in the system loaded with sonic image ?

[sudhanshukumar22]

root@sonic:/# ps -aux | grep zebra
frr 39 0.0 0.3 647216 24488 pts/0 Sl 08:42 0:00 /usr/lib/frr/zebra -A 127.0.0.1 -s 90000000 -M dplane_fpm_sonic -M snmp --asic-offload=notify_on_offload
root 299 0.0 0.0 3324 1388 pts/1 S+ 09:05 0:00 grep zebra
Does this mean that dplane_fpm_sonic is running in the system ?

[ahsalam]

@sudhanshukumar22 dplane_fpm_sonic is running in the system.