-
Notifications
You must be signed in to change notification settings - Fork 750
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added a test case for fast reboot from other vendor NOS to SONiC #6348
Added a test case for fast reboot from other vendor NOS to SONiC #6348
Conversation
Details: A new test case was added to fast reboot script. This test case mimics a fast reboot from other vendor nos to SONiC, by flushing all DBs that are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other nos.
742b546
to
ff5c90f
Compare
This pull request introduces 1 alert when merging 871a82c into bdd7663 - view on LGTM.com new alerts:
|
Details: Added the other_vendor_nos parameter as False by default to the AdvancedReboot class, and updated the reboot test cases correspondingly.
This pull request introduces 1 alert when merging 36e72ff into 9074e57 - view on LGTM.com new alerts:
|
This pull request introduces 1 alert when merging 6aba89e851e4b0934fe5a3f165c46ce8c174b74c into d6d8d9c - view on LGTM.com new alerts:
|
@@ -116,7 +117,9 @@ def __extractTestParam(self): | |||
if self.rebootLimit is None: | |||
if self.kvmTest: | |||
self.rebootLimit = 200 # Default reboot limit for kvm | |||
elif 'warm-reboot' in self.rebootType: | |||
if self.other_vendor_nos: | |||
self.rebootLimit = 50 # Threshold set for reboot from other vendor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this 50s expectation set? I don't think this is correct. Expectation for fastboot from vendor NOS is still <30s.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The flushing of the DBs is increasing the downtime in a few seconds, we would like to push it with the current downtime.
@@ -42,6 +43,21 @@ def test_fast_reboot(request, get_advanced_reboot, verify_dut_health, | |||
advancedReboot.runRebootTestcase() | |||
|
|||
|
|||
@pytest.mark.usefixtures('get_advanced_reboot') | |||
def test_fast_reboot_from_other_vendor(duthosts, rand_one_dut_hostname, request, get_advanced_reboot, verify_dut_health, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it will be better to add this test in a new script. test_fastboot_vendor_to_sonic.py
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The script test_advanced_reboot initiates various types of reboot, such as warm reboot, or fast reboot, with different parameters such as test_warm_reboot_mac_jump or test_cancelled_fast_reboot.
The new other vendor reboot test case is another parameter of advanced reboot and isn't different from other reboot types. In addition, all reboot types are using the infrastructure and fixtures of the advanced reboot script. Creating a new script will require duplication of fixtures and infrastructure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless I am missing something, creating a new test script will not require duplication of fixtures or infra. Any new test script will only have to import the fixtures, just how this script is doing.
None of the fixtures used by tests/platform_tests/test_advanced_reboot.py
are actually defined in this file. Same will be the case for test_fastboot_vendor_to_sonic.py
.
I am thinking that in future we will add more cases to non-SONiC to SONiC upgrade/reboot path. With tweaking parameters and also by doing a real (not just mock) upgrade. This deems a new category/script as it grows.
But I agree we should not have it as 50
We should have small diff between the 2 flows
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Liat Grozovik ***@***.***>
Sent: Monday, October 3, 2022 9:28:16 AM
To: sonic-net/sonic-mgmt ***@***.***>; sonic-net/sonic-mgmt ***@***.***>
Cc: Review requested ***@***.***>
Subject: Re: [sonic-net/sonic-mgmt] Added a test case for fast reboot from other vendor NOS to SONiC (PR #6348)
Imo It takes much more time when you delete the configuration
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Vaibhav Hemant Dixit ***@***.***>
Sent: Monday, October 3, 2022 9:21:06 AM
To: sonic-net/sonic-mgmt ***@***.***>
Cc: Liat Grozovik ***@***.***>; Review requested ***@***.***>
Subject: Re: [sonic-net/sonic-mgmt] Added a test case for fast reboot from other vendor NOS to SONiC (PR #6348)
@vaibhavhd commented on this pull request.
________________________________
In tests/common/fixtures/advanced_reboot.py<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsonic-net%2Fsonic-mgmt%2Fpull%2F6348%23discussion_r985407752&data=05%7C01%7Cliatg%40nvidia.com%7C0bf52de362fd4a60830008daa50774c9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638003748687786475%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3I6U8OksAV6mpyLx%2FIwrYPxkdeJP5pJZWgwH%2BOpPr%2FI%3D&reserved=0>:
@@ -116,7 +117,9 @@ def __extractTestParam(self):
if self.rebootLimit is None:
if self.kvmTest:
self.rebootLimit = 200 # Default reboot limit for kvm
- elif 'warm-reboot' in self.rebootType:
+ if self.other_vendor_nos:
+ self.rebootLimit = 50 # Threshold set for reboot from other vendor
Why is this 50s expectation set? I don't think this is correct. Expectation for fastboot from vendor NOS is still <30s.
________________________________
In tests/platform_tests/test_advanced_reboot.py<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsonic-net%2Fsonic-mgmt%2Fpull%2F6348%23discussion_r985408285&data=05%7C01%7Cliatg%40nvidia.com%7C0bf52de362fd4a60830008daa50774c9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638003748687786475%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ymdDvMXRFHiTq2B%2FwdZMSp6%2BKqxf9v5%2FDHf9JYs7Ee8%3D&reserved=0>:
@@ -129,3 +145,17 @@ def test_cancelled_warm_reboot(request, add_fail_step_to_reboot, verify_dut_heal
add_fail_step_to_reboot('warm-reboot')
advancedReboot = get_advanced_reboot(rebootType='warm-reboot', allow_fail=True)
advancedReboot.runRebootTestcase()
+
+def flush_dbs(duthost):
+ """
+ This Function will flush all unnecessary databases, to mimic reboot from other vendor
+ """
+ logger.info('Flushing databases from switch')
+ db_dic = { 0: 'Application DB',
+ 1: 'ASIC DB',
+ 2: 'Counters DB',
+ 5: 'Flex Counters DB',
+ 6: 'State DB'
+ }
+ for db in db_dic.keys():
+ duthost.shell('redis-cli -n {} flushdb'.format(db))
Please add a new line at the EOF
________________________________
In tests/platform_tests/test_advanced_reboot.py<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsonic-net%2Fsonic-mgmt%2Fpull%2F6348%23discussion_r985409412&data=05%7C01%7Cliatg%40nvidia.com%7C0bf52de362fd4a60830008daa50774c9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638003748687786475%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zRlPDoxu%2BF2aLIfS0YiZC%2F8V6hAlidgqDi9%2BMZTtt5Y%3D&reserved=0>:
@@ -42,6 +43,21 @@ def test_fast_reboot(request, get_advanced_reboot, verify_dut_health,
advancedReboot.runRebootTestcase()
***@***.***('get_advanced_reboot')
+def test_fast_reboot_from_other_vendor(duthosts, rand_one_dut_hostname, request, get_advanced_reboot, verify_dut_health,
I think it will be better to add this test in a new script. test_fastboot_vendor_to_sonic.py?
—
Reply to this email directly, view it on GitHub<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsonic-net%2Fsonic-mgmt%2Fpull%2F6348%23pullrequestreview-1127838073&data=05%7C01%7Cliatg%40nvidia.com%7C0bf52de362fd4a60830008daa50774c9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638003748687786475%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=adJ2uEHvR0PBeDOnKZ2iGRtSp7AwCJNUKlkSUdqJyVg%3D&reserved=0>, or unsubscribe<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKTABA32XMQALRJ6YLDGXH3WBJ3NFANCNFSM6AAAAAAQNEMAJY&data=05%7C01%7Cliatg%40nvidia.com%7C0bf52de362fd4a60830008daa50774c9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638003748687786475%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zZwjqUyuRQisBwCU3cLgdkoNXn7%2FmgZ7NLzUOAVUStw%3D&reserved=0>.
You are receiving this because your review was requested.Message ID: ***@***.***>
|
6aba89e
to
dac48be
Compare
The pre-commit check detected issues in the files touched by this pull request. For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame Detailed pre-commit check results: To run the pre-commit checks locally, you can follow below steps:
|
This pull request introduces 1 alert when merging dac48be96aa0db8a54c7dcc84172266c74b9739c into 61acf83 - view on LGTM.com new alerts:
|
dac48be
to
e19dbbf
Compare
The pre-commit check detected issues in the files touched by this pull request. For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame Detailed pre-commit check results: To run the pre-commit checks locally, you can follow below steps:
|
This pull request introduces 2 alerts when merging e19dbbff603d03471bc42d6a64e8d5e79dc57ca8 into da453f9 - view on LGTM.com new alerts:
|
e19dbbf
to
49ceee1
Compare
The pre-commit check detected issues in the files touched by this pull request. For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame Detailed pre-commit check results: To run the pre-commit checks locally, you can follow below steps:
|
Details: A new test case was added to fast reboot script. This test case mimics a fast reboot from other vendor nos to SONiC, by flushing all DBs that are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other nos.
49ceee1
to
78bac90
Compare
The pre-commit check detected issues in the files touched by this pull request. For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame Detailed pre-commit check results: To run the pre-commit checks locally, you can follow below steps:
|
/azpw run |
/AzurePipelines run |
Azure Pipelines successfully started running 1 pipeline(s). |
The pre-commit check detected issues in the files touched by this pull request. For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame Detailed pre-commit check results: To run the pre-commit checks locally, you can follow below steps:
|
@vaibhavhd - could you please merge this PR? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a non blocking comment.
Description of PR
Summary:
A new test case was added to fast reboot script.
This test case mimics a fast reboot from other vendor NOS to SONiC, by flushing all DBs that are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other NOSes.
Type of change
Back port request
Approach
What is the motivation for this PR?
The current flow doesn't support booting from other vendor nos, and by that, we cannot measure and determine the performance of the fast-reboot process in that specific sub-case.
The new test case enables us to mimic the flow of booting from other nos and by verifying the performance of this kind of reboot.
How did you do it?
I added a new flag of "other_vendor_nos" which was added to the original fast_reboot test case in False mode and in the new test case in True mode, and added this flag to the AdvancedReboot class.
This flag will initiate pre-boot function, that flushes all relevant DBs. By flushing them, the fast reboot flow is initiated with clean DBs, as if the system boots from other vendor nos.
How did you verify/test it?
I ran full regression on community testbeds,
ran the test on the regular fast boot mode and on the new mode and verified the reboot process succeed.
Any platform specific information?
No
Supported testbed topology if it's a new test case?
t0