Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a test case for fast reboot from other vendor NOS to SONiC #6348

Merged

Conversation

AharonMalkin
Copy link
Contributor

@AharonMalkin AharonMalkin commented Sep 15, 2022

Description of PR

Summary:
A new test case was added to fast reboot script.
This test case mimics a fast reboot from other vendor NOS to SONiC, by flushing all DBs that are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other NOSes.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012

Approach

What is the motivation for this PR?

The current flow doesn't support booting from other vendor nos, and by that, we cannot measure and determine the performance of the fast-reboot process in that specific sub-case.
The new test case enables us to mimic the flow of booting from other nos and by verifying the performance of this kind of reboot.

How did you do it?

I added a new flag of "other_vendor_nos" which was added to the original fast_reboot test case in False mode and in the new test case in True mode, and added this flag to the AdvancedReboot class.
This flag will initiate pre-boot function, that flushes all relevant DBs. By flushing them, the fast reboot flow is initiated with clean DBs, as if the system boots from other vendor nos.

How did you verify/test it?

I ran full regression on community testbeds,
ran the test on the regular fast boot mode and on the new mode and verified the reboot process succeed.

Any platform specific information?

No

Supported testbed topology if it's a new test case?

t0

@AharonMalkin AharonMalkin changed the title Aharon other vendor reboot Added a test case for fast reboot from other vendor NOS to SONiC Sep 15, 2022
Details:
A new test case was added to fast reboot script.
This test case mimics a fast reboot from other vendor nos to SONiC, by flushing all DBs that
are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other nos.
@AharonMalkin AharonMalkin force-pushed the aharon_other_vendor_reboot branch from 742b546 to ff5c90f Compare September 15, 2022 08:23
@lgtm-com
Copy link

lgtm-com bot commented Sep 15, 2022

This pull request introduces 1 alert when merging 871a82c into bdd7663 - view on LGTM.com

new alerts:

  • 1 for Use of the return value of a procedure

roy-sror
roy-sror previously approved these changes Sep 19, 2022
Details:
Added the other_vendor_nos parameter as False by default to the AdvancedReboot class,
and updated the reboot test cases correspondingly.
@lgtm-com
Copy link

lgtm-com bot commented Sep 20, 2022

This pull request introduces 1 alert when merging 36e72ff into 9074e57 - view on LGTM.com

new alerts:

  • 1 for Use of the return value of a procedure

roy-sror
roy-sror previously approved these changes Sep 21, 2022
@lgtm-com
Copy link

lgtm-com bot commented Sep 28, 2022

This pull request introduces 1 alert when merging 6aba89e851e4b0934fe5a3f165c46ce8c174b74c into d6d8d9c - view on LGTM.com

new alerts:

  • 1 for Use of the return value of a procedure

@@ -116,7 +117,9 @@ def __extractTestParam(self):
if self.rebootLimit is None:
if self.kvmTest:
self.rebootLimit = 200 # Default reboot limit for kvm
elif 'warm-reboot' in self.rebootType:
if self.other_vendor_nos:
self.rebootLimit = 50 # Threshold set for reboot from other vendor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this 50s expectation set? I don't think this is correct. Expectation for fastboot from vendor NOS is still <30s.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flushing of the DBs is increasing the downtime in a few seconds, we would like to push it with the current downtime.

tests/platform_tests/test_advanced_reboot.py Outdated Show resolved Hide resolved
@@ -42,6 +43,21 @@ def test_fast_reboot(request, get_advanced_reboot, verify_dut_health,
advancedReboot.runRebootTestcase()


@pytest.mark.usefixtures('get_advanced_reboot')
def test_fast_reboot_from_other_vendor(duthosts, rand_one_dut_hostname, request, get_advanced_reboot, verify_dut_health,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be better to add this test in a new script. test_fastboot_vendor_to_sonic.py?

Copy link
Contributor Author

@AharonMalkin AharonMalkin Oct 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script test_advanced_reboot initiates various types of reboot, such as warm reboot, or fast reboot, with different parameters such as test_warm_reboot_mac_jump or test_cancelled_fast_reboot.
The new other vendor reboot test case is another parameter of advanced reboot and isn't different from other reboot types. In addition, all reboot types are using the infrastructure and fixtures of the advanced reboot script. Creating a new script will require duplication of fixtures and infrastructure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I am missing something, creating a new test script will not require duplication of fixtures or infra. Any new test script will only have to import the fixtures, just how this script is doing.
None of the fixtures used by tests/platform_tests/test_advanced_reboot.py are actually defined in this file. Same will be the case for test_fastboot_vendor_to_sonic.py.

I am thinking that in future we will add more cases to non-SONiC to SONiC upgrade/reboot path. With tweaking parameters and also by doing a real (not just mock) upgrade. This deems a new category/script as it grows.

@liat-grozovik
Copy link
Collaborator

liat-grozovik commented Oct 3, 2022 via email

@liat-grozovik
Copy link
Collaborator

liat-grozovik commented Oct 11, 2022 via email

@AharonMalkin AharonMalkin force-pushed the aharon_other_vendor_reboot branch from 6aba89e to dac48be Compare October 20, 2022 16:30
@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/test_advanced_reboot.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

ansible/roles/test/files/ptftests/advanced-reboot.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/vsts/.cache/pre-commit/repot_wd1lly/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@lgtm-com
Copy link

lgtm-com bot commented Oct 20, 2022

This pull request introduces 1 alert when merging dac48be96aa0db8a54c7dcc84172266c74b9739c into 61acf83 - view on LGTM.com

new alerts:

  • 1 for Use of the return value of a procedure

@AharonMalkin AharonMalkin force-pushed the aharon_other_vendor_reboot branch from dac48be to e19dbbf Compare October 31, 2022 12:08
@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/test_advanced_reboot.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

ansible/roles/test/files/ptftests/advanced-reboot.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/vsts/.cache/pre-commit/repo3sbowq1j/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@lgtm-com
Copy link

lgtm-com bot commented Oct 31, 2022

This pull request introduces 2 alerts when merging e19dbbff603d03471bc42d6a64e8d5e79dc57ca8 into da453f9 - view on LGTM.com

new alerts:

  • 1 for Use of the return value of a procedure
  • 1 for Unused local variable

@AharonMalkin AharonMalkin force-pushed the aharon_other_vendor_reboot branch from e19dbbf to 49ceee1 Compare November 3, 2022 12:33
@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/test_advanced_reboot.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

ansible/roles/test/files/ptftests/advanced-reboot.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/vsts/.cache/pre-commit/repobsbfc1ap/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Details:
A new test case was added to fast reboot script.
This test case mimics a fast reboot from other vendor nos to SONiC, by flushing all DBs that
are part of the boot process, so the boot will be initiated with clean DBs as if it was booting from other nos.
@AharonMalkin AharonMalkin force-pushed the aharon_other_vendor_reboot branch from 49ceee1 to 78bac90 Compare November 3, 2022 12:40
@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/test_advanced_reboot.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

ansible/roles/test/files/ptftests/advanced-reboot.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/vsts/.cache/pre-commit/repo01riejjf/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@AharonMalkin
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing tests/platform_tests/test_advanced_reboot.py

check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

ansible/roles/test/files/ptftests/advanced-reboot.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/vsts/.cache/pre-commit/repom0msdyqq/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@roy-sror
Copy link
Contributor

roy-sror commented Nov 8, 2022

@vaibhavhd - could you please merge this PR?

Copy link
Contributor

@vaibhavhd vaibhavhd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a non blocking comment.

@liat-grozovik liat-grozovik merged commit a18318e into sonic-net:master Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants