Define and adapt the QA team repositories #5475
Comments
Generic SummaryMigration
Structure[wazuh-qa]
|─ dependencies
| ├ requirements
| ├ generic
| ├ constants
| ├ requests
| ├ file
| ├ exceptions
| ├ threading
| ├ time
| ├ utils
| └ commands
| ├ tools
| └ wazuh_api
| └ monitors
| └ database
| └ certificates
|─ tests
| ├ non_functional
| ├ footprint
| └── sca
| └ functional
| ├ utils
| ├ component
| ├ integration
| ├ systems
| └ e2e
|─ Jenkins
|─ Documentation with changelog, versioning and Readme. |
Update reportWorking with @pcapellan in parallel to define possible structures and technologies for the new QA repository that will store the QA framework, Jenkins, and test code. Two boilerplate versions were currently presented in a quick sync through the Slack chat, but they are still Work in Progress. QU3B1M proposal (WIP)Wazuh QA RepoThis repository will store the Wazuh QA framework, the different tests suites, and the QA Jenkins environment code. In the initial version only some basic tools of the framework will be developed/migrated, with the Deployability tests suite, and the new Jenkins environment with the approach of Everything-as-a-code. Wazuh QA FrameworkThe QA framework is a Python package with all the required tools and utilities to easily interact with the Wazuh components, the idea for it is to be a "cypress" for Wazuh, this framework must be easy to use, understand, and maintain.
Tests suitesWe must clearly define the different types of tests in a way that anyone can identify the kind of test to develop, which suite it belongs and where to store it. The idea is to have a clear structure for the tests, so it's easier to maintain and update them.
Basic structure proposal.
├── LICENSE
├── README.md
├── changelog
├── framework
│ ├── README.md
│ ├── src
│ │ ├── README.md
│ │ ├── tools
│ │ │ ├── README.md
│ │ │ ├── jobflow
│ │ │ │ └── README.md
│ │ │ ├── monitors
│ │ │ │ └── README.md
│ │ │ └── provisioner
│ │ │ └── README.md
│ │ └── utils
│ │ └── README.md
│ └── tests
│ └── README.md
├── jenkins
│ └── README.md
└── tests
├── README.md
├── functional
│ └── README.md
└── non_functional
└── README.md |
ReviewGreat job @wazuh/devel-qa-div3 ! some comments based on what we've spoken:
|
ReviewI think it's important to define if we are going to write the tests with the gherkin syntax and if we are going to use the page object model pattern for e2e test cases. If this were the case I would suggest, for the testing section, a structure like this: qa-wazuh/
│
├── features/
│ ├── feature_files/ # contains gherkin files and test steps
│ │ ├── logcollector.feature
│ │ └── syscheck.feature
│ └── steps/ # contains the python files that implement the steps defined in feature_files
│ ├── __init__.py
│ ├── logcollector_steps.py
│ └── syscheck_steps.py
│
├── pages/ # applies for e2e tests
│ ├── __init__.py
│ ├── events_page.py
│ └── dashboard_page.py
│ └── inventory_page.py
│
│
└── tests/
├── __init__.py
├── test_cases/ # contains the test files written with pytest
│ ├── __init__.py
│ ├── test_logcollector.py
│ └── test_syscheck.py
│
└── resources/ # contains resources for testing: data generation, configurations, etc.
|
Structure UpdateConsidering the previous recommendations, a possible structure could look something like this .
├── .github
│ └── workflows
│ ├── run_unitests.yaml
│ └── tools_coverage.yaml
├── README.md
├── tests
│ ├── test_functional
│ │ ├── test_e2e
│ │ │ ├── features
│ │ │ ├── page_objects
│ │ │ └── tests # Or steps_definitions/
│ │ └── test_system
│ │ └── test_deployability
│ │ ├── features
│ │ └── tests # Or steps_definitions/
│ └── test_non_functional
│ └── test_stress
│ ├── features
│ └── tests # Or steps_definitions/
└── tools
├── generic
│ ├── src
│ │ ├── logger
│ │ └── monitors
│ └── tests
│ ├── logger
│ └── monitors
├── jenkins
│ ├── src
│ └── tests
├── jobflow
│ ├── src
│ └── test
├── provision
│ ├── src
│ └── test
└── wazuh_qa_framework
├── src
│ ├── api_client
│ ├── constants
│ └── utils
└── tests
├── api_client
├── constants
└── utils
I would prefer to keep the "test"_ prefix, as it helps pytest to identify these directories. pytest -m <tag> |
|
GJ! |
Description
To improve the structure and maintainability of the
wazuh-qaandwazuh-jenkinsrepositories, we need to define and adapt the current design. This will help organize tests and improve practices.Functional requirements
Tasks
Wazuh QA automation repository
This repository will store the Wazuh QA framework, the different test suites, and the QA Jenkins environment code. In the initial version, only some basic tools of the different repositories will be developed/migrated, with the Deployability tests suite, and the new Jenkins environment with the approach of Everything-as-a-code.
Repository structure
After analyzing the current repository's structure, and the different proposals, we can conclude that our new structure must follow the next key points:
Taking into account the previous points, we propose the following structure:
Other option
Proposed structure documentation and justification
Currently, we are facing two issues. First, our main
wazuh-qarepository has a lot of useful code, but as its structure is not well defined, it is difficult to maintain and scale, the test types are not well defined, and the tools and utils are all part of a big framework that does everything (its scope is not well defined). Second, we have other repositories with useful code, but they are not part of the main repository, so we have content spread across different repositories making it hard to maintain.So, the idea is to avoid this situation by creating a new structure that allows us to scale and maintain the code easily, having a well-defined scope for each tool and test type. It also includes code from the other repositories owned by QA
To achieve this goal, the proposed structure is divided into two main directories:
tests/andtools/which at the same time are divided into subdirectories with more subdirectories, going from a generic scope to a more specific one.The
tests/directory is divided into the main tests categories functional and non-functional, from that point the categories can be divided into more specific test types, such as end-to-end, and system for the functional tests, and performance and stress for the non-functional tests. Each of these directories could contain helpers or fixtures used only by the tests in that scope and its subdirectories.For the
tools/directory, the structure is different, we don't have 'categories', in this directory resides all the tools that we use in our QA process, such as Jenkins, JobFlow, Provision, and wazuh_qa_framework, alongside all the unit tests for each tool. Its basic structure is divided intosrc/andtests/, where thesrc/directory contains the source code of the tools, and thetests/directory contains the unit tests for the tools.Another thing to consider is that the
wazuh_qa_frameworkis defined only to be a 'wazuh components handler' tool, this way we can have a well-defined scope for this tool, and avoid making it 'infinite' as it is now.Proposal, process, and good practices documentation
Initial migration process
In this first approach, we will only migrate the newly developed tools and tests by the QA team, which are DTT1 and the new Jenkins environment. This migration flow consists of the following simple steps:
1. Review the current code
2. Analyze and propose an adaptation for the new structure
3. Migrate the code with the pertinent modifications
As the code is recently developed, we don't expect to have many issues during the migration process, but if we find any, we must document them and propose a solution.
Tests development process
The development of new tests must follow a well-defined process to ensure that the tests are well-developed and maintained. A proposal for this process is defined as follows:
1. Analyze and define the scope and type of the test scenario. (could use a guide like this)
2. Define a proposal set of Test Cases
3. Develop the approved test cases with the required helpers or fixtures
Other Recommendations
Tools development process
The tools development process must also follow a well-defined process to ensure its correct development. A proposal for this process is defined as follows:
1. Analyze the requested feature or tool
2. Define the structure of the tool and technologies to use
3. Develop the tool with the required unit tests
Good practices
Useful sources
Development
QA Automation
The text was updated successfully, but these errors were encountered: