diff --git a/deployability/modules/provision/README.MD b/deployability/modules/provision/README.MD index 2047b51235..39777635e2 100644 --- a/deployability/modules/provision/README.MD +++ b/deployability/modules/provision/README.MD @@ -1,130 +1,78 @@ ## Provision module ### User documentation - -The execution of the Provision is done through the installation of the library Worfklow library or launching the module using python command. The execution can be done from any operating system. -Initially, Python libraries must be installed. It is recommended to use virtual environments. Follow the technical documentation at https://docs.python.org/3/library/venv.html. +The Provision module installs libraries, dependencies and applications on the allocated infrastructure (From Allocation module) +This module can be executed as follows: + A. Installing and using Workflow engine + B. Direct execution -1. Activate the environment: +#### A. Installing and using Workflow engine - ```bash - source {venv directory}/bin/activate - ``` +The execution of the workflow is done through the installation of its library. -2. Clone the `wazuh-qa` repository: +Initially, Python libraries must be installed. we recommended the use of virtual environments. Follow the technical documentation at https://docs.python.org/3/library/venv.html. - Navigate to the project directory and switch to the project branch: +1. Create the python environment: - ```bash - cd wazuh-qa - git checkout {project-branch} - ``` -> Note: temporary dev project-branch is `enhancement/4495-DTT1` + ```bash + python3 -m venv {environment_name} + ``` -3. Install requirements: +2. Activate the environment: - ```bash - pip3 install -r deployability/deps/requirements.txt - ``` + ```bash + source {venv directory}/bin/activate + ``` -Now, it is possible to use worklow engine library to launch provision module doing the following steps: +3. Clone the `wazuh-qa` repository: -1. Install the Workflow engine library and its launcher: + Navigate to the project directory and switch to the project branch: - While in wazuh-qa: + ```bash + https://github.com/wazuh/wazuh-qa.git + cd wazuh-qa + git checkout {project-branch} + ``` - ```bash - cd modules - pip3 uninstall -y workflow_engine && pip3 install . - ``` +4. Install requirements: + + ```bash + pip3 install -r deployability/deps/requirements.txt + ``` + +5. Install the Workflow engine library and its launcher: + + While in wazuh-qa: -2. Test Fixture to Execute: + ```bash + cd modules + pip3 uninstall -y workflow_engine && pip3 install . + ``` - It will be necessary to create a fixture (yaml file) where the infrastructure, provisioning, and tests to be executed will be declared. + Run the module by doing the following steps: - >Note: It is possible to find some fixture examples in deployability/modules/workflow_engine/examples/ +6. Test fixture to execute: + + It is required to create a fixture (YAML file) where the infrastructure, provisioning, and tests to be executed is declared. + + >Note: You can find some fixture examples in '[deployability/modules/workflow_engine/examples/](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/workflow_engine/examples)' Example: ```bash version: 0.1 - description: This workflow is used to test agents deployment por DDT1 PoC + description: This workflow is used to provision agents hosts por DDT1 PoC variables: agents-os: - linux-ubuntu-22.04-amd64 - manager-os: linux-ubuntu-22.04-amd64 infra-provider: vagrant working-dir: /tmp/dtt1-poc - + tasks: - # Generic agent test task - - task: "run-agent-tests-{agent}" - description: "Run tests uninstall for the {agent} agent." - do: - this: process - with: - path: python3 - args: - - modules/testing/main.py - - inventory: "{working-dir}/agent-{agent}/inventory.yaml" - - dependencies: - - manager: "{working-dir}/manager-{manager-os}/inventory.yaml" - - agent: "{working-dir}/agent-{agent}/inventory.yaml" - - tests: "install,register,stop" - - component: "agent" - - wazuh-version: "4.7.1" - - wazuh-revision: "40709" - depends-on: - - "provision-install-{agent}" - - "provision-manager" - foreach: - - variable: agents-os - as: agent - - # Generic agent test task - - task: "run-agent-tests-uninstall-{agent}" - description: "Run tests uninstall for the {agent} agent." - do: - this: process - with: - path: python3 - args: - - modules/testing/main.py - - inventory: "{working-dir}/agent-{agent}/inventory.yaml" - - dependencies: - - manager: "{working-dir}/manager-{manager-os}/inventory.yaml" - - tests: "uninstall" - - component: "agent" - - wazuh-version: "4.7.1" - - wazuh-revision: "40709" - depends-on: - - "run-agent-tests-{agent}" - - "provision-uninstall-{agent}" - foreach: - - variable: agents-os - as: agent - - # Unique manager provision task - - task: "provision-manager" - description: "Provision the manager." - do: - this: process - with: - path: python3 - args: - - modules/provision/main.py - - inventory-manager: "{working-dir}/manager-{manager-os}/inventory.yaml" - - install: - - component: wazuh-manager - type: package - depends-on: - - "allocate-manager" - - # Unique manager allocate task - - task: "allocate-manager" - description: "Allocate resources for the manager." + - task: "allocate-{agent}" + description: "Allocate resources for the {agent} agent." do: this: process with: @@ -133,10 +81,10 @@ Now, it is possible to use worklow engine library to launch provision module doi - modules/allocation/main.py - action: create - provider: "{infra-provider}" - - size: large - - composite-name: "{manager-os}" - - inventory-output: "{working-dir}/manager-{manager-os}/inventory.yaml" - - track-output: "{working-dir}/manager-{manager-os}/track.yaml" + - size: small + - composite-name: "{agent}" + - inventory-output: "{working-dir}/agent-{agent}/inventory.yaml" + - track-output: "{working-dir}/agent-{agent}/track.yaml" cleanup: this: process with: @@ -144,9 +92,11 @@ Now, it is possible to use worklow engine library to launch provision module doi args: - modules/allocation/main.py - action: delete - - track-output: "{working-dir}/manager-{manager-os}/track.yaml" - - # Generic agent provision task + - track-output: "{working-dir}/agent-{agent}/track.yaml" + foreach: + - variable: agents-os + as: agent + - task: "provision-install-{agent}" description: "Provision resources for the {agent} agent." do: @@ -155,62 +105,23 @@ Now, it is possible to use worklow engine library to launch provision module doi path: python3 args: - modules/provision/main.py - - inventory-agent: "{working-dir}/agent-{agent}/inventory.yaml" - - inventory-manager: "{working-dir}/manager-{manager-os}/inventory.yaml" + - inventory: "{working-dir}/agent-{agent}/inventory.yaml" + - dependencies: + - agent: "{working-dir}/agent-{agent}/inventory.yaml" - install: - - component: wazuh-agent - type: package - component: curl + - component: python + type: source + version: "{python-version}" + - component: virtualenv + type: pip + - component: deps/requirements.txt + type: pip + - component: pyyaml + type: pip + version: 6.0.1 depends-on: - "allocate-{agent}" - - "provision-manager" - foreach: - - variable: agents-os - as: agent - - # Generic agent provision task - - task: "provision-uninstall-{agent}" - description: "Provision resources for the {agent} agent." - do: - this: process - with: - path: python3 - args: - - modules/provision/main.py - - inventory-agent: "{working-dir}/agent-{agent}/inventory.yaml" - - inventory-manager: "{working-dir}/manager-{manager-os}/inventory.yaml" - - uninstall: - - component: wazuh-agent - type: package - depends-on: - - "provision-install-{agent}" - foreach: - - variable: agents-os - as: agent - - # Generic agent allocate task - - task: "allocate-{agent}" - description: "Allocate resources for the {agent} agent." - do: - this: process - with: - path: python3 - args: - - modules/allocation/main.py - - action: create - - provider: "{infra-provider}" - - size: small - - composite-name: "{agent}" - - inventory-output: "{working-dir}/agent-{agent}/inventory.yaml" - - track-output: "{working-dir}/agent-{agent}/track.yaml" - cleanup: - this: process - with: - path: python3 - args: - - modules/allocation/main.py - - action: delete - - track-output: "{working-dir}/agent-{agent}/track.yaml" foreach: - variable: agents-os as: agent @@ -218,7 +129,7 @@ Now, it is possible to use worklow engine library to launch provision module doi Following the schema of the example: - Configure the following parameters depending on your test case: + Configure the following parameters depending on your allocation: ```yaml variables/agent-os @@ -235,38 +146,37 @@ Now, it is possible to use worklow engine library to launch provision module doi depends-on ``` - >Note: In args, configure the launcher's path correctly (main.py files in each module), and to fill `depends-on`, consider the steps of your test (allocation, provision, and test) - -3. Execution of Command (local): + >Note: In args, configure the launcher's path correctly (main.py files in each module), and to fill `depends-on`, consider the steps of your provision (allocation and provision) - Execute the command by referencing the parameters required by the library (launcher). - - ```bash - python3 -m workflow_engine {.yaml fixture path} - ``` +7. Command execution (local): - Example + Execute the command by referencing the parameters required by the library (launcher). + + ```bash + python3 -m workflow_engine {.yaml fixture path} + ``` - ```bash - python3 -m workflow_engine modules/workflow_engine/examples/dtt1-agents-poc.yaml - ``` + Example - > Note The command execution can also be mediated through Jenkins. + ```bash + python3 -m workflow_engine modules/workflow_engine/examples/dtt1-agents-poc.yaml + ``` +#### B. Direct execution -If one wishes to execute the provision module without installing the workflow engine, they can proceed by using the launcher (module/provision/main.py): +To execute the Allocation module without installing the Workflow engine, you can use the launcher ('[module/provision/main.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/main.py)'): 1. Execution While in wazuh-qa/deployability ```bash - python3 module/provision/main.py --inventory-manager { inventory .yaml} --install "{'component': '{component}', 'type': '{type}'}" + python3 module/provision/main.py --inventory-manager { inventory .yaml} --install "{'component': '{component}', 'type': '{type}', 'version': '{version}'}" ``` Example: ```bash - python3 module/provision/main.py --inventory-manager /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml --install "{'component': 'wazuh-manager', 'type': 'package'}" + python3 module/provision/main.py --inventory-manager /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml --install "{'component': 'pyyaml', 'type': 'pip', 'version' : '6.0.1'}" ``` --- @@ -280,42 +190,58 @@ It can receive instructions either through the command line or be mediated throu In both cases, the following information is required: ```bash - - task: "provision-manager" - description: "Provision the manager." + - task: "provision-install-{agent}" + description: "Provision resources for the {agent} agent." do: this: process with: path: python3 args: - modules/provision/main.py - - inventory-manager: "{working-dir}/manager-{manager-os}/inventory.yaml" + - inventory: "{working-dir}/agent-{agent}/inventory.yaml" + - dependencies: + - agent: "{working-dir}/agent-{agent}/inventory.yaml" - install: - - component: wazuh-manager - type: package + - component: curl + - component: python + type: source + version: "{python-version}" + - component: virtualenv + type: pip + - component: deps/requirements.txt + type: pip + - component: pyyaml + type: pip + version: 6.0.1 + depends-on: + - "allocate-{agent}" + foreach: + - variable: agents-os + as: agent ``` -The snippet obtained from a Workflow fixture highlights the parameters necessary for its operation. These include the inventory path, the component to install, and the installation type. +The snippet obtained from a Workflow fixture highlights the parameters necessary for its operation. These include the inventory path, the dependency/library to install, and the installation type. -Of course, it requires an infrastructure on which to perform its actions. +It requires an infrastructure on which to perform its actions. The module is composed of: -- Launcher (/wazuh-qa/deployability/modules/provision/main.py) +- Launcher ('[/wazuh-qa/deployability/modules/provision/main.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/main.py)') Entry point for the workflow or the user who wishes to execute a provision. -- Parameter validator (/wazuh-qa/deployability/modules/provision/models.py) +- Parameter validator ('[/wazuh-qa/deployability/modules/provision/models.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/models.py)') Validator for parameters entered by the Workflow or the user. -- Module functions (/wazuh-qa/deployability/modules/provision/provision.py) +- Module functions ('[/wazuh-qa/deployability/modules/provision/provision.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/provision.py)') Module-specific functions responsible for triggering the provision. -- Component type categorizer (/wazuh-qa/deployability/modules/provision/component_type.py) +- Component type categorizer ('[/wazuh-qa/deployability/modules/provision/component_type.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/component_type.py)') Allows obtaining and expanding data based on initially received parameters. -- Executor (/wazuh-qa/deployability/modules/provision/actions.py) +- Executor ('[/wazuh-qa/deployability/modules/provision/actions.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/actions.py)') Executes the provision using Actions in the generic module. -- Utils (/wazuh-qa/deployability/modules/provision/utils.py) +- Utils ('[/wazuh-qa/deployability/modules/provision/utils.py](https://github.com/wazuh/wazuh-qa/tree/master/deployability/modules/provision/utils.py)') Enables reading and changing the format of YAML files to executable formats.