Skip to content

Commit

Permalink
Replace workflow_engine references to jobflow
Browse files Browse the repository at this point in the history
  • Loading branch information
Marcelo Ariel Hamra committed Jun 7, 2024
1 parent df1d15c commit 8b07c42
Show file tree
Hide file tree
Showing 64 changed files with 337 additions and 249 deletions.
12 changes: 6 additions & 6 deletions deployability/README.MD
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## JobFlow - Deployability general documentation
## Deployability general documentation

### User documentation

Expand Down Expand Up @@ -41,14 +41,14 @@ While in wazuh-qa:

```
cd modules
pip3 uninstall -y workflow_engine && pip3 install .
pip3 uninstall -y jobflow && pip3 install .
```

5. Test Fixture to Execute

It will be necessary to create a fixture (yaml file) where the infrastructure, provisioning, and tests to be executed will be declared.

> Note: It is possible to find some fixture examples in deployability/modules/workflow_engine/examples/
> Note: It is possible to find some fixture examples in deployability/modules/jobflow/examples/

Example:
Expand Down Expand Up @@ -247,13 +247,13 @@ depends-on
Execute the command by referencing the parameters required by the library (launcher).

```
python3 -m workflow_engine {.yaml fixture path}
python3 -m jobflow {.yaml fixture path}
```

Example

```
python3 -m workflow_engine modules/workflow_engine/examples/dtt1-agents-poc.yaml
python3 -m jobflow modules/jobflow/examples/dtt1-agents-poc.yaml
```


Expand Down Expand Up @@ -285,7 +285,7 @@ wazuh-qa/
│ ├── generic
│ ├── provision
│ ├── testing
│ └── workflow_engine
│ └── jobflow
└── plugins
```
Deployability contains the following directories:
Expand Down
10 changes: 5 additions & 5 deletions deployability/modules/allocation/README.MD
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## JobFlow - Allocation Module
## Allocation Module

### User documentation

Expand Down Expand Up @@ -56,14 +56,14 @@ Now, it is possible to use the Worklow engine library to launch the provision mo

```bash
cd modules
pip3 uninstall -y workflow_engine && pip3 install .
pip3 uninstall -y jobflow && pip3 install .
```

2. Test Fixture to Execute:

It will be necessary to create a fixture (yaml file) where the infrastructure, provisioning, and tests to be executed will be declared.

>Note: It is possible to find some fixture examples in [deployability/modules/workflow_engine/examples/](../workflow_engine/examples)
>Note: It is possible to find some fixture examples in [deployability/modules/jobflow/examples/](../jobflow/examples)
Example:

Expand Down Expand Up @@ -161,13 +161,13 @@ Now, it is possible to use the Worklow engine library to launch the provision mo
Execute the command by referencing the parameters required by the library (launcher).

```bash
python3 -m workflow_engine {.yaml fixture path}
python3 -m jobflow {.yaml fixture path}
```

Example

```bash
python3 -m workflow_engine modules/workflow_engine/examples/dtt1-agents-poc.yaml
python3 -m jobflow modules/jobflow/examples/dtt1-agents-poc.yaml
```


Expand Down
8 changes: 8 additions & 0 deletions deployability/modules/jobflow.egg-info/PKG-INFO
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Metadata-Version: 2.1
Name: jobflow
Version: 1.0
Summary: Wazuh testing utilities to help programmers automate deployment tests
Home-page: https://github.com/wazuh
Author: Wazuh
Author-email: [email protected]
License: GPLv2
73 changes: 73 additions & 0 deletions deployability/modules/jobflow.egg-info/SOURCES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
setup.py
jobflow/README.MD
jobflow/__init__.py
jobflow/__main__.py
jobflow/models.py
jobflow/requirements-dev.txt
jobflow/schema_validator.py
jobflow/task.py
jobflow/workflow_processor.py
jobflow.egg-info/PKG-INFO
jobflow.egg-info/SOURCES.txt
jobflow.egg-info/dependency_links.txt
jobflow.egg-info/entry_points.txt
jobflow.egg-info/not-zip-safe
jobflow.egg-info/top_level.txt
jobflow/__pycache__/__init__.cpython-310.pyc
jobflow/__pycache__/__main__.cpython-310.pyc
jobflow/__pycache__/models.cpython-310.pyc
jobflow/__pycache__/schema_validator.cpython-310.pyc
jobflow/__pycache__/task.cpython-310.pyc
jobflow/__pycache__/workflow_processor.cpython-310.pyc
jobflow/examples/agent/aws/Jenkins-agents-test.yaml
jobflow/examples/agent/aws/test-agent-basic-info.yaml
jobflow/examples/agent/aws/test-agent-complete-macOS.yaml
jobflow/examples/agent/aws/test-agent-complete.yaml
jobflow/examples/agent/aws/test-agent-restart-ins-prov.yaml
jobflow/examples/agent/aws/test-agent-stop-ins-prov.yaml
jobflow/examples/agent/aws/test-agent-suse.yaml
jobflow/examples/agent/aws/test-agent-time.yaml
jobflow/examples/agent/aws/test-agent-uninstall-ins-prov.yaml
jobflow/examples/agent/aws/test-agent-windows-complete.yaml
jobflow/examples/agent/aws/test-full-agent-with-live.yaml
jobflow/examples/agent/vagrant/test-agent-basic-info-vagrant.yaml
jobflow/examples/agent/vagrant/test-agent-complete-1.yaml
jobflow/examples/agent/vagrant/test-agent-complete-2.yaml
jobflow/examples/agent/vagrant/test-agent-complete-macOS.yaml
jobflow/examples/agent/vagrant/test-agent-restart-ins-prov-1.yaml
jobflow/examples/agent/vagrant/test-agent-restart-ins-prov-2.yaml
jobflow/examples/agent/vagrant/test-agent-stop-ins-prov-1.yaml
jobflow/examples/agent/vagrant/test-agent-stop-ins-prov-2.yaml
jobflow/examples/agent/vagrant/test-agent-uninstall-ins-prov-1.yaml
jobflow/examples/agent/vagrant/test-agent-uninstall-ins-prov-2.yaml
jobflow/examples/agent/vagrant/test-agent-windows-complete.yaml
jobflow/examples/central_components/aws/Jenkins-central-component-test.yaml
jobflow/examples/central_components/aws/dtt1-central_components-poc-aws.yaml
jobflow/examples/central_components/vagrant/dtt1-central_components-poc-vagrant.yaml
jobflow/examples/manager/aws/Jenkins-manager-test.yaml
jobflow/examples/manager/aws/dtt1-managers-poc-aws.yaml
jobflow/examples/manager/vagrant/dtt1-managers-poc-vagrant.yaml
jobflow/logger/__init__.py
jobflow/logger/config.yaml
jobflow/logger/filter.py
jobflow/logger/logger.py
jobflow/logger/__pycache__/__init__.cpython-310.pyc
jobflow/logger/__pycache__/logger.cpython-310.pyc
jobflow/schemas/schema_v1.json
jobflow/tests/TESTING-README.md
jobflow/tests/conftest.py
jobflow/tests/test_dag.py
jobflow/tests/test_schema_validator.py
jobflow/tests/test_task.py
jobflow/tests/test_workflow_file.py
jobflow/tests/test_workflow_processor.py
jobflow/tests/__pycache__/conftest.cpython-310-pytest-7.4.4.pyc
jobflow/tests/__pycache__/test_dag.cpython-310-pytest-7.4.4.pyc
jobflow/tests/__pycache__/test_schema_validator.cpython-310-pytest-7.4.4.pyc
jobflow/tests/__pycache__/test_task.cpython-310-pytest-7.4.4.pyc
jobflow/tests/__pycache__/test_workflow_file.cpython-310-pytest-7.4.4.pyc
jobflow/tests/__pycache__/test_workflow_processor.cpython-310-pytest-7.4.4.pyc
jobflow/tests/data/wf-ko-no-path-on-cleanup.yaml
jobflow/tests/data/wf-ko-no-path-on-do.yaml
jobflow/tests/data/wf-ko-schema-error.yaml
jobflow/tests/data/wf-ok.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

2 changes: 2 additions & 0 deletions deployability/modules/jobflow.egg-info/entry_points.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[console_scripts]
engine = jobflow.__main__:main
1 change: 1 addition & 0 deletions deployability/modules/jobflow.egg-info/not-zip-safe
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

1 change: 1 addition & 0 deletions deployability/modules/jobflow.egg-info/top_level.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
jobflow
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ To use this module, you should use a Debian-based system, we recommend using Ubu

```bash
cd modules
pip3 uninstall -y workflow_engine && pip3 install .
pip3 uninstall -y jobflow && pip3 install .
```

5. Test Fixture to Execute:

It will be necessary to create a fixture (yaml file) where the infrastructure, provisioning, and tests to be executed will be declared.

>Note: It is possible to find some fixture examples in deployability/modules/workflow_engine/examples/
>Note: It is possible to find some fixture examples in deployability/modules/jobflow/examples/
Example:

Expand Down Expand Up @@ -240,13 +240,13 @@ To use this module, you should use a Debian-based system, we recommend using Ubu
Execute the command by referencing the parameters required by the library (launcher).

```bash
python3 -m workflow_engine {.yaml fixture path}
python3 -m jobflow {.yaml fixture path}
```

Example

```bash
python3 -m workflow_engine modules/workflow_engine/examples/dtt1-agents-poc.yaml
python3 -m jobflow modules/jobflow/examples/dtt1-agents-poc.yaml
```

> Note The command execution can also be mediated through Jenkins.
Expand All @@ -260,7 +260,7 @@ To use this module, you should use a Debian-based system, we recommend using Ubu
Its function is to allow the ordered and structured execution in steps of allocation, provision, and testing.

`The JobFlow workflow engine` receives instructions through a `YAML document`, the structure of which can be exemplified in tests found in:
`wazuh-qa/deployability/modules/workflow_engine/examples`
`wazuh-qa/deployability/modules/jobflow/examples`

**In these tests**:
- Tasks: define the steps.
Expand Down Expand Up @@ -303,15 +303,15 @@ tasks:
as: agent
```

These tasks are executed by the `JobFlow workflow engine` launcher installed as workflow_engine library in your virtual environment.
These tasks are executed by the `JobFlow workflow engine` launcher installed as jobflow library in your virtual environment.

This launcher receives the parameters, sets up the test logs, and proceeds with the ordered execution.

The parameters sent from the launcher are processed by deployability/modules/workflow_engine/models.py, which checks the nature of the parameters sent and filters out incorrect parameters.
The parameters sent from the launcher are processed by deployability/modules/jobflow/models.py, which checks the nature of the parameters sent and filters out incorrect parameters.

![image](https://github.com/wazuh/wazuh-qa/assets/125690423/32aa77b7-f294-41ac-af93-db8a084dbad1)

These are then sent to `deployability/modules/workflow_engine/workflow_processor.py`, where using `deployability/modules/schemas`, instructions in YAML are received and the schema of the instructions is checked.
These are then sent to `deployability/modules/jobflow/workflow_processor.py`, where using `deployability/modules/schemas`, instructions in YAML are received and the schema of the instructions is checked.

The commands are executed in the WorkflowProcessor of the same file, which also handles parallel executions and aborts failed executions.

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))
sys.path.append(project_root)

from workflow_engine.workflow_processor import WorkflowProcessor
from workflow_engine.models import InputPayload
from jobflow.workflow_processor import WorkflowProcessor
from jobflow.models import InputPayload


def parse_arguments() -> argparse.Namespace:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ tasks:
- track-output: "{working-dir}/manager-{manager-os}/track.yaml"
- label-termination-date: "1d"
- label-team: "qa"
- instance-name: "manager"
on-error: "abort-all"
cleanup:
this: process
Expand Down Expand Up @@ -57,6 +58,7 @@ tasks:
- track-output: "{working-dir}/agent-{agent}/track.yaml"
- label-termination-date: "1d"
- label-team: "qa"
- instance-name: "agent-instance-{agent}"
on-error: "abort-all"
foreach:
- variable: agent-os
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ def _load_config() -> None:

_load_config()

logger = logging.getLogger("workflow_engine")
logger = logging.getLogger("jobflow")
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from pathlib import Path
from ruamel.yaml import YAML

from workflow_engine.logger.logger import logger
from jobflow.logger.logger import logger

class SchemaValidator:
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import time

from abc import ABC, abstractmethod
from workflow_engine.logger.logger import logger
from jobflow.logger.logger import logger

class Task(ABC):
"""Abstract base class for tasks."""
Expand Down
Loading

0 comments on commit 8b07c42

Please sign in to comment.