Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more entry to the docs #244

Merged
merged 6 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .config/dictionary.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ envtmpdir
envtmpdir
fileh
fixturenames
libera
metafunc
nocolor
passenv
Expand Down
186 changes: 2 additions & 184 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,6 @@ Install from pypi:
pip install tox-ansible
```

## Dependencies

`tox-ansible` will install additional dependencies if necessary:

- `tox` version 4.0 or greater.
- `pytest-ansible` version 3.1.0 or greater.
- `pytest`
- `pytest-xdist`
- `pyyaml`

## Usage

From the root of your collection, create an empty `tox-ansible.ini` file and list the available environments:
Expand Down Expand Up @@ -145,181 +135,9 @@ skip =
devel
```

This will skip tests in any environment that uses Ansible 2.9 or the devel branch. The list of strings is used for a simple string in string comparison of environment names.

## Overriding the configuration

Any configuration in either the `[testenv]` section or am environment section `[testenv:integration-py3.11-{devel,milestone}]` can override some or all of the `tox-ansible` environment configurations.

For example

```ini

[testenv]
commands_pre =
true

[testenv:integration-py3.11-{devel,milestone}]
commands =
true
```

will result in:

```ini
# tox-ansible.ini
[testenv:integration-py3.11-devel]
commands_pre = true
commands = true
```

Used without caution, this configuration can result in unexpected behavior, and possible false positive or false negative test results.

## Passing command line arguments to ansible-test / pytest

The behavior of the `ansible-test` (for `sanity-*` environments) or `pytest` (for `unit-*` and `integration-*` environments) commands can be customized by passing further command line arguments to it, e.g., by invoking `tox` like this:

```bash
tox -f sanity --ansible --conf tox-ansible.ini -- --test validate-modules -vvv
```

The arguments after the `--` will be passed to the `ansible-test` command. Thus in this example, only the `validate-modules` sanity test will run, but with an increased verbosity.

Same can be done to pass arguments to the `pytest` commands for the `unit-*` and `integration-*` environments:

```bash
tox -e unit-py3.11-2.15 --ansible --conf tox-ansible.ini -- --junit-xml=tests/output/junit/unit.xml
```

## Usage in a CI/CD pipeline

The repo contains a GitHub workflow that can be used in a GitHub actions CI/CD pipeline. The workflow will run all tests across all available environments unless limited by the `skip` option in the `tox-ansible.ini` file.

Each environment will be run in a separate job. The workflow will run all jobs in parallel.

The GitHub matrix is dynamically created by `tox-ansible` using the `--gh-matrix` and `--ansible` flags. The list of environments is converted to a list of entries in json format and added to the file specified by the "GITHUB_OUTPUT" environment variable. The workflow will read this file and use it to create the matrix.

A sample use of the GitHub workflow might look like this:

```yaml
name: Test collection

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

on:
pull_request:
branches: [main]
workflow_dispatch:

jobs:
tox-ansible:
uses: tox-dev/tox-ansible/.github/workflows/run.yml@main
```

Sample `json`

```json
[
// ...
{
"description": "Integration tests using ansible-core devel and python 3.11",
"factors": ["integration", "py3.11", "devel"],
"name": "integration-py3.11-devel",
"python": "3.11"
}
// ...
]
```

## Testing molecule scenarios

Although the `tox-ansible` plugin does not have functionality specific to molecule, it can be a powerful tool to run `molecule` scenarios across a matrix of Ansible and Python versions.

This can be accomplished by presenting molecule scenarios as integration tests available through `pytest` using the [pytest-ansible](https://github.com/ansible-community/pytest-ansible) plugin, which is installed when `tox-ansible` is installed.

Assuming the following collection directory structure:

```bash
namespace.name
├── extensions
│ ├── molecule
│ │ ├── playbook
│ │ │ ├── create.yml
│ │ │ ├── converge.yml
│ │ │ ├── molecule.yml
│ │ │ └── verify.yml
│ │ ├── plugins
│ │ │ ├── create.yml
│ │ │ ├── converge.yml
│ │ │ ├── molecule.yml
│ │ │ └── verify.yml
│ │ ├── targets
│ │ │ ├── create.yml
│ │ │ ├── converge.yml
│ │ │ ├── molecule.yml
│ │ │ └── verify.yml
├── playbooks
│ └── site.yaml
├── plugins
│ ├── action
│ │ └── action_plugin.py
│ ├── modules
│ │ └── module.py
├── tests
│ ├── integration
│ │ │── targets
│ │ │ ├── success
│ │ │ │ └── tasks
│ │ │ │ └── main.yaml
│ │ └── test_integration.py
├── tox-ansible.ini
└── tox.ini
```

Individual `molecule` scenarios can be added to the collection's extension directory to test playbooks, roles, and integration targets.

In order to present each `molecule` scenario as an individual `pytest` test a new `helper` file is added.

```python
# tests/integration/test_integration.py

"""Tests for molecule scenarios."""
from __future__ import absolute_import, division, print_function

from pytest_ansible.molecule import MoleculeScenario


def test_integration(molecule_scenario: MoleculeScenario) -> None:
"""Run molecule for each scenario.

:param molecule_scenario: The molecule scenario object
"""
proc = molecule_scenario.test()
assert proc.returncode == 0
```

The `molecule_scenario` fixture parametrizes the `molecule` scenarios found within the collection and creates an individual `pytest` test for each which will be run during any `integration-*` environment.

This approach provides the flexibility of running the `molecule` scenarios directly with `molecule`, `pytest` or `tox`. Additionally, presented as native `pytest` tests, the `molecule` scenarios should show in the `pytest` test tree in the user's IDE.

## How does it work?

`tox` will, by default, create a Python virtual environment for a given environment. `tox-ansible` adds Ansible collection specific build and test logic to tox. The collection is copied into the virtual environment created by tox, built, and installed into the virtual environment. The installation of the collection will include the collection's collection dependencies. `tox-ansible` will also install any Python dependencies from a `test-requirements.txt` (or `requirements-test.txt`) and `requirements.txt` file. The virtual environment's temporary directory is used, so the copy, build and install steps are performed with each test run ensuring the current collection code is used.

`tox-ansible` also sets the `ANSIBLE_COLLECTIONS_PATH` environment variable to point to the virtual environment's temporary directory. This ensures that the collection under test is used when running tests. The `pytest-ansible-units` pytest plugin injects the `ANSIBLE_COLLECTIONS_PATH` environment variable into the collection loader so ansible-core can locate the collection.

`pytest` is ued to run both the `unit` and `integration tests`.
`ansible-test sanity` is used to run the `sanity` tests.

For full configuration examples for each of the sanity, integration, and unit tests including the commands being run and the environment variables being set and passed, see the following:

- [integration](docs/integration.ini)
- [sanity](docs/sanity.ini)
- [unit](docs/unit.ini)
This will skip tests in any environment that uses Ansible 2.9 or the devel branch. The list of strings is used for a simple string in string comparison of environment names. Here is the [guide] to override `tox-ansible` environment configuration.

See the [tox documentation](https://tox.readthedocs.io/en/latest/) for more information on tox.
[guide]: https://ansible.readthedocs.io/projects/tox-ansible/configuration/#overriding-the-configuration

## Note to version 1.x users

Expand Down
41 changes: 41 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Configuration

`tox-ansible` should be configured using a `tox-ansible.ini` file. Using a `tox-ansible.ini` file allows for the introduction of the `tox-ansible` plugin to a repository that may already have an existing `tox` configuration without conflicts. If no configuration overrides are needed, the `tox-ansible.ini` file may be empty but should be present. In addition to all `tox` supported keywords the `ansible` section and `skip` keyword are available:

```ini
# tox-ansible.ini
[ansible]
skip =
2.9
devel
```

This will skip tests in any environment that uses Ansible 2.9 or the devel branch. The list of strings is used for a simple string in string comparison of environment names.

### Overriding the configuration

Any configuration in either the `[testenv]` section or an environment section `[testenv:integration-py3.11-{devel,milestone}]` can override some or all of the `tox-ansible` environment configurations.

For example

```ini

[testenv]
commands_pre =
true

[testenv:integration-py3.11-{devel,milestone}]
commands =
true
```

will result in:

```ini
# tox-ansible.ini
[testenv:integration-py3.11-devel]
commands_pre = true
commands = true
```

Used without caution, this configuration can result in unexpected behavior, and possible false positive or false negative test results.
50 changes: 50 additions & 0 deletions docs/contributor_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Contributor Guide

To contribute to `tox-ansible`, please use pull requests on a branch of your own fork.

After [creating your fork on GitHub], you can do:

```shell-session
$ git clone --recursive [email protected]:your-name/tox-ansible
$ cd tox-ansible
$ git checkout -b your-branch-name
# DO SOME CODING HERE
$ git add your new files
$ git commit -v
$ git push origin your-branch-name
```

You will then be able to create a pull request from your commit.

Prerequisites:

1. All fixes to core functionality (i.e. anything except docs or examples) should
be accompanied by tests that fail prior to your change and succeed afterwards.

2. Before sending a PR, make sure that `tox -e lint` passes.

Feel free to raise issues in the repo if you feel unable to contribute a code
fix.

## Talk to us

Use Github [discussions] forum or for a live chat experience try
`#ansible-devtools` IRC channel on libera.chat or Matrix room
[#devtools:ansible.com](https://matrix.to/#/#devtools:ansible.com).

For the full list of Ansible IRC and Mailing list, please see the [Ansible
Communication] page. Release announcements will be made to the [Ansible
Announce] list.

Possible security bugs should be reported via email to
<mailto:[email protected]>.

## Code of Conduct

Please see the official [Ansible Community Code of Conduct].

[discussions]: https://github.com/ansible/tox-ansible/discussions
[ansible communication]: https://docs.ansible.com/ansible/latest/community/communication.html
[ansible announce]: https://groups.google.com/forum/#!forum/ansible-announce
[Ansible Community Code of Conduct]: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html
[creating your fork on github]: https://docs.github.com/en/get-started/quickstart/contributing-to-projects
18 changes: 18 additions & 0 deletions docs/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Frequently asked questions

## How does it work?

`tox` will, by default, create a Python virtual environment for a given environment. `tox-ansible` adds Ansible collection specific build and test logic to tox. The collection is copied into the virtual environment created by tox, built, and installed into the virtual environment. The installation of the collection will include the collection's collection dependencies. `tox-ansible` will also install any Python dependencies from a `test-requirements.txt` (or `requirements-test.txt`) and `requirements.txt` file. The virtual environment's temporary directory is used, so the copy, build and install steps are performed with each test run ensuring the current collection code is used.

`tox-ansible` also sets the `ANSIBLE_COLLECTIONS_PATH` environment variable to point to the virtual environment's temporary directory. This ensures that the collection under test is used when running tests. The `pytest-ansible-units` pytest plugin injects the `ANSIBLE_COLLECTIONS_PATH` environment variable into the collection loader so ansible-core can locate the collection.

`pytest` is used to run both the `unit` and `integration tests`.
`ansible-test sanity` is used to run the `sanity` tests.

For full configuration examples for each of the sanity, integration, and unit tests including the commands being run and the environment variables being set and passed, see the following:

- [integration](https://github.com/ansible/tox-ansible/blob/main/docs/integration.ini)
- [sanity](https://github.com/ansible/tox-ansible/blob/main/docs/sanity.ini)
- [unit](https://github.com/ansible/tox-ansible/blob/main/docs/unit.ini)

See the [tox documentation](https://tox.readthedocs.io/en/latest/) for more information on tox.
17 changes: 17 additions & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Installation

Getting started with tox-ansible is as simple as:

```bash
pip install tox-ansible
```

## Dependencies

`tox-ansible` will install additional dependencies if necessary:

- `tox` version 4.0 or greater.
- `pytest-ansible` version 3.1.0 or greater.
- `pytest`
- `pytest-xdist`
- `pyyaml`
Loading