Validate Tests

[lazypower]

The tests to validate layer-filebeat are -ancient- and I don't think they've been used basically at all. We need some level of automated test suite in here so I can go into "satisfy the tests" mode to accept PR's with my limited time.

I'm calling all hands who care about the future of these charms and their longevity to lend a hand in contributing tests or test-harness setup.

Acceptance Criteria:

• Has automated unit tests (where applicable)
• Passes flake8 checks
• Has at least a single/simple integration test to validate log shipping happens through
  • Directly to ES
  • Routed through Logstash

With those two primary use cases sorted, plus the project meta I feel like we'll have a good enough path of coverage that I can ship features sent in by the community.

[lazypower]

@jamesbeedy @tbaumann can I lean on you to submit validation runs using the python juju framework to drive an automation test? (specifically wrt kafka) and/or a hand in setting up a test harness job using jenkins pipelines?

[tbaumann]

I didn't have much success running charm tests. But that is mostly to inadequacy of tools and documentation. With some pointers what to actually do, I'm sure I can get some stuff done.

[lazypower]

I'll spec out the elasticsearch / filebeat tests sometime this week and follow up with what I come up with. I'm sure i'll find dragons too :)

[lazypower]

Note to self: Take a look at this over the weekend. @jamesbeedy i saw some new elasticsearch work dropped in #juju this week. Are those in the store/available? Should I begin basing work off of that stack of charms?

[jamesbeedy]

@chuckbutler not in the store yet. I’ll cut a beta of the new elasticsearch charm this weekend. It uses unreleased charms.reactive, charmhelpers, and juju, so still a bit fresh. I think as long as you are running 2.3beta the built charm should work for you though. I’ll make note of this.

[lazypower]

Solid, I'll give it a go on both then once i've got the tests sorted + charm details. No rush/pressure I just wanted to make sure I wasn't targeting something deprecated since i've been out of the loop for a good while.

[jamesbeedy]

“submit validation runs using the python juju framework to drive an automation test? (specifically wrt kafka) and/or a hand in setting up a test harness job using jenkins pipelines?” - trying to hook this all up for my org right now, possibly I can roll the beats charms into the mix when things are up and running.

[barryprice]

Had a look at this when working on a feature addition, the existing tests are amulet-based which I suspect is no longer supported?

The instructions for amulet suggest installing from the PPA, which hasn't had a release since Focal (20.04), and installing the latest amulet via pip results in tracebacks as amulet uses outdated syntax:

Traceback (most recent call last):
  File "/home/barryprice/Documents/Canonical/charms/layer-filebeat/./tests/10-deploy-with-logstash", line 3, in <module>
    import amulet
  File "/home/barryprice/Documents/venvs/filebeat/lib/python3.10/site-packages/amulet/__init__.py", line 3, in <module>
    from .deployer import Deployment
  File "/home/barryprice/Documents/venvs/filebeat/lib/python3.10/site-packages/amulet/deployer.py", line 12, in <module>
    from path import tempdir
ImportError: cannot import name 'tempdir' from 'path' (/home/barryprice/Documents/venvs/filebeat/lib/python3.10/site-packages/path/__init__.py)

'tempdir' was renamed to 'TempDir' in 2018, and the backwards-compatible alias was removed in 2020, so it looks as if the tests need rewriting to drop amulet, assuming there are no updates coming there.