Inverse markers

[kjeldflarup]

There is a posibility to use markers
@pytest.mark.playstation

And then run pytest -m "not playstation"
That would run all my tests except the playstation tests. Now if I want a complete playstation test, I just type pytest.
That however gives me errors on xbox, Wii, gamebody, nintendods etc.

Thus a test must be executed in this cumbersome way
pytest -m "not xbox and not Wii and not gamebody and not nintendods"

Is there a smarter way to do this? The markers operates quite inverse of what I need.

I could of course use skip, but then I would get a zillion skip messages, where I only wanted to know the skip messages for the playstation.

A decorator like @pytest.ignoreif(... ) would be nice.

[pytestbot]

GitMate.io thinks possibly related issues are #277 (Markers as functions), #1603 (Get marker list of an item), #631 (markers leaking), #568 (markers stains on all related classes), and #1415 (Ability to register markers from plugins).

[robert-cody]

pytest -m 'playstation'?

[kjeldflarup]

pytest -m "playstation" does not run common tests for all platforms.

[robert-cody]

That's what you need, right?

You need:

pytest -m 'not xbox and not Wii and not gamebody and not nintendods'

Smarter way for the same result:

pytest -m 'playstation'

This command will run "complete playstation test", but not others.

[kjeldflarup]

No Your solution does not run test_common_login

@pytest.mark.playstation
def test_playstation():
    pass

def test_common_login():
    pass

[RonnyPfannschmidt]

as far as i understood you are trying to use markers for environment selection - mark expressions are simply not designed for that, and shouldn't be

[robert-cody]

You can apply mark 'common' for common tests and run it like this:
pytest -m 'common or playstation'

[kjeldflarup]

@RonnyPfannschmidt is there any way to do environment selection then?

[RonnyPfannschmidt]

@kjeldflarup i do have something about it in a work project, unfortunately it's not going to be extracted soon, but basically this needs to be experimented with in a plug-in since its easy to get the actual semantics wrong as long as there isn't more than one consumer

[kjeldflarup]

I found a solution by making pytest_collection_modifyitems remove the tests.
In this way I could also control ex. a --nightly switch to extend the test.

def pytest_addoption(parser):
    group = parser.getgroup("DEIF")
    group.addoption(
        "--duttype",
        action="store",
        dest="name",
        default=None,
        help='When dut marked, only run this dut.',
    )

def pytest_collection_modifyitems(session,config,items):
    duttype = config.getoption("--duttype")
    keep, discard = [], []
    i = 0
    for item in items:
        duts = item.get_marker('dut')
        if duts:
            # search the dut marker, to see if duttype is marked
            # multiple dut markers allowed
            found = False
            for dut in duts:
                for dutargs in dut.args:
                    if dutargs == duttype:
                        found = True
            if found:
                keep.append(item)
            else:
                discard.append(item)
        else:
            # dut marker not set, always run the test
            keep.append(item)

    items[:] = keep
    # The deselected information is not interesting
    #config.hook.pytest_deselected(items=discard)

[esben]

See https://pytest.readthedocs.io/en/reorganize-docs/example/markers.html#marking-platform-specific-tests-with-pytest
which can simple be extended to work with a command-line specified (target) platform, if sys.platform does not suffice.

[Zac-HD]

Closing this as a combination of inactive, has workarounds, and markers seeming like the wrong solution 😕