Allow a fixture to be reset on a given condition

[brianbruggeman]

We'd like a way of resetting a specific fixture (or maybe all of them) when an error condition occurs within a test.

Currently, I'm using pytest to test pyspark jobs. One of the limitations with spark is that the spark context is a singleton. As a consequence, if any single test fails, the spark context is rendered useless (more or less it's in an exception state). Creating an initial spark context takes time and can add a considerable amount of time. We can use fixtures to build that context:

@pytest.fixture(scope="session")
def spark_context(request, spark_settings):
    """ fixture for creating a spark context

    Args:
        request: pytest.FixtureRequest object
    """
    import sys
    import os
    from <redacted> import get_spark_context

    if sys.version.startswith('3'):
        os.environ['PYSPARK_PYTHON'] = 'python3'

    os.environ['SPARK_ENV_LOADED'] = "True"
    os.environ.pop('PYSPARK_SUBMIT_ARGS', None)

    spark_context_name = 'pytest-spark-context-unit-testing'
    spark_context = get_spark_context(appName=spark_context_name, **spark_settings)

    def close_context():
        spark_context.stop()
        spark_context._jvm.System.clearProperty("spark.driver.port")
    request.addfinalizer(close_context)

    return spark_context

Notably, we're using a scope of "session". This is to reduce the amount of time it takes to run any one individual spark job. However, due to the aforementioned problem of state, if any individual test fails and it is using the spark_context fixture above, then every subsequent test will also fail because the singleton spark_context fixture retains the state from previous runs.

If we had a way of resetting fixtures (or a specific fixture) when a failure occurs, this would be greatly preferred over recreating a spark_context for each individual test.

[RonnyPfannschmidt]

reset as a concept would introduce many breakages for dependent fixtures

thew concept that can probably work is to schedule a fixture and its dependent fixtures for the teardown of the next "minor" scope, so even a session scoped fixture may be removed at the

another way to handle it would be to make the fixture value a proxy object that supports resets/restarts on its own, so the lifetime of the inner object would no longer be exposed to the outer users of the fixture

[nicoddemus]

another way to handle it would be to make the fixture value a proxy object that supports resets/restarts on its own, so the lifetime of the inner object would no longer be exposed to the outer users of the fixture

@brianbruggeman I suggest trying the approach suggested b+y @RonnyPfannschmidt. IOW, make spark_context support a reset() method or such which can restore it to a valid state, or use a proxy object which recreates a new spark_context when a test failure occurs.

It seems to me implementing this support in pytest would be really complex and not worth in terms of future maintenance for a use case which is fairly exotic IMHO.

Feel free to follow up with any questions and we will be glad to help.

[brianbruggeman]

reset as a concept would introduce many breakages for dependent fixtures

I'm aware; this would require something like a DAG to determine dependencies so that they could be reset.

another way to handle it would be to make the fixture value a proxy object that supports resets/restarts on its own, so the lifetime of the inner object would no longer be exposed to the outer users of the fixture

This might be viable.

It seems to me implementing this support in pytest would be really complex and not worth in terms of future maintenance for a use case which is fairly exotic IMHO.

Maybe I could help? This response is somewhat distressing as it implies that you're unwilling to make changes because it doesn't fit your own set of use-cases. I was sort of under the impression that the point of open source was that not everyone has the same set of use cases and having a larger group here contribute makes the whole project better. But if this really is the viewpoint of your group, I'm less likely to want to contribute. Related more to your point, I think my request would be valuable for pytest related to any type of shared resource which stores state. The current scoping is sort of a stop-gap to address some of these issues, but certainly shouldn't be considered adequate for all use-cases.

[RonnyPfannschmidt]

@brianbruggeman while i feel with you, we have to cut at some things

reset aware dependency injection on a conceptual level is incredibly hard to get right, so if we tried to add it at the dependency injection level we would open the door to a multitude of hard to fix/comprehend bugs, which is pretty much suicide on a primarily volunteer driven project

on the other side, a system for expressing re-settable proxies with a distinction of inner re-settable implementation and outer facade can be layered into a dependency injection system without direct support reset support inside of the dependency injection system

while part of open-source is allowing others to chime in, its also very important to help people avoid playing architectural suicide, each value that is added also adds a cost, and sometimes the cost is very deceptive and incomprehensibly high in the end

sometimes it is critical to prevent people from adding concepts that are very much add odds at the same layer, in particular if putting them on different layers will add the same value but with vastly lower cost

[nicoddemus]

@RonnyPfannschmidt's response is to the point, let me just add a few more comments:

Maybe I could help? This response is somewhat distressing as it implies that you're unwilling to make changes because it doesn't fit your own set of use-cases.

I didn't mean it to come across that way, and I apologize if it did; we are more than happy to accept PRs for improvements and bug fixes, but we also have to balance that with the fact that new features always imply more maintenance costs down the road. For this specific problem it seemed to me that the easiest solution is to use a proxy object like @RonnyPfannschmidt suggested, as it is trivial to implement and can solve your problem right away. Also, I'm under the impression that the need for this feature is a rare occurrence, but this is based on my personal experiences only and I might be dead wrong.

Having said that, pytest is highly customizable so you can create a plugin which supports that use-case. That might be a good way to demonstrate a valid implementation and perhaps even in the future it might get added to the core. 😉