allow fixtures to provide extra info when a test fails

[hpk42]

Often in more complex functional settings I want to see some information about database file paths, hosts, ports etc. Printing it easily looses it in an ocean of other printed stuff. We could add an API request.add_info_on_failure(msg) which is shown as a separate section (just as with capturing).

[hpk42]

to be clear: i only want this information to be shown when a test fails and it's specific to that test and the fixtures it uses.

[RonnyPfannschmidt]

can we instead have reports of fixtures be separated and debuggable?

[RonnyPfannschmidt]

the idea is, that for fixtures that may even have different scopes, the reports should be accessible from other test items as well

we might also want to avoid running tests, for example if a session scoped database connection fails, its should just reprot that to each test using it

an comprehensible for a more fine grained reporting is on my todo

[hpk42]

could yo elaborate "be separated and debuggable"? do you mean one section per fixture which adds failure info?

[hpk42]

avoiding running tests/caching fixture failures is a separate issues and unreleated.

[RonnyPfannschmidt]

first we should have a way to have separate reports for each fixture, this is also important to comptehend setup of tests that use a cached fixture,

currently the teardown and setup of cached items and their scope is structurally just put on random test items that happen to be at the intersection

i would like to see that reported in a more comprehensible manner (ie. separate from test items that just happen to be there)

[hpk42]

now i am beggiing to understand i think, thanks. So you want to have reports for each fixture and they would be shown by default on failure, possibly (or switched in via an option). For that we need to have a protocol/hooks/API which is called when a fixture function is called i guess. That's an interesting proposition but obviously much larger scale change than what i am proposing.

[RonnyPfannschmidt]

i have a proposal in mind for sectioned reports that would support nesting and context,
but the topic needs a spec and a longer discussion