Can't build with --enable-tests in presence of circular dependencies

[bos]

Update by @ezyang: when per-component build merges #3662 the builder bits are all done; all we need to do is teach the solver how to make test suite deps private, and then propagate that through the nix-local-build code.

---

The text library has a tests stanza that I can't use because I get an error of this form:

Preprocessing test suite 'tests' for text-0.11.2.2...
<command line>: cannot satisfy -package-id text-0.11.2.2-inplace: 
    text-0.11.2.2-inplace is shadowed by package text-0.11.2.2-fa0303a547febb0fcc06906255b11d31

I believe the problem is caused by the fact that a library that text's tests depend on uses text itself.

I realise that this is likely to be difficult to fix, but it's a problem for "core" libraries: @tibbe reports that containers faces the same difficulty.

[bos]

You can see another instance of this bug in mwc-random issue 15.

[kosmikus]

Yes, see #935 for the containers problem.

[kolmodin]

The binary library faces the same problem.

[BardurArantsson]

I can certainly see that this is a problem, but does anyone have any reasonable suggestions for a fix? I mean, if you have cyclical dependencies that's (in itself) a huge problem for the integrity of your tests. Should cabal(-install) try to achieve some sort of fixed-point, or what?

AFAICT this is generally an issue of a "lack of Backpack", i.e. there are really no library "interfaces" -- only implementations. As such I don't see how this can actually be (even theoretically) be solved given the current state of GHC/Cabal.

Unless we get Backpack (or similar API separation) I don't see any other real solution other than breaking apart libraries appropriately such that types and instances are separate from implementation. (But even that may not be enough.)

My pronouncement would therefore be: "Break the dependency, libraries!" (I know it's not necessarily simple.)

EDIT: ... and that this isn't something that Cabal/cabal-install can (even theoretically) solve. Thus it should be closed.

[ttuegel]

@BardurArantsson The solution to this is actually very simple to state: "sandboxes always". Rather than installing the library-to-be-tested into the in-place package database, it should be installed into a sandbox. Then, its test dependencies can be installed into the sandbox. Finally, the tests can be built and run using the sandbox version of the library. This would be intellectually simple to do, but there's just a lot of things that need to be moved about to do it.

[BardurArantsson]

Ok... let's see it! What things need to be moved about and how would you do it? :)

I understand that if the circularity is only because test-X-depends-on-impl-Y-which-depends-on-tests-for-X, it should be easy do to a sort of "tiered" compilation where the two impls are compiled first (without testing) and then tests are run afterwards. However, I didn't get the impression that that was the case from the discussion. My bad, if that was wrong. Especially since there should be no way for an impl-of-X to depend on Y-and-test-of-Y, only Y. If it's just a matter of not running tests in some circumstances, then I'm with you.

I still maintain that you should "just" separate your tests into a separate project in that case.

[ttuegel]

Ok... let's see it! What things need to be moved about and how would you do it? :)

1. Split the configure phase apart so each component can be configured separately. This will be where things are moved around, but I can't go into more detail without going and doing it.
2. Configure tests to build against the in-place package database by default (in Cabal).
3. Have cabal-install configure tests to build against the sandbox, rather than the in-place package database.

I still maintain that you should "just" separate your tests into a separate project in that case.

Cabal doesn't support tests-only packages, and cabal-install doesn't (yet) have features for working with multiple packages simultaneously. There are open tickets for both of these features.

[dcoutts]

We think we've got a solution to this, see the last section of http://www.well-typed.com/blog/2015/07/cabal-setup-deps/

This would also lets us get rid of the fallback for when configure fails to find a solution.

[tibbe]

Anyone willing to implement?

[BardurArantsson]

Cabal doesn't support tests-only packages, and cabal-install doesn't (yet) have features for working with multiple packages simultaneously. There are open tickets for both of these features.

Yes it does. You just avoid calling it a "test" project. Instead you call it a "testkit" and make it a real (normal) project. Then you depend on it in your "real" project's "test" section.

(I think the concept should be easily understood by looking a bit at the documentation for akka-testkit project.)

The general approach is more general than just this use case, but unfortunately I don't have a Haskell (demo) project to publish just ATM. (I mean, I have it, but it's not quite ready for publication.)

Does this make sense? Hopefully, I'll be able to publish my example soonish, but please look into akka-testkit in the mean time, or ask further questions! This really isn't that difficult (without extending Cabal or whatever).

[dcoutts]

See also PR #3422

[ezyang]

When per-component build merges #3662 the builder bits are all done; all we need to do is teach the solver how to make test suite deps private, and then propagate that through the nix-local-build code.

[grayjay]

I'm closing this in favor of #1575, which has more discussion.