Nondeterministic groundhog compile errors

[mightybyte]

I have been getting occasional nondeterministic compile errors with groundhog for quite some time. I haven't said anything before now because I have no idea how to reproduce them or supply a minimal test case demonstrating the problem. I recently upgraded one of my large groundhog projects to GHC 7.8 and now the problem seems to be happening more often. Here's the error I get:

src/Field.hs:140:35:
    Could not deduce (PrimitivePersistField String)
      arising from a use of ‘toPrimitivePersistValue’
    from the context (DbDescriptor db)
      bound by the type signature for
                 toPrimitivePersistValue :: DbDescriptor db =>
                                            proxy db -> Field -> PersistValue
      at src/Field.hs:140:5-27
    In the expression: toPrimitivePersistValue p
    In the expression: toPrimitivePersistValue p $ show x
    In an equation for ‘toPrimitivePersistValue’:
        toPrimitivePersistValue p x = toPrimitivePersistValue p $ show x

src/Field.hs:141:44:
    Could not deduce (PrimitivePersistField String)
      arising from a use of ‘fromPrimitivePersistValue’
    from the context (DbDescriptor db)
      bound by the type signature for
                 fromPrimitivePersistValue :: DbDescriptor db =>
                                              proxy db -> PersistValue -> Field
      at src/Field.hs:141:5-29
    In the second argument of ‘($)’, namely
      ‘fromPrimitivePersistValue p x’
    In the expression: read $ fromPrimitivePersistValue p x
    In an equation for ‘fromPrimitivePersistValue’:
        fromPrimitivePersistValue p x
          = read $ fromPrimitivePersistValue p x

These errors always seem to come in pairs. Here is the code on lines 139-141 of that file.

instance PrimitivePersistField Field where
    toPrimitivePersistValue p x = toPrimitivePersistValue p $ show x
    fromPrimitivePersistValue p x = read $ fromPrimitivePersistValue p x

I've also seen the error in other places where I'm defining a PrimitivePersistField instance. The bizarre thing is that the solution to the problem is simply cabal clean and then build again. This usually makes the problem go away. Since the upgrade from GHC 7.6 to 7.8.4, the problem comes up more often and now sometimes I have to do several cabal cleans before it will compile successfully. Unfortunately I can't show you the code because it's a proprietary project and I haven't ever been able to reproduce the problem anywhere else. I know this doesn't give you much to go on, but I wanted to open this issue to at least make the problem known.

[lykahb]

Thank you for the report. This looks weird, I've never seen nondeterministic compile errors. The lines 139-141 for a plain datatype compile just fine on my machine.

[mightybyte]

Yeah, I imagine you would have to have a bigger project to trigger it, but I have no idea what the trigger might be.

[mightybyte]

FWIW, that error was showing up somewhere around module 28 of 135 or so.

[mightybyte]

It dawned on me that perhaps threading could cause behavior like this, so I started compiling with -j1. I haven't built it enough times to be confident yet, but thus far the error has not happened a single time when I've compiled with -j1. If this is indeed the reason, it seems like this would probably be classified as a GHC bug. I'll check around and see if anything like that exists there. But as a practical matter of making groundhog better for its users it might be worth thinking if there's anything you could do differently to help this situation.

[lykahb]

I remember having missing instances error when I was linking against a wrong package version (not groundhog) several years ago... Groundhog does not seem to do much hackery to cause the bug. The most hacky pragma I use, UndecidableInstances is supposed to have deterministic behavior. Please reopen this ticket if you discover a simple way to reproduce it.

[mightybyte]

Ok, thanks to help from Richard Eisenberg at hac-phi I have figured out this issue. It was caused because we were using this set of imports.

import Database.Groundhog.Core
import Database.Groundhog.Generic

Your instances are defined in Instances.hs. The above two modules export the type classes but not the instances. The instances are only exported from Database.Groundhog. So this was definitely a problem with our code, but it would sometimes build correctly if a module that imported Database.Groundhog was compiled first because of GHC's instance caching.

In short, the root cause of this is groundhog's orphan instances. You can eliminate the possibility of encountering this problem by exporting the instances from .Core and .Generic.

[lykahb]

Thank you for the analysis. I would expect troubles with orphan instances only when they are defined several times for the same type. This looks like a GHC bug. I will see if something can be done about the orphan instances without breaking changes.

[mightybyte]

The strange thing about this is that it is possible to fix the problem at all three points in the chain: me, groundhog, and GHC. Me by importing the instance I need. You by not exporting orphan instances. And GHC by not caching instances. So there's no shortage of opportunity to blame other parties here. I think fixes should be pursued on all fronts.