Fix support for newer stackage snapshots

[anka-213]

Among others, support for Cabal 3.0 and the MonadFail proposal.

Fixes #47.

This is a work in progress, because I was unable to make it backwards-compatible with old versions, since alex doesn't seem to support CPP pragma.

[anka-213]

Does anyone understand why it fails to build the debian package?

[odanoburu]

@anka-213 my guess from reading these lines

make[3]: Entering directory '/home/runner/work/gf-core/gf-rgl'
runghc Setup.hs build
Building [Prelude]
Setup.hs: gf: rawSystem: runInteractiveProcess: exec: does not exist (No such file or directory)

is that the executable gf is not in the PATH; you could modify the call to Setup.hs so that you specify where the gf executable is.

are you able to run this build locally? (using a container)

[anka-213]

How would I do that?

[odanoburu]

@anka-213 sorry, I didn't look at your PR, just at the logs, and I assumed they had to do with the package build. as they haven't (as far as I can see), I don't see why the build would break (assuming it works in your local machine). you could

1. trigger a re-build and see if it fixes itself (one can hope!), or
2. you could try to debug locally

for 1 you can push an empty commit or someone in the GF org can probably press so button in the github actions interface (I know how to do it with travis ci, not exactly with github actions)

for 2 you can install docker or podman or use a virtual machine to emulate the ubuntu 18.04 environment where the build happens, and run the steps steps on the build file

maybe @johnjcamilleri knows what's wrong exactly!

[johnjcamilleri]

Hi, sorry for being a bit slow to react. Yeah I agree with the above, don't quite understand why this commit should break the build. I've resubmitted to GitHub actions but I assume it will fail again in the same way. Will need to debug locally when I get a chance.

[johnjcamilleri]

It seems the error is coming from the RGL build script: https://github.com/GrammaticalFramework/gf-rgl/blob/master/Setup.hs
I guess this is some issue with Haskell versions (sorry can't look deeper right now).

[anka-213]

I looked closer at the log and realised that the problem was exactly the lack of backwards compatibility that I mentioned above. Let's see how it goes now that I have (hopefully) restored backwards compatibility.

[anka-213]

I added sudo apt update to the "Build Debian Package" github action, because it failed with a 404 without it.

[johnjcamilleri]

Looks good to me! Thanks for your contribution Andreas. Two comments:

• Can you add a stack file for GHC 8.8.4 / Stack LTS-16.13? (stack-ghc8.8.4.yaml)
• When I try build with stack-ghc7.10.3.yaml I get errors about alex/happy. This is not a bug you've introduced, but I wonder if you know how to fix it? It would be nice to guarantee that GF can still be built with GHC < 8.

[anka-213]

I was unable to reproduce the alex/happy trouble. I did have some earlier caching problems with alex though, so maybe removing .stack-work/dist/dist-newstyle might help.

On the other hand, I did still have a few errors on that version, caused by a missing MonadFail instance for StateT. I'll try adding the orphan instance via transformers-compat and see if that helps.

[johnjcamilleri]

Hmm, strange. What I get boils down to:

⋮
alex > Building alex-3.1.7...
alex > Preprocessing executable 'alex' for alex-3.1.7...
alex > setup: The program 'happy' is required but it could not be found
⋮
happy> build       
happy> Building happy-1.19.5...
happy> Preprocessing executable 'happy' for happy-1.19.5...
happy> setup: The program 'happy' is required but it could not be found

What version of Stack are you using? Maybe you have happy installed globally?

[anka-213]

Ah, right. Yes, I did have them installed globally. Now the dependencies are cached in my global stack directory. I guess I'll have to remove it to be able to reproduce.

[anka-213]

@johnjcamilleri I think this is the issue: commercialhaskell/stack#157 I'm not completely sure what we can do about it from inside gf. Maybe override the version of some dependency?

[anka-213]

@johnjcamilleri It should be fixed now. Tell me if there are any more problems.

[johnjcamilleri]

Unfortunately I am still getting some errors when trying to build with the different GHC versions.
(Side note: we should have a GitHub action with builds with all the different Stack files)

7.10.3
OK

8.0.2
OK

8.2.2
OK

8.4.4
OK

8.6.5

[ 39 of 166] Compiling GF.Grammar.CanonicalJSON
                
/Users/john/repositories/GF/anka-gf-core/src/compiler/GF/Grammar/CanonicalJSON.hs:129:19: error:
    • No instance for (Control.Monad.Fail.MonadFail Result)
        arising from a do statement
        with the failable pattern ‘"Str"’
    • In a stmt of a 'do' block: "Str" <- readJSON o
      In the first argument of ‘(<|>)’, namely
        ‘do "Str" <- readJSON o
            return StrType’
      In the first argument of ‘(<|>)’, namely
        ‘do "Str" <- readJSON o
            return StrType
           <|>  
             do "Float" <- readJSON o
                return FloatType’
    |           
129 |   readJSON o = do "Str"   <- readJSON o; return StrType
    |                   ^^^^^^^^^^^^^^^^^^^^^
                
/Users/john/repositories/GF/anka-gf-core/src/compiler/GF/Grammar/CanonicalJSON.hs:189:19: error:
    • No instance for (Control.Monad.Fail.MonadFail Result)
        arising from a do statement
        with the failable pattern ‘"_"’
    • In a stmt of a 'do' block: "_" <- readJSON o
      In the first argument of ‘(<|>)’, namely
        ‘do "_" <- readJSON o
            return WildPattern’
      In the first argument of ‘(<|>)’, namely
        ‘do "_" <- readJSON o
            return WildPattern
           <|>  
             do p <- readJSON o
                return (ParamPattern (Param p []))’
    |           
189 |   readJSON o = do "_" <- readJSON o; return WildPattern
    |                   ^^^^^^^^^^^^^^^^^
                
/Users/john/repositories/GF/anka-gf-core/src/compiler/GF/Grammar/CanonicalJSON.hs:240:19: error:
    • No instance for (Control.Monad.Fail.MonadFail Result)
        arising from a do statement
        with the failable pattern ‘"_"’
    • In a stmt of a 'do' block: "_" <- readJSON o
      In the first argument of ‘(<|>)’, namely
        ‘do "_" <- readJSON o
            return Anonymous’
      In the expression:
        do "_" <- readJSON o
           return Anonymous
          <|> VarId <$> readJSON o
    |           
240 |   readJSON o = do "_" <- readJSON o; return Anonymous
    |                   ^^^^^^^^^^^^^^^^^
                
Completed 2 action(s).

--  While building package gf-3.10.4 using:
      /Users/john/repositories/GF/anka-gf-core/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/setup/setup --builddir=.stack-work/dist/x86_64-osx/Cabal-2.4.0.1 build lib:gf exe:gf --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

8.8.3/4

[149 of 166] Compiling GF.Command.Interpreter

/Users/john/repositories/GF/anka-gf-core/src/compiler/GF/Command/Interpreter.hs:32:26: error:
    • Could not deduce (MonadFail m)
        arising from a use of ‘interpretPipe’
      from the context: (Output m, TypeCheckArg m)
        bound by the type signature for:
                   interpretCommandLine :: forall (m :: * -> *).
                                           (Output m, TypeCheckArg m) =>
                                           CommandEnv m -> String -> m ()
        at src/compiler/GF/Command/Interpreter.hs:28:1-84
      Possible fix:
        add (MonadFail m) to the context of
          the type signature for:
            interpretCommandLine :: forall (m :: * -> *).
                                    (Output m, TypeCheckArg m) =>
                                    CommandEnv m -> String -> m ()
    • In the first argument of ‘mapM_’, namely ‘(interpretPipe env)’
      In the expression: mapM_ (interpretPipe env) pipes
      In a case alternative:
          Just pipes -> mapM_ (interpretPipe env) pipes
   |
32 |     Just pipes -> mapM_ (interpretPipe env) pipes
   |                          ^^^^^^^^^^^^^^^^^


--  While building package gf-3.10.4 using:
      /Users/john/repositories/GF/anka-gf-core/.stack-work/dist/x86_64-osx/Cabal-3.0.1.0/setup/setup --builddir=.stack-work/dist/x86_64-osx/Cabal-3.0.1.0 build lib:gf exe:gf --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

[johnjcamilleri]

Also I think we don't need Stack files for both 8.8.3 and 8.8.4, just the latest per major version number (A.B) should suffice.

[anka-213]

I think I've fixed most of those issues now. I'm looking into adding github actions for all stack files.

Btw, should we update the default stack file to use the newest version, or is there a reason to pin it to an old version of GHC?

[johnjcamilleri]

is there a reason to pin it to an old version of GHC?

No particular reason, other than GHC updates are frequently breaking and chasing the latest version means you constantly have to update your code (as you know all too well). I don't know, is there some consensus on what version of GHC is a a good stable choice atm?

[anka-213]

Now I've added Github actions that tries to build it for all the versions with a stack file. It seems like it was successful.

It doesn't try to build with the c-runtime currently, but that could be a later issue.

On another note, should we update all the snapshots to the latest version with that ghc version, according to the list here: https://www.stackage.org/?

[johnjcamilleri]

Now I've added Github actions that tries to build it for all the versions with a stack file. It seems like it was successful.

Excellent!

On another note, should we update all the snapshots to the latest version with that ghc version, according to the list here: https://www.stackage.org/?

I don't know what you mean, we already have the latest minor version per major version? e.g. 8.2.2 is the latest 8.2 version, etc.

[anka-213]

I was specifically thinking of the one for ghc-8.6.5, which was lts-14.3 instead of lts-14.27. I've bumped it now.

[johnjcamilleri]

Great. Are we ready to merge then?

[anka-213]

The haskell-language-server only has support for ghc-8.6.5 and up, so I bumped the default stack file to that version. If that is fine, we're ready to merge. If not, I can revert that change and then merge.

[johnjcamilleri]

Unfortunately something is horribly wrong. The latest version of the GF shell doesn't produce any output:

$ gf TAEng.gf
                              
         *  *  *              
      *           *           
    *               *         
   *                          
   *                          
   *        * * * * * *       
   *        *         *       
    *       * * * *  *        
      *     *      *          
         *  *  *              
                              
This is GF version 3.10.4. 
Built on darwin/x86_64 with ghc-8.2, flags: interrupt server
License: see help -license.   

linking ... OK

Languages: TAEng
TA> gr
0 msec

If I git checkout bca0691 (before your changes), then the exact procedure above correctly produces a random tree. This applies to linearisation too, and I guess all shell commands. Like they're being ignored somehow (just a hunch).

[anka-213]

Oh, no! I'll look into it. I might have made a mistake with the conversion to Monad fail somewhere.

[johnjcamilleri]

Could this be the culprit:

[anka-213]

Oh, oops! Yes, that's probably the problem. I was debugging an issue and apparently forgot to restore the original code!