Build GHC from source (#4567)

ChangeLog.md

@@ -55,6 +55,11 @@ Major changes:
   builds, this functionality is no longer useful. For an example, please see
   [Building Haskell Apps with
   Docker](https://www.fpcomplete.com/blog/2017/12/building-haskell-apps-with-docker).
+* Support building GHC from source (experimental)
+    * Stack now supports building and installing GHC from source. The built GHC
+      is uniquely identified by a commit id and an Hadrian "flavour" (Hadrian is
+      the newer GHC build system), hence `compiler` can be set to use a GHC
+      built from source with `ghc-git-COMMIT-FLAVOUR`
 
 Behavior changes:
 * `stack.yaml` now supports `snapshot`: a synonym for `resolver`. See [#4256](https://github.com/commercialhaskell/stack/issues/4256)

doc/yaml_configuration.md

@@ -450,6 +450,80 @@ compiler: ghcjs-0.1.0.20150924_ghc-7.10.2
 compiler-check: match-exact
 ```
 
+#### Building GHC from source (experimental)
+
+(Since 2.0)
+
+Stack supports building the GHC compiler from source. The version to build and
+to use is defined by a a Git commit ID and an Hadrian "flavour" (Hadrian is the
+build system of GHC) with the following syntax:
+
+```yaml
+compiler: ghc-git-COMMIT-FLAVOUR
+```
+
+In the following example the commit ID is "5be7ad..." and the flavour is
+"quick":
+
+```yaml
+compiler: ghc-git-5be7ad7861c8d39f60b7101fd8d8e816ff50353a-quick
+```
+
+By default the code is retrieved from the main GHC repository. If you want to
+select another repository, set the "compiler-repository" option:
+
+```yaml
+compiler-repository: git://my/ghc/repository
+# default
+# compiler-repository: https://gitlab.haskell.org/ghc/ghc.git
+```
+
+Note that Stack doesn't check the compiler version when it uses a compiler built
+from source. Moreover it is assumed that the built compiler is recent enough as
+Stack doesn't enable any known workaround to make older compilers work.
+
+Building the compiler can take a very long time (more than one hour). Hint: for
+faster build times, use Hadrian flavours that disable documentation generation.
+
+#### Global packages
+
+The GHC compiler you build from sources may depend on unreleased versions of
+some global packages (e.g. Cabal). It may be an issue if a package you try to
+build with this compiler depends on such global packages because Stack may not
+be able to find versions of those packages (on Hackage, etc.) that are
+compatible with the compiler.
+
+The easiest way to deal with this issue is to drop the offending packages as
+follows. Instead of using the packages specified in the resolver, the global
+packages bundled with GHC will be used.
+
+```yaml
+drop-packages:
+- Cabal
+- ...
+```
+
+Another way to deal with this issue is to add the relevant packages as
+`extra-deps` built from source. To avoid mismatching versions, you can use
+exactly the same commit id you used to build GHC as follows:
+
+```
+extra-deps:
+- git: https://gitlab.haskell.org/ghc/ghc.git
+  commit: 5be7ad7861c8d39f60b7101fd8d8e816ff50353a
+  subdirs:
+    - libraries/Cabal/Cabal
+    - libraries/...
+```
+
+#### Bootstrapping compiler
+
+Building GHC from source requires a working GHC (known as the bootstrap
+compiler). As we use a Stack based version of Hadrian (`hadrian/build.stack.sh` in
+GHC sources), the bootstrap compiler is configured into `hadrian/stack.yaml` and
+fully managed by Stack.
+
+
 ### ghc-options
 
 (Since 0.1.4)

src/Stack/Config.hs

@@ -65,6 +65,7 @@ import           Stack.Build.Haddock (shouldHaddockDeps)
 import           Stack.Storage (initStorage)
 import           Stack.SourceMap
 import           Stack.Types.Build
+import           Stack.Types.Compiler
 import           Stack.Types.Config
 import           Stack.Types.Docker
 import           Stack.Types.Nix
@@ -207,6 +208,9 @@ configFromConfigMonoid
          configHideTHLoading = fromFirstTrue configMonoidHideTHLoading
 
          configGHCVariant = getFirst configMonoidGHCVariant
+         configCompilerRepository = fromFirst
+            defaultCompilerRepository
+            configMonoidCompilerRepository
          configGHCBuild = getFirst configMonoidGHCBuild
          configInstallGHC = fromFirstTrue configMonoidInstallGHC
          configSkipGHCCheck = fromFirstFalse configMonoidSkipGHCCheck

src/Stack/Config/Nix.hs

@@ -75,6 +75,7 @@ nixCompiler compilerVersion =
             _ -> "haskell.compiler.ghc" <> T.concat (x : y : minor)
         _ -> Left $ stringException "GHC major version not specified"
     WCGhcjs{} -> Left $ stringException "Only GHC is supported by stack --nix"
+    WCGhcGit{} -> Left $ stringException "Only GHC is supported by stack --nix"
 
 -- Exceptions thown specifically by Stack.Nix
 data StackNixException

src/Stack/Script.hs

@@ -254,6 +254,8 @@ allExposedModules gpd = do
       checkCond (PD.Impl compiler range) = case curCompiler of
         ACGhc version ->
           pure $ compiler == GHC && version `withinRange` range
+        ACGhcGit {} ->
+          pure $ compiler == GHC
         ACGhcjs version _ghcVersion ->
           pure $ compiler == GHCJS && version `withinRange` range
       -- currently we don't do flag checking here