Stack on nixos fails when ghc not available in nixpkgs, e.g. ghc802

[duog]

Please follow the steps below for reporting a bug:

Make sure that you are using the latest release (currently stack-1.9.3).
See the upgrade instructions to upgrade.

Please use the following schema for your bug report:

General summary/comments (optional)

LTS-9.21 requires ghc-8.0.2 which is not available in current nixpkgs.

Steps to reproduce

nix-shell -p stack
stack unpack transformers
cd transformers
cat >stack.yaml <<< "resolver: 9.32"
stack build

Expected

It would be nice if Stack would fall back to installing ghc-8.0.2 as it does on other platforms. If that's too hard, an error message saying something like "resolver 9.21 requires ghc-8.0.2. your does not have a derivation for this version, probably because it's too old. Sorry."

Actual

error: attribute 'ghc802' missing, at (string):1:43
(use '--show-trace' to show detailed location information)

Stack version

$ stack --version
1.9.3 x86_64 hpack-0.31.1

Method of installation

nix-shell -p stack

[dbaynard]

@nh2 Can you take a look at this? I'm not sure what ought to happen.

[ketzacoatl]

@dbaynard I think the simple first step would be to catch the error and provide a more useful explanation to the user. As @duog pointed out, a simple message like the following would be really helpful:

resolver 9.21 requires ghc-8.0.2. your does not have a derivation for this version, probably because it's too old. Sorry.

Do we know where the relevant code is?

[nh2]

Perhaps it should be some form of wrapper around getCompilerVersion from

stack/src/Stack/Nix.hs

Line 64 in 217fa42

runShellAndExit mprojectRoot getCompilerVersion getCmdArgs = do

?

[mpickering]

IMO if stack insists on using this wrapper then it should pin the version of nixpkgs (for the GHC version) rather than using the system nixpkgs.

[maxkrieger]

Are there any workarounds/tips for this? It's difficult to configure IDE tools etc to use --no-nix, system-ghc, etc. I have the latest stack (1.9.3). I don't think the solutions here helped either: #4196.

EDIT: a workaround that fixed it for me temporarily is adding the following lines to ~/.stack/config.yaml, but that doesn't work for builds that require nix packages like zlib:

system-ghc: true
nix:
  enable: false

[qrilka]

@maxkrieger I suppose you may install older GHC from @mpickering 's repo - https://discourse.nixos.org/t/new-nur-repo-for-ghc-binaries/2660 and then you should be ablo to use nix mode

[maxkrieger]

@qrilka I don't think it helped, here's my attempt:

~/.stack/config.yaml:

system-ghc: true
nix:
        enable: true
        pure: true
        packages: [llvm, zlib, ncurses, ghc]
        add-gc-roots: true

~$ nix-env -e ghc-8.0.2-binary
~$ nix-env -iA nixpkgs.nur.repos.mpickering.ghc.ghc802
~$ cd myProject/
myProject/$ stack build
error: attribute 'ghc802' missing, at (string):1:65
(use '--show-trace' to show detailed location information)

Alternatively, through nix-shell:

myProject/$ nix-shell -p nur.repos.mpickering.ghc.ghc802 stack zlib
> ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.0.2
> stack build
error: attribute 'ghc802' missing, at (string):1:65
(use '--show-trace' to show detailed location information)

[qrilka]

@maxkrieger sorry for that incorrect assumption, I should have tried it first. The situation is a bit more complicated actually but it looks like there is a workaround: use a custom shell.nix file.
E.g. on my laptop I have NixOS 18.09 and it looks to contain the main latest GHC versions, so I tried not that widely used 8.4.1 and with the following shell.nix:

with (import <nixpkgs> {});
let ghc = nur.repos.mpickering.ghc.ghc841;
in haskell.lib.buildStackProject {
  inherit ghc;
  name = "myEnv";
  buildInputs = [ pcre ];
}

I get:

stack-1.9.3 --nix-shell-file=shell.nix exec -- ghc --version

and building also appears to work.
The main problem here is that you need to map properly GHC version in your stack.yaml and in shell.nix. Without a shell file that's done automatically but that method relies on GHC being available from nixpkgs.
BTW there is an alternative method - you could just add nur.repos.mpickering.ghc.ghc841 as nixpkgs.haskell.compiler.ghc841 in your configuration.nix. With that you won't need shell.nix

[maxkrieger]

@qrilka Thanks so much, your shell solution worked--couldn't quite deduce the syntax for "adding" the repo "as" the compiler package.

Also, what makes GHC unavailable from nixpkgs? I can install haskell.compiler.ghc802 just fine. Is it a naming mismatch?

[qrilka]

@maxkrieger the idea is to add missing attribute from Mathew's repo. Without knowing details of your setup I can't say for sure why it's missing. Example scenario is that you could use unstable channel or the recently released 19.0.3 which don't include this version, e.g. see https://github.com/NixOS/nixpkgs-channels/blob/nixos-unstable/pkgs/top-level/haskell-packages.nix
At the same time I don't quite understand how could you install haskell.compiler.ghc802 (as you're saying it above) if it's not supposed to be available

[domenkozar]

The error here should report what the user needs to do as it's pretty common to happen.

[ketzacoatl]

@domenkozar thank you for raising that point, I wholeheartedly agree!

@qrilka, I would be happy to help with the update to error messaging, but would probably need some guidance on what changes to make.

[qrilka]

@ketzacoatl I think what's needed here is adding some hints when error comes from nix-shell invocation at https://github.com/commercialhaskell/stack/blob/master/src/Stack/Nix.hs#L109
Probably error message "attribute 'ghcXXX' missing" could be checked against to give a more concrete advice.

[ketzacoatl]

@qrilka I see that uses exec from RIO.Process - https://www.stackage.org/haddock/lts-17.9/rio-0.1.20.0/RIO-Process.html#g:6 - but that says it does not return, so I'm a little confused: how would that code catch and process the error from nix-shell thru exec? Would we still use exec, or do we need to switch to another function? Thanks for the guidance!

[qrilka]

Sorry for a late reply, yes, I suppose we'll need to use something like readProcessStderr instead of exec.

[sullyj3]

Is this part from nix? I was very confused when I attempted to re-run stack with the --show-trace flag and it didn't exist.

error: attribute 'ghc8106' missing, at (string):1:43
(use '--show-trace' to show detailed location information)

It seems like defaulting to Nix mode is maybe not a good idea if a simple stack new test fails on NixOS stable? I'm not sure what causes it to use nix mode, does stack do this automatically when it detects it's running on NixOS? Or is this maybe a customization that NixOS has applied to the stack package? This behavior was very confusing to me.

I think the default should probably be act as usual with the ability to opt in to using Nix. At the very least until stack new isn't broken.

[naufik]

I'm having the same issue running stack new on NixOS. It seems to be the case that stack attempts to create a nix environment by default when it is installed from nixpkgs (stable 21.05 channel):

~ % nix-shell -p ghc stack cabal-install # all from stable

[nix-shell:~]$ stack new hello-world
error: attribute 'ghc8107' missing, at (string):1:43
(use '--show-trace' to show detailed location information)

As of the time of writing, the nixos-21.05 stable channel only has ghc-8.10.4. ghc-8.10.7 is only available under nixos-unstable. It's pretty strange that it's trying to fetch a package from the future.

This is my hunch on what I think happened under the hood:

• Calling stack new <name> attempts to create a nix-environment.
• Stack declares haskell.compiler.ghc8107 (this is an alias for ghc-8.10.7) as a dependency to nix - however it is unavailable unless the nix that stack is calling has the unstable channel/branch enabled.
• Nix is unavailable to evaluate haskell.compiler.ghc8107, and throws that error. Keep in mind that haskell.compiler is still a valid namespace/attribute and hence it only errors out on the ghc8107 bits.

The following workaround worked for me:

# ~/.stack/config.yaml
nix:
  enable: false

No need to use system-ghc because the problem lies mainly in how it fetches the GHC.

It seems like the core problem is that Stack is hardcoded to use whatever version of GHC Stack things should be available when running into nix mode, without necessarily checking whether haskell.compiler.ghcXXXX is available in the nixos-channels used.

Stack seems to enable nix-mode automatically when it detects NixOS.

stack/src/Stack/Config/Nix.hs

Lines 39 to 41 in 255cd83

	-- Enable Nix-mode by default on NixOS, unless Docker-mode was specified
	osIsNixOS <- isNixOS
	let nixEnable0 = fromFirst osIsNixOS nixMonoidEnable

While I think that the above can be appropriate. I agree with @sullyj3's suggestion that it is rather confusing to have this be default when stack new is unlikely to work.

The "proper" workaround to enable nix mode would be to create a channel in your local that contains whatever version the haskell.compiler.ghcXXXX that stack is asking.

A solution I can think of for this would be to allow a way to optionally version-lock the ghc version stack looks for during initial launch - this way, we can ensure that the ghc version requested exists in the same nixpkgs branch as stack when a stable release is done by nixpkgs. However, I've never used stack in nix-mode before so I'm quite unsure how it manages different ghc versions in this mode as it seems to have the caveat of only allowing the use of ghc versions available in nixpkgs.

[kephas]

As a note, it's also a problem with Nix outside NixOS. With this in ~/.stack/config.yaml:

nix:
  enable: true
  pure: true

I also had this error when trying to create a new project (or trying to build exisitng projects):

pierre:~$ stack new example
error: attribute 'ghc8104' missing, at (string):1:43
(use '--show-trace' to show detailed location information)

I had to disable nix in ~/.stack/config.yaml and now uses this default.nix with resolver: lts-18.8:

with import (fetchTarball https://github.com/NixOS/nixpkgs/archive/c4ed2a618c5f8b51c1be5983c9c2e662d76679ad.tar.gz) {};
let ghc = haskell.compiler.ghc8106;
in haskell.lib.buildStackProject {
    inherit ghc;
    name = "example";
    src = ./.;
}

[sjfloat]

None of these work arounds have worked for me. Is this still being investigated?

[hhefesto]

specifying nixpkgs with path worked for me:

nix:
  enable: true
  path:
    - "nixpkgs=https://github.com/NixOS/nixpkgs/archive/22.05.tar.gz"