cabal can not find pkg-config even though the right directory has been added to the extra-prog-path

[phadej]

This happens on windows, where pkg-config is in mingw path, but not in global PATH. And there are good reasons to not add mingw-path to global $PATH (hint: it will break everything else on the windows system).

I experienced this myself trying to use pkgconfig on Windows. And apparently others as well https://discourse.haskell.org/t/installing-a-library-with-c-dependencies-on-windows/8557/4

[gbaz]

which version is this on? 3.10? head?

[phadej]

3.10

[jasagredo]

Msys2 at the top, Powershell at the bottom. My pkg-config is not on the global PATH and everything works as expected.

More information is needed on this issue to reproduce it.

[phadej]

@jasagredo it may been fixed in 3.12, as #9519 was fixed.

Did you try with cabal-install-3.10 (and I probably had 3.10.1.0)

[jasagredo]

Yeah, I did some work on the extra-prog-path (breaking other stuff in the way 😁). I'm pretty sure it has been fixed for 3.12? I think there is no support for previous releases.

Do you think we can close the issue?

[jasagredo]

Seems it was released in 3.10.3.0: https://github.com/haskell/cabal/blob/master/release-notes/Cabal-3.10.3.0.md

[phadej]

Do you think we can close the issue?

I don't care. I solved my original problem by not relying on pkg-config at all (it's a PITA on windows anyway, even if cabal finds it). So I cannot verify whether the 3.10.3.0 or 3.12.1.0 solve the original problem, as there aren't one anymore.

[jasagredo]

I am pretty happy with pkg-config on Windows. I haven't had any problems so far.

Anyways, I'm then closing the issue as it seems the original problem was fixed already.

[phadej]

I am pretty happy with pkg-config on Windows.

If you know how to get e.g. windows libpq (Postgres) to have a .pc file (e.g. on GHA environment), I'll be all ears. In fact, it looks like there are no pkg-config entry even on macOS.

Using e.g. pacman to install postgresql lib is an option, but I don't think that's what most people would do.

[jasagredo]

I do use pacman always to install my stuff.

➜ pacman -S mingw-w64-clang-x86_64-postgresql
...

➜ cat /clang64/lib/pkgconfig/libpq.pc
prefix=/clang64
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libpq
Description: PostgreSQL libpq library
URL: https://www.postgresql.org/
Version: 16.2
Requires:
Requires.private: libssl, libcrypto
Cflags: -I${includedir}
Libs: -L${libdir} -lpq
Libs.private:  -lpgcommon -lpgport -lintl -lssl -lm -lwldap32 -lshell32 -lws2_32 -lsecur32

➜ pkg-config --libs libpq
-lpq

[phadej]

@jasagredo as said, that's not an option. The GHA (and I assume many people) don't do that. They would use https://www.postgresql.org/download/windows/

[jasagredo]

Not sure I agree. GHA has msys2, can't you install the pacman library there?

[phadej]

This is drifting off-topic.

[yutotakano]

I think this is the most relevant issue for my current problem, and I'm not sure if this is a bug in Cabal. I'm running the recommended Cabal version (3.12.1.0) installed through GHCup. I have an issue where Cabal can't find pkg-config, even though cabal exec is able to find it.
 
GHCup has automatically added the extra-prog-path as follows (which I think are fine):

❯ cabal user-config diff
+ G: \Programs\ghcup\msys64\usr\bin
+ extra-include-dirs: G:\Programs\ghcup\msys64\mingw64\include
+ extra-lib-dirs: G:\Programs\ghcup\msys64\mingw64\lib
+ extra-prog-path: G:\Programs\ghcup\bin,

I've installed pkg-config (and libopus, which is what I eventually want to link to) using the following commands:

❯ cabal exec -- pacman -S mingw64/mingw-w64-x86_64-pkg-config
❯ cabal exec -- pacman -S mingw64/mingw-w64-x86_64-opus

cabal exec is able to find pkg-config, thanks to it adding the mingw64/bin path to PATH before executing the command:

❯ cabal exec -v -- pkg-config --version
Including the following directories in PATH:
- G:\Programs\cabal\store\ghc-9.4.8\ansi-terminal-1.1.2-810a86b6f424ce3b08fd8f7f7909ec793e1d5f50\bin
- G:\Programs\cabal\store\ghc-9.4.8\hsc2hs-0.68.10-1f430dcbea282b42b4bba76a7098cb181eeb1f6d\bin
- G:\Programs\cabal\store\ghc-9.4.8\prettyprinter-1.7.1-18c19a3f47946841ff912285fd4bc0a157112ffc\bin
- G:\Programs\ghcup\bin
- G:\Programs\cabal\bin
- G:\Programs\ghcup\msys64\mingw64\bin
- G:\Programs\ghcup\msys64\usr\bin
creating G:\Projects\Programming\yutotakano\opus\dist-newstyle\tmp
Running: "G:\Programs\ghcup\msys64\mingw64\bin\pkg-config.exe" "--version"
0.29.2

However, if I include pkgconfig-depends: opus to my .cabal file, Cabal is suddenly unable to find pkg-config:

❯ cabal build -v
Running: "G:\Programs\ghcup\bin\ghc.exe" "--print-global-package-db"
Reading available packages of hackage.haskell.org...
Using most recent state specified from most recent cabal update
index-state(hackage.haskell.org) = 2025-02-06T08:45:58Z
Failed to query pkg-config, Cabal will continue without solving for pkg-config
constraints: Cannot find pkg-config program
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] next goal: opus (user goal)
[__0] rejecting: opus-0.2.1.0 (conflict: pkg-config package opus-any, not found in the pkg-config database)
[__0] fail (backjumping, conflict set: opus)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: opus (2)
CallStack (from HasCallStack):
  dieWithException, called at src\Distribution\Client\ProjectPlanning.hs:710:21 in cabal-install-3.12.1.0-inplace:Distribution.Client.ProjectPlanning

Does anybody know how I might resolve this, and what's going wrong internally with Cabal? I was under the impression that cabal exec uses the same PATH information as during a normal cabal build, so I'm surprised that one is able to find pkg-config while the other not.

If this issue looks like a Cabal bug, could the issue be reopened?

[jasagredo]

I don't have an immediate answer. Let's investigate.

The first thing to check is what is there in your global cabal config which seems it would be in G:\Programs\cabal\config.

Then I would make sure what MSYS2 environment you are running this shell in, what does echo $MSYSTEM return? Or are you running in PowerShell?

Next I'd try to see if you can get a version for the library calling pkg-config --modversion opus.

If all looks right, I would check the v3 output of cabal.

[jasagredo]

I was able to reproduce the behavior you are experiencing:

➜ cabal exec -- which pkg-config
/clang64/bin/pkg-config

➜ Get-Command pkg-config
Get-Command: The term 'pkg-config' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

This was in PowerShell. The issue is that the package db is populated before custom paths are added to the PATH.

[yutotakano]

I was able to reproduce the behavior you are experiencing:

Wow, thanks! I was going to reply to your first reply tomorrow (I'm preoccupied this weekend), and I will, but it's reassuring that you've also encountered the same issue this quickly.

My shell for reference is straight up normal CMD, not a mingw-bash or powershell shell. So the only way I interface with the MSYS2 environment is through cabal exec -- and I would've expected that building a package with Cabal would search that same environment.

[jasagredo]

To be precise, it is GHCup the one that installed your MSYS2, and the recommended way to interface with it is ghcup run --mingw-path -- <msys2 commands here>. It just so happens that cabal exec also loads up the extra paths added to the configuration, so it will kind of work. The difference is that ghcup run -m will not do extra work, while cabal exec will resolve the cabal configuration and all, so it can take a while on some cases.

I think the issue is legitimate and re-reading this thread I think @phadej's request at the top was also legitimate, it was just that I didn't understand it at that time. Cabal configures some of the programs without taking into account the extra-prog-paths in the config.

I also learned that pkg-config doesn't seem to be added right away to the program DB and instead we ask for it several times:

➜ cabal build -v3 | grep "Searching" | grep pkg-config
Searching for pkg-config in path.
Searching for pkg-config in path.
Searching for pkg-config in path.

[yutotakano]

Thank you for the detailed inspection! My repro from the Windows cmd shell looks exactly like yours, pretty much. I also appreciate your explanation of GHCup setting up MSYS2 (and extra-prog-path), I was slightly confused myself where I got it installed from.

Just to summarise my understanding (please correct me if I'm wrong, I'm thinking by typing):

• The MSYS2 environment/shell is able to find pkg-config without extra trouble since it prepends mingw64 PATHs by default (thus ghcup run -m can find pkg-config, and cabal exec -- works too when we don't require pkg-config in our cabal config)
• When we have a pkgconfig-depends clause, cabal build/exec -v3 starts looking for pkg-config before loading the PATH information from extra-prog-path directories. Theoretically that's fine if we were also calling cabal from within MSYS2 since (per above) the shell prepends mingw64 paths. But when calling from powershell/cmd, it uses pure Windows PATHs so is unable to find pkg-config until later (?)

So theoretically the solution seems to be to try the PATH addition at the very beginning of cabal's pipeline. I don't think doing so will have negative consequences on other OSes if we do this anyway later?

Unfortunately, I'm not entirely versed in the cabal codebase so I can't tell if this is difficult architecturally or not. Perhaps it being difficult to do is why cabal checks for pkg-config multiple times like you said (i.e., it may have been designed to modify PATH in between those calls, deliberately checking for pkg-config before and after for some reason).

[jasagredo]

Your understanding is right. cabal exec probably fails to find pkg-config too, but continues anyway to the point where it runs the executable given after the --. At that point the PATH includes the extra-prog-path additions from the config so it finds pkg-config.exe.

This should not be very difficult. It is a matter of passing down the data read from loadConfig which includes those extra paths to the places where we try to find the pkgConfigProgram.

[jasagredo]

In particular we should call prependProgramSearchPath

cabal/Cabal/src/Distribution/Simple/Program/Db.hs

Line 241 in 595d023

prependProgramSearchPath

when running this action

cabal/cabal-install-solver/src/Distribution/Solver/Types/PkgConfigDb.hs

Line 64 in 595d023

mpkgConfig <- needProgram verbosity pkgConfigProgram progdb

with the paths that should come from the configProgramPathExtra in savedConfigureFlags in SavedConfig returned by loadConfig

cabal/cabal-install/src/Distribution/Client/Config.hs

Line 950 in 595d023

loadConfig :: Verbosity -> Flag FilePath -> IO SavedConfig

[yutotakano]

I follow! I'm slightly confused though:

• cabal exec calls cabal-install.Distribution.Client.CmdExec.execAction
• At the very start, execAction calls cabal-install.Distribution.Client.ProjectOrchestration.runProjectPreBuildPhase
• which then immediately calls cabal-install.Distribution.Client.ProjectPlanning.rebuildInstallPlan
• which calls (in the same file) phaseConfigureCompiler that returns a progdb, and prodb is eventually used within phaseRunSolver to call getPkgConfigDbDirs and readPkgConfigDb both of which try to get pkg-config (and fails in our case)

But within phaseConfigureCompiler if we check how progdb is created, it is using prependProgramSearchPath already, thus it should have found pkg-config if I'm understanding the flow correctly!

[jasagredo]

@yutotakano You should be able to use cabal from #10790 (either compile it locally or download it from CI once it is built, it will appear at the bottom of this page, but I'd rather build it myself with cabal build exe:cabal --project-file cabal.release.project) and it should work. I was able to make use of pkg-config from PowerShell with that cabal.

But within phaseConfigureCompiler if we check how progdb is created, it is using prependProgramSearchPath already, thus it should have found pkg-config if I'm understanding the flow correctly!

It was adding the local paths, but not the ones in the global config. My PR adds those too

[yutotakano]

@jasagredo Thank you! I compiled it locally and verified that it is able to find pkg-config and thus opus as well! My cabal build succeeds :)

I was prepared to dive in myself but I was a bit worried, so I just want to say a big thank you for implementing the PR!