Cabal confused with sublibrary when same-named remote package exists

[merijn]

Describe the bug
According to the documentation, the correct way to refer to the (unnamed) main library of a package foo is foo:foo, however specifying bounds on foo:foo when a local library named foo exists, gets interpreted as a bound on the local library. As a result it's impossible to have a package with local library foo:conduit that depends on conduit or conduit:conduit, as any bounds get interpreted as bounds on foo:conduit.

To Reproduce
Steps to reproduce the behavior:

Create a package with a local library whose name matches an existing package, and try to add a bound on the existing package.

$ cabal build ...

Expected behavior
Any bounds on conduit:conduit apply to the conduit package/library, not the local library named conduit.

System information

• cabal version 3.10

Additional context
Add any other context about the problem here.

[phadej]

@merijn which cabal-version you are using. How foo:foo is treated has changed in cabal-version: 3.4 exactly to avoid name shadowing issues.

From https://cabal.readthedocs.io/en/stable/file-format-changelog.html#cabal-version-3-4

Dependencies to sublibraries must be specified explicitly, even for current package. For example: build-depends: mypackage:internal-lib This way you can have an internal library with the same name as some external dependency.

Please provide a reproducer.

[merijn]

Originally this was cabal-version: 3.0 with cabal-install 3.10. I was just trying cabal-version: 3.4, but the linked change requiring that internal packages are fully qualified does not seem to properly work either, i.e. both using conduit or conduit:conduit as dependency still gets interpreted as a dependency on the internal library.

This is rather "in progress" code, so I will try to shrink it to a reproducer.

[merijn]

I created a reproducer here: https://github.com/merijn/cabal-repro

You can try as-is, or trying changing the conduit dependency to conduit:conduit, either will result in the same error:

Error: cabal: The package has an impossible version range for a dependency on
an internal library: conduit ^>=1.3.5. This version range does not include the
current package, and must be removed as the current package's library will
always be used.

If I understand the docs linked by phadej, only repro:conduit should ever refer to the local component in 3.4 and later.

[merijn]

This seems to imply that the 3.4 change never actually worked as intended, behaviour is identical to cabal-version 3.0, only the error has slightly changed.

[Kleidukos]

I can reproduce this on the 3.6 provided by ghcup and 3.11

[phadej]

I cannot reproduce.

[polinukli] /codetmp/cabal-repro master % cabal --version
cabal-install version 3.6.2.0
compiled using version 3.6.2.0 of the Cabal library 
[polinukli] /codetmp/cabal-repro master % cabal build --dry-run
Build profile: -w ghc-8.6.5 -O1
In order, the following would be built (use -v for more details):
 - conduit-1.3.5 (lib) (requires build)
 - repro-0.1.0 (lib) (first run)
 - repro-0.1.0 (lib:conduit) (first run)

cabal-plan prints

% cabal-plan
using '/codetmp/cabal-repro' as project root

Tree
~~~~

repro-0.1.0
 └─ base-4.12.0.0
     ├─ ghc-prim-0.5.3
     │   └─ rts-1.0
     ├─ integer-gmp-1.0.2.0
     │   └─ ghc-prim-0.5.3 ┄┄
     └─ rts-1.0 ┄┄
repro-0.1.0
 [repro-0.1.0:lib:"conduit"]
 ├─ base-4.12.0.0 ┄┄
 └─ conduit-1.3.5

as expected.

Cannot reproduce with cabal-install-3.10.1.0 either:

[polinukli] /codetmp/cabal-repro master % ~/.ghcup/bin/cabal-3.10.1.0 --version      
cabal-install version 3.10.1.0
compiled using version 3.10.1.0 of the Cabal library 
[polinukli] /codetmp/cabal-repro master % ~/.ghcup/bin/cabal-3.10.1.0 build --dry-run
Build profile: -w ghc-8.6.5 -O1
In order, the following would be built (use -v for more details):
 - conduit-1.3.5 (lib) (requires build)
 - repro-0.1.0 (lib) (first run)
 - repro-0.1.0 (lib:conduit) (first run)

[phadej]

I see. When you actually try to build the package, something in Cabal building code fails. I.e. the important bit is

cabal: Failed to build repro-0.1.0. The failure occurred during the configure
step.

cabal-install's solver and Cabal-syntax parser work correctly. It's latter stages which seem to fail, i.e. actually executing the build plan.

---

EDIT: compare with cabal-version: 3.0 behavior:

% cabal build --dry-run 
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] next goal: repro (user goal)
[__0] rejecting: repro-0.1.0 (conflict: repro==0.1.0, repro => repro (lib
conduit)^>=1.3.5)
[__0] fail (backjumping, conflict set: repro)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: repro

There the solving fails as name shadowing occurs.

[merijn]

This probably means the next cabal-install should include a bumped cabal-version, else there is no way to exclude older versions from attempting to build packages that rely on this behaviour, leading to errors for users.

[phadej]

else there is no way to exclude older versions from attempting to build packages that rely on this behaviour, leading to errors for users.

No need to complicate things. Spec didn't change.

If people use newer cabal-install (which bundles newer Cabal library) the stuff will just work. So if user of old cabal install runs into this bug, they should upgrade cabal-install (e.g. to a minor release with a bugfix). If there are build-type: Custom packages involved (so bundled Cabal is not used), they can specify constraints or move away from using build-type: Custom (as using build-type: Custom is asking for troubles)

[merijn]

I mean, the documented/specced behavirour of hasn't changed since cabal-version 3.4, but it's also never actually worked With Cabal, but the user has no indication that this is an issue with Cabal, and they should upgrade. Instead they just get a failing build and assume the package is broken.

I dunno what versions of cabal-install are in use "in the wild", but I'm gonna assume that "the next 3.10 minor release" will be a vast, vast minority. Leading to packages cryptically breaking on the vast majority of cabal-install's out there...

I can already hear the sound of tens of issues reporting build being broken...

[fgaz]

@merijn are you suggesting that the fix shouldn't be backported? Distros, even stable ones, will update to patch releases, so I don't think there will be any major divide. The fix is also mentioned in the changelog, so it's not like it isn't documented

[merijn]

@fgaz I'm not saying it shouldn't be backported (it wouldn't break anything that works currently).

I was more saying that if I want to publish something to hackage that relies on this behaviour, there is no way to prevent older (broken) Cabal's from attempting to build it. Like, if I finish updating my packages to use multiple public libraries, there is no way for me to indicate that the build is broken on cabal 3.4 through to some versions if 3.10, because the latest cabal-version is 3.8.

If the next cabal release (3.12?) includes a cabal-version bump, then that means I could use that cabal-version in my package to avoid people attempting to build it with broken versions of cabal-install. Because it seems inevitable that otherwise anyone relying on the specified behaviour will get a bunch of issues of "hey, this package is broken" from people with out of date cabal-install.

[phadej]

@merijn, there is no way to prevent someone building broken stuff with old cabal-install. Old spec versions cannot be changed, old cabal-install interpreter them in the best way they know.

You can try to convince Hackage maintainers to make a change so Hackage would reject packages with internal libraries and cabal-version between 3.4 and next one. I.e. prevent distribution.

I don't know if there is next cabal-version coming up with some other features. I would not make an empty Cabal spec version just to have a "bug fix" marker.

if I finish updating my packages to use multiple public libraries, there is no way for me to indicate that the build is broken on cabal 3.4 through to some versions if 3.10,

That's life. Sometimes software is buggy. cabal-version tells the version of the spec, not the needed Cabal or cabal-install version to work with them. (They are related for convenience, different choice from e.g. GHC2021 language implemented first in GHC version I cannot remember without looking up).

EDIT addition: With Cabal-syntax and Cabal split, syntactical features are all contained in Cabal-syntax. The name shadowing change was purely syntactical. (I guess the previous Cabals are also buggy, they just cannot observe the bug).
Having cabal-version to indicate the lower bound of spec, Cabal-syntax, Cabal (and cabal-install or worse also stack) is just not scalable.

It's almost viable (but plenty of work) to build .cabal packages without using Cabal library, just Cabal-syntax. (Some cabal-version guarded stuff -- like how cabal_macros.h is generated -- are unfortunately in Cabal. I think these bits should be moved, but I have no word in how Cabal is developed anymore). You parse the package file and figure out how to call GHC to make/build a package. Someone somewhere might do that: hadrian for example.