Allow --pvp-bounds to use multiple snapshots

[mgsloan]

It recently occurred to me that we could have --pvp-bounds use broader constraints, by having it take into account multiple stackage snapshots. The simplest cut at this would be a funky field like other-resolvers: [lts-2.0, lts-3.19]. This would broaden the constraints placed on dependencies, to include the versions in both snapshots.

Ideally, this feature would be implemented such that these configurations actually build and pass tests. I think this can be done by having sdist and upload support building and testing the tarballs against all configurations (#717).

Since you might need a different set of extra-deps for a given resolver, we really need to have multiple project configurations. I can think of two ways to do this:

• Support multiple configurations in one file, as requested in Feature request : multiple build configurations #392. In Using environment variables in stack.yaml #1375 , avoiding complications of the stack.yaml format is discussed.
• Use multiple stack.yaml files when running sdist. I don't like this because it'd be easy to forget to specify the additional files, and we can't just slurp in all the *.yaml files.

With configuration inheritance, I bet just overriding the resolver would work for many people's usages. So, I'm beginning to lean towards doing something like that. I don't think it'd be all that complicated (it's already a monoid, we're already using inheritance for the non-project options), and it's useful for other reasons.

I think this is a very principled way of setting your version constraint bounds: pick the extremes of the snapshots that you want to support, and perhaps even a few in the middle. Then, actually test that these snapshots work. This does not guarantee that all the versions in the middle will work, it's likely that they will. For the exceptional cases where this doesn't hold, the user can put fancier version constraints into the cabal files, which won't get overridden.

Side note: Another thing to consider is whether this should support inplace modification of cabal files. I think it's weird that we can end up in a situation where the repo's cabal file doesn't have constraints, whereas the hackage package does. However, implementing this shouldn't be blocked on that - refactoring cabal files seems tricky.

[borsboom]

Side note: Another thing to consider is whether this should support inplace modification of cabal files. I think it's weird that we can end up in a situation where the repo's cabal file doesn't have constraints, whereas the hackage package does. However, implementing this shouldn't be blocked on that - refactoring cabal files seems tricky.

Another problem with this is that the .cabal file's bounds are an input to --pvp-bounds (if the .cabal file specifies a bound, it will be passed through unchanged), so making it an output would basically mean that the bounds will never change again even if you change the resolver.

[mgsloan]

Another problem with this is that the .cabal file's bounds are an input to --pvp-bounds (if the .cabal file specifies a bound, it will be passed through unchanged), so making it an output would basically mean that the bounds will never change again even if you change the resolver.

Yes, this is tricky. A few possible ways to get around this:

1. Have a list of manual version bound overrides in the stack.yaml. Like ghc-options and flags, these could be specified for all packages ("*"), or specific packages. So, this way, all version bounds in cabal files would be specified by the stack.yaml.

2. Have a section of the build-depends which is marked by a comment indicating that they are user-specified. This doesn't seem so good, because it would require a parser which actually preserves this info.

3. Do something strange with the automatically generated bounds, such as adding an unnecessary .0 suffix to the version numbers involved in the bound. This seems too magical to me, but I think it'd work..

[mgsloan]

I think the first step towards making this happen is to open an issue which specifies how multiple configurations work, and how configuration inheritance works. This task can be considered entirely separately to this stuff about determining version constraints.

Despite earlier qualms with making stack.yaml more complicated, between this issue, #846 (comment) , and the general commonness of having multiple configurations, I'm thinking it's worth it!

We just need to define the following:

• The Monoid instance for project configurations
• How to list different configuration variants.
• How to specify that a given variant is the default. (could be as simple as naming it default)

[sol]

This issue was one of the motivations to add support for hpack. In case somebody who is working on this needs modifications to hpack, please don't be shy to ask or open PRs.

[mgsloan]

I've just discussed something related to this in #haskell-stack . @chreekat and @bitemyapp have convinced me that we probably shouldn't extend the stack.yaml format with such fanciness.

So, instead I think the functionality described here should be implemented in a tool external to stack. Here's how that might look:

1. Run generate-pvp-bounds my-pkg config1.yaml config2.yaml config3.yaml.
2. It would then run stack --stack-yaml config1.yaml list-dependencies etc
3. Parse the listed versions and find constraint intervals that contain them all.
4. Output cabal file with updated constraints.

I'd be keen on helping with such a tool if someone wants to implement it.

[sjakobi]

I've just discussed something related to this in #haskell-stack . @chreekat and @bitemyapp have convinced me that we probably shouldn't extend the stack.yaml format with such fanciness.

So, instead I think the functionality described here should be implemented in a tool external to stack. Here's how that might look:

While I agree that it's good to keep stack.yamls simple, I don't see why that means that the bounds generation needs be done by an external tool.

How about specifying the extra stack.yamls to consider with an optional flag to --pvp-bounds, e.g. --also-considering?
Example command: stack sdist --pvp-bounds both --also-considering stack-7.8.yaml stack-8.0.yaml.

[mgsloan]

@sjakobi That's a fair point. Considering that --pvp-bounds already exists, that would be a very reasonable addition to it. I've opened #2262 to track it