Current and future status of project stack.yaml's

[jneira]

• To reflect some recent discussions in the irc channel and get more broad feedback

• Actually we have a stack.yaml for each supported ghc

• Keeping them up-to-date needs some effort, most times mechanical and boring but sometimes tricky

• The install script uses those stack.yaml's to define the list of supported/available ghcs to install hls, even if the script uses cabal underneath

  • But we are discussing deprecate and remove the install script

• Pros:

  • They make easier to possible contributors using stack start to work on hls, so we get more chances to get contributions.
  • They make easier for advanced users using stack build hls from source and we are suggesting that as workaround for some issues
    • The alternative to the install script, ghcup compile hls only allow using cabal for the build
  • It is much harder to create a new stack.yaml from the air for those all possible case uses than update an existing one. stack init does not work for hls.
  • More broadly, we send out a signal to the haskell community: hls aims to support stack in parity with cabal for users, maintainers and contributors. I firmly think that it helps to the friendly coexistence of both tools in the community.

• Contras

  • We have to maintain all those stack.yamls, which is a error prone and boring task most of times and a tricky one sometimes
  • We have to maintain the circleci setup and worry about it
  • Contributors not using stack can find difficulties in update those stack.yamls and it could make them stop working on hls (as commented by @fendor)

• Alternatives:

1. Remove all stack.yamls and require cabal or nix to work on hls (i think nobody has suggested that at the moment but for completeness)
2. Keep only one stack.yaml with the last lts or other (suggested by @wz1000)
3. Keep two stack.yamls: last lts and nightly (they could be the same ghc at some point though) (suggested by @fendor? not sure)
4. Keep one stack.yaml for supported ghc: it will remove the ghcs of minor versions (one or two) (suggested by @simonmichael iirc)
5. Keep the actual state of things

Keeping some of the stack.yamls makes pros and contras only vary quantitatively.

Personally i am for 4 or 5.

[pepeiborra]

I vote for either 1 or 2.

Some of the Pros are a bit thin:

• New contributors - it's not a huge ask for new contributors to use ghcup + cabal, so I'm not sure this counts as a Pro.
• Advanced users - by definition these users can build with any tool, so the absence of stack.yaml wouldn't matter to them, so this doesn't count as a Pro either.

The only valid Pro imho is "Sending a message". Perhaps we should ask the current regular contributors who cares about this message, otherwise it seems unfair to impose this overhead on them.

Also, I suspect that none of the current contributors use Stack. So the last Contra applies basically to all of us. not just some.

[jneira]

New contributors - it's not a huge ask for new contributors to use ghcup + cabal, so I'm not sure this counts as a Pro.

would you contribute to a project which asks you to use only stack?

stack is more used between beginners and intermmediate haskellers, its defaults and automanage of ghc and msys2 are more appealing for them.
But beginners/intermmediate haskellers usually are not maintainers of the projects only possible contributors. So they dont have neither voice nor vote in the decisions. We have to represent them in their absence.

I am using mostly cabal nowadays but i still remember when i started to program in haskell using stack (cabal has improved since that). You had to learn a language with a high learning curve and fight oddities of two build tools.

Even for more experienced developers having to learn how to use a new build tool can be hard, and i think it could push away from contribute to the project.

And contributions are being increasingly rare in the open source world so i think we have to carefully consider the balance of what we are asking for, and what we offer.

Otoh, it seems to me than maintaining stack.yaml's is no so hard. Most of times you forgot to update it when adding a dependency, circleci complains about and it is matter to add one or two lines in all files. Boring but easy. You dont have to worry about until the main work done is done (like hlint)
Deeper refactors and cleaning out of stack.yaml are more infrequent (and i did several of them :-P)
have we been seriously blocked due to outdated stack.yamls more than one day?

Advanced users - by definition these users can build with any tool, so the absence of stack.yaml wouldn't matter to them, so this doesn't count as a Pro either.

Dont see why an advanced user can't use and know about only one of both tools. Maybe the time saved from learn and fight bugs of one of them gave them more time to learn haskell :-P

Also, I suspect that none of the current contributors use Stack.

@Ailrun was using stack and also some of contributors. @simonmichael if i am not wrong and and some more that I don't remember

[pepeiborra]

would you contribute to a project which asks you to use only stack?

I definitely would. If I want to contribute to a project, the build system is no obstacle.

stack is more used between beginnersand intermmediate haskellers, its defaults and automanage of ghc and msys2 are more appealing for them. But beginners/intermmediate haskellers usually are not maintainers of the projects only possible contributors. So they dont have neither voice nor vote in the decisions. We have to represent them in their absence.

I'm not arguing against beginners, don't turn this around please. I am saying that asking them to use cabal to contribute to HLS is perfectly reasonable.

And contributions are being increasingly rare in the open source world so i think we have to carefully consider the balance of what we are asking for, and what we offer.

I don't understand this point, but let's avoid turning this into a flame war.

Otoh, it seems to me than maintaining stack.yaml's is no so hard. Most of times you forgot to update it when adding a dependency, circleci complains about and it is matter to add one or two lines in all files. Boring but easy. You dont have to worry about until the main work done is done (like hlint) Deeper refactors and cleaning out of stack.yaml are more infrequent (and i did several of them :-P) have we been seriously blocked due to outdated stack.yamls more than one day?

Completely disagree. It's time consuming, there's just no good way to do it, feels wrong to have to do this by hand when there is an automated solution, requires extra CI, and quite frankly I doubt that anyone uses them.

On that last point, I wonder if there is any way to obtain usage data.

[drsooch]

Just as input for sake of discussion, I use stack on a regular basis. I used stack when I contributed to HLS, so there are some of us out there. My reasoning for using stack is similar to what @jneira is saying, it took care of a lot of the grunt work, when I first started using Haskell, and I've never really bothered to swap full-time to cabal.

That being said I frequently swap between the two build systems for no particular reason. I think they are near identical in their user experience and forcing one option (in my opinion) shouldn't require extra effort from the developer.

Also to reiterate what @pepeiborra said, I don't think using a different build tool will ever turn away a developer. If HLS required one or the other I would willingly switch to that tool.

I also don't see a problem with maintaining at least one stack.yaml. Much less cognitive overhead, plus stack takes care of the GHC versioning it could make it easy for new contributors to jump to the maintained verison.

[Ailrun]

Also, I suspect that none of the current contributors use Stack.

@Ailrun was using stack and also some of contributors.

quite frankly I doubt that anyone uses them.

I'm still using stack, for both contribution and installation. Actually, I almost never use Cabal directly unless I'm forced to do so for some reasons. Surely, when the cons definitely outweigh the pros, and we decide to remove stack.yamls, I'm willing to use Cabal for HLS. I just prefer to use Stack if possible.

Advanced users - by definition these users can build with any tool, so the absence of stack.yaml wouldn't matter to them, so this doesn't count as a Pro either.

I agree with this to some degree, but as @jneira said, it makes the process much easier and gives more motivations to the advanced users to try some contribution. Though this is definitely not a strong argument, as you already criticized.

[Ailrun]

Among the main suggestions, 4 is maybe a good alternative, at least for me. The issues are often about the latest minor versions, and most of my projects are developed in those, so I found myself often install only the latest minor version for some major versions. If this is common for other maintainers/contributors, I think it is reasonable to remove other stack.yamls for previous minor versions.

[pepeiborra]

@Ailrun can you clarify why one single stack.yaml wouldn't be good enough for your workflow?

Also, would you still contribute to HLS if you were asked to migrate to cabal?

[Ailrun]

@Ailrun can you clarify why one single stack.yaml wouldn't be good enough for your workflow?

Package versions for each stack.yaml is fixed, so if there is only one stack.yaml, then, for example, it would be quite tedious to make it work with for GHC 8.6.5 and GHC 9.0.1 from that one stack.yaml (and I'll eventually create two stack.yaml at least locally, which will make the package lists quickly out-of-sync). If I'm the only one with Stack, then sure, it's maybe better to deal with it from my side, but @jneira mentioned at least one other contributor (@simonmichael ), and there might be some other (possibly infrequent) contributors.

Also, would you still contribute to HLS if you were asked to migrate to cabal?

But, yes, as I said, I just want to avoid the situation if possible, but even if it is unavoidable, I'm still willing to contribute to HLS.

[Ailrun]

Oh, while I wrote the above comment, I've come up with one another "alternative":

• Remove all but one stack.yaml that is "trivially" compatible (so no-to-minimal alternation is required to use it for, say, other GHC versions, preferably a fixed LTS version), and keep only the stack.yaml that requires some special treatments (such as the one for GHC 9.0.1)

[jneira]

I'm not arguing against beginners, don't turn this around please. I am saying that asking them to use cabal to contribute to HLS is perfectly reasonable.

Sorry i did not want to mean you personally were arguing against beginners (i tried to make my sentences abstract and impersonal but still). Only wanted to stress my point: stack is more used by beginners/intermmediate haskellers and making easier hack with stack on hls will make more probable their possible contributions/became advanced users.

I doubt that anyone uses them.

Also, would you still contribute to HLS if you were asked to migrate to cabal?

So there are maitainers using stack (maybe some others not participating here).
But i am worried about the potential new contributors/advanced users which will answer "no" to that question.
But we will never aware of that and that loss will never take in account.

[pepeiborra]

Removing all but one stack.yaml is the most voted option, I think we have a winner 💯💯💯.
I would vote for keeping the one on Nightly so that it keeps fixing itself.

NB: Simon Marlow is definitely not an HLS contributor. I don't even think he uses HLS currently (or any of the derivatives that we have at Meta). I think you are referring to @simonmichael

[jneira]

NB: Simon Marlow is definitely not an HLS contributor. I don't even think he uses HLS currently. I think you are referring to @simonmichael

ouch, thanks for noting it, will correct mentions

[jneira]

If you dont mind i would wait some time to see we get more feedback.
Also i would try to reach some time of consensus before having to vote to decide what to do.

[jneira]

Remove all but one stack.yaml that is "trivially" compatible (so no-to-minimal alternation is required to use it for, say, other GHC versions, preferably a fixed LTS version), and keep only the stack.yaml that requires some special treatments (such as the one for GHC 9.0.1)

Interesting how would that differ from Keep two stack.yamls: last lts and nightly (they could be the same ghc at some point though) (for example taking the actual situation as base)

[Ailrun]

In that option, we may need more than one "nightly" versions; For example, if we add the support for GHC 9.2.1, we need at least three (LTS, 9.0.1, 9.2.1) IMO.

[jneira]

ok, after some time to let users give their feedback I think it is time to try to reach a compromise: I think an intermediate alternative could be keep at minimum two stack.yaml: one for last lts and other for nightly, which will be updated periodically by contributors interested in keep it up-to-date.
It could be extended temporary to one more stack.yaml when nightly changes the ghc version, to let users some time to update (as proposed by @Ailrun)

The actual install script depends on stack.yaml so I would ask to make the removal follow the install script deprecation plan outlined in #2491

[jneira]

thoughts about the previous comment @pepeiborra @wz1000 @Ailrun @drsooch?
i would like to announce the install script and stack.yamls deprecation in the next release and start the removal process if we decide that
the plan i suggested for the install script and proposed for stack.yamls (as the script depends directly on their existence) is:

• Announce the install script deprecation in the next release (1.6.0), pointing to this issue to allow users comment about
• Make a pr to update doc installation instructions adding new ones without using the script and noting it is deprecated, just before or after the release
• After 1.7.0 remove the install script and any mention in docs (but a small note noticing it is not available anymore)

[pepeiborra]

alternative could be keep at minimum two stack.yaml: one for last lts and other for nightly, which will be updated periodically by contributors interested in keep it up-to-date.

Can you clarify what the last part means? I assume you are suggesting dropping the Circle CI, otherwise I don't see how updating the stack descriptors would be optional.

[jneira]

Can you clarify what the last part means? I assume you are suggesting dropping the Circle CI, otherwise I don't see how updating the stack descriptors would be optional.

I mean update the snapshot date in the resolver field, you can keep it without modifications most of the times but it is good to update it anyways from time to time (not only to get a new ghc version but to simplify extra deps f.e.)

[pepeiborra]

Keeping only 2 or 3 Stack descriptors sounds like a welcome improvement to me.

[Ailrun]

That plan sounds reasonable to me.

[jneira]

Great, thanks all! i think we can start to do it then