Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove sandbox functionality #6445

Closed
phadej opened this issue Dec 18, 2019 · 24 comments
Closed

Remove sandbox functionality #6445

phadej opened this issue Dec 18, 2019 · 24 comments

Comments

@phadej
Copy link
Collaborator

phadej commented Dec 18, 2019

Related is e.g. #6444 (comment)

Maybe it's still a bit too early to remove v1-build, but sandboxes could IMHO be removed. If someone really needs them, they can be simulated.

Yet we need to do some cleanup, as v2-build is the default now.

@CMCDragonkai
Copy link

Just a question. In my Gitlab CI/CD I used .cabal-sandbox as the CI cache. Now if sandboxes are removed or I switch over to V2 commands. What is the appropriate thing to cache to speed up CI builds? Is it .cabal/store?

@phadej
Copy link
Collaborator Author

phadej commented Dec 20, 2019

@CMCDragonkai yes. And .cabal/packages, when you clean a bit from there. See e.g. https://github.com/haskell-CI/haskell-ci/blob/f3e046c5ba5ba252ae44f82b2ecfb1e2d8760c0a/.travis.yml#L17-L31

EDIT: so packages .tar.gz aren't downloaded, also there's 01-index.tar.gz which would speed-up cabal update, as only delta is downloaded (and no need to cache quickly rebuildable stuff).

@phadej phadej added this to the 3.4 milestone Dec 21, 2019
@Mistuke
Copy link
Collaborator

Mistuke commented Jan 19, 2020

Maybe it's still a bit too early to remove v1-build, but sandboxes could IMHO be removed. If someone really needs them, they can be simulated.

How? :) I still use them to reproduce GHC issues people submit so I don't mess up my global config but genuinely unsure how to simulate them. The biggest thing I use them for is to have a package-db with all the dependencies of the repro the user submitted.

Since I often repro with ghc development builds I have to run with --allow-newer so I don't want these packages in my global store-dir as as soon as the ghc version changes I gotta rebuild again anyway (and the space isn't cleaned up automatically).

But new-install 's dist folder only contains a package-db with the package only, not it's dependencies.. so I end up having to use --store-dir to point it locally and then use two package-dbs where sandbox just has one so it's easier to use.

@hasufell
Copy link
Member

Sandboxes are still used by some users and are very robust. Recovering from problems/failure is easy, while with v2-build, it basically boils down to removing the entire store. Semi-reproducible bugs like #6483 are a good example

@phadej
Copy link
Collaborator Author

phadej commented Jan 22, 2020

My opinion is: people who need sandboxes could keep using old cabal-install versions. The less code there is blocking new development work, the better.

@hasufell
Copy link
Member

The less code there is blocking new development work, the better.

I think that's a little quick. In case the code is really a problem, there should at least be a call for maintainers picking that up. We should know clearly whether there is no interest in this feature or not.

@Mistuke
Copy link
Collaborator

Mistuke commented Jan 23, 2020

My opinion is: people who need sandboxes could keep using old cabal-install versions. The less code there is blocking new development work, the better.

I could.. If new GHC didn't often require bleeding edge Cabal.

@Mistuke
Copy link
Collaborator

Mistuke commented Jan 23, 2020

In any case, whatever you guys decide as maintainers I'll live with :) especially if it's getting in the way of new things.

@hasufell
Copy link
Member

Also note, you'll probably break tools like https://github.com/haskell-mafia/mafia when removing sandboxes.

@hvr
Copy link
Member

hvr commented Mar 31, 2020

Well, back in 2018 a ticket for "Investigate porting to use cabal new-build" was filed at haskell-mafia/mafia#240 -- but it doesn't seem like there has been much activity looking into how to adapt to cabal's announced v2-style UI. And I've got a hunch this isn't going to change unless there's a forcing function to do so.

So I'm for removing these cabal sandbox codepaths at last -- they're scheduled for removal anyway sooner or later (so the cost of migrating to a different mechanism will be incurred anyway); but delaying doing so translates into a maintainance/development cost for us, and consequently removing it sooner rather than later minimises the overall cost.

Fwiw, while I haven't used Mafia myself, I believe it should be possible to emulate most of cabal sandbox-like workflows atop of cabal v2-build by orchestrating cabal accordingly and controlling the paths where it stores things. I'm happy to help brainstorm w/ the Mafia devs how to do this if they can elaborate on Mafia's requirements and what operations it needs to effect on a cabal project.

@hasufell
Copy link
Member

So there was no survey or announcement to figure out if sandboxes are still a feature the community wants?

@hvr
Copy link
Member

hvr commented Mar 31, 2020

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

$ /opt/cabal/2.4/bin/cabal sandbox 
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.

Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.

For more information, see: https://wiki.haskell.org/Cabal/NewBuild

Moreover, the wording

For those who do not wish to use the new functionality, the classic project style will not be removed immediately, but these legacy commands will require the usage of the v1- prefix as of Cabal 3.0 and will be removed in a future release.

has been in the user guide since 2018 as well: https://cabal.readthedocs.io/en/latest/nix-local-build-overview.html

@hasufell
Copy link
Member

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

The cabal help output is large and confusing. I remember people in #haskell largely had the impression that it's just about the deprecation of default behavior, not deprecation of the feature.

has been in the user guide since 2018 as well: https://cabal.readthedocs.io/en/latest/nix-local-build-overview.html

Again, I don't think that is a discussion or even a proper announcement.

@phadej
Copy link
Collaborator Author

phadej commented Apr 22, 2020

I'm developing new feature: active-repositories. It would allow using multiple simultaneous repositories (e.g. hackage.haskell.org + head.hackage) more "deterministic", as the ordering
could be configured. One usage (e.g. for Nix users) is active-repositories: :none - i.e. you wouldn't need to spin complete new ~/.cabal/config just to make "offlline" (but not really) mode.

It's development is currently blocked transitively on this issue.

One needs to remove legacy local repositories (#6729), as they are unnamed so cannot be specified in active-repositories.

However, the above is again blocked on sandboxes, as legacy repositories seems to be used in the sandbox code.


One way forward is just to make sandbox code compile, and disable failing tests. It won't be hard removal, but most likely sandboxes will be broken in 3.4.

@phadej
Copy link
Collaborator Author

phadej commented Apr 22, 2020

I'll proceed with above "just break sandboxes" approach in two weeks, i.e. Wednesday 2020-05-06.

@23Skidoo
Copy link
Member

Shipping a known-broken implementation is probably worse than outright removing sandboxes.

@emilypi
Copy link
Member

emilypi commented Apr 22, 2020

@phadej thank you for this. I'm 100% in support of your approach! If you need any help, please feel free to direct me towards anything you need done 😄

@hvr
Copy link
Member

hvr commented Apr 22, 2020

@23Skidoo just to clarify, by

Shipping a known-broken implementation is probably worse than outright removing sandboxes.

You do mean, that given that sandboxes will be "known-broken" in 3.4 anyway, it's reasonable (i.e. less "worse") to "outright remove" them with cabal 3.4 (which I'd agree with)?

@23Skidoo
Copy link
Member

Yeah, if we don't have the bandwidth to keep supporting sandboxes, let's just remove them.

@phadej phadej mentioned this issue May 6, 2020
@phadej phadej modified the milestones: Considered for 3.4, 3.4.0.0 May 14, 2020
@phadej
Copy link
Collaborator Author

phadej commented May 15, 2020

Sandboxes are removed, and I send a message to cabal-devel/

@phadej phadej closed this as completed May 15, 2020
@xplat
Copy link

xplat commented Jun 9, 2020

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

$ /opt/cabal/2.4/bin/cabal sandbox 
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.

Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.

This message ends up looking really deceptive, referring to a new-sandbox command that did not exist and was never intended to exist. It's even more unfortunate given how many commands in the v2 series were intended to exist but took a long time to materialize. Water under the bridge now, but this understandably came as a surprise to a lot of people.

@phadej
Copy link
Collaborator Author

phadej commented Jun 9, 2020

I can only repeat that changing to use cabal-install-3.4 is not mandatory. If you are happy with previous versions and there is no pressing need to upgrade, don't.

Also cabal-install-3.4 will only be released in about three months (the earliest!) so there is still time to prepare.

@ivanperez-keera
Copy link
Contributor

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

From what I see, the decision is made, so I'm saying this with the hope to find a better way to stay on top of these discussions and contribute in a timely manner when and where they happen. And, just in case, I absolutely understand that maintaining these features can be a burden and as a user all I have is gratitude for cabal (and the whole Haskell ecosystem) existing in the first place, and I thank all who make it happen.

I saw these warnings years ago and I never interpreted this as "open to discussion", nor did I find a pointer to express an opinion.

As in prior occasions, I had to opt to try and "stay alive" in any way I could: the Haskell ecosystem generally changes so, so much that I've always found myself having to choose between creating up to date Haskell tools, or creating real-world applications with (not-so-new) Haskell. However, the burden of doing both has been increasingly higher in any of the environments where I worked. I made a decision to opt for the latter: it is a no-brainer IMO, but my opinion seems to counter that of others.

I'd like to inquire about the best way to stay on top of these discussions, and participate sooner if possible. For example, being subscribed to Cabal on github means having to process an average of 2x issues a day, just in case a discussion about a breaking feature comes up, and each issue will be an asynchronous conversation thread on its own. We also have GHC, base, plus the hundreds of libraries that projects tend to depend on these days because the general tendency for fine partitioning and additional levels of abstraction.

And just to be clear, I'm not trying to disguise a complaint as a question. I see that people seem to be able to cope with these levels of complexity, and I wonder what I'm missing, or what methods others may have found. In the last 20 years, projects have become much, much harder to maintain IMO.

@phadej
Copy link
Collaborator Author

phadej commented Jul 8, 2020

@ivanperez-keera I mentioned this in https://mail.haskell.org/pipermail/cabal-devel/2020-May/010482.html and will soon (in July) post a call-for-features for 3.6 with preliminary list of what I know is worked on. cabal-devel is very low-traffic list.

okeuday added a commit to CloudI/CloudI that referenced this issue Aug 5, 2022
…3.4 which has removed the v1-sandbox command (haskell/cabal#6445) and all sandbox functionality.  cabal v2-install --lib appears unable to provide a local installation in a sandbox-like directory though that functionality may be pursued as an "env" command (haskell/cabal#6481).  Support of GHC >= 9.0 will be blocked on cabal-install's ability to use local installations of libraries.  Changes based on the current cabal-install support have been added for further improvement.  Update Haskell network dependency to 3.1.2.7 .  Update Haskell zlib dependency to 0.6.3.0 .
okeuday added a commit to CloudI/CloudI that referenced this issue Aug 10, 2022
…ox functionality was removed (cabal-install >= 3.4) without replacement functionality (haskell/cabal#6445).  cabal-install source code releases are unable to bootstrap with local dependencies due to undependable releases that promote opaque binaries (haskell/cabal#7904).  Manually create a Cabal sandbox without cabal-install for Haskell GHC compilation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants