Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement brew cask upgrade #3396

Merged
merged 15 commits into from
Nov 27, 2017
Merged

Implement brew cask upgrade #3396

merged 15 commits into from
Nov 27, 2017

Conversation

amyspark
Copy link
Contributor

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew tests with your changes locally?

This pull request fixes Homebrew/homebrew-cask#29301 by implementing the brew cask upgrade command.
This is an initial version; I've added tests that cover a correct upgrade flow, but I would be especially interested if anyone could check that step 7 (if install fails, revert) works correctly.

Please review, and thanks for the comments!

@reitermarkus
Copy link
Member

step 7 (if install fails, revert)

What/where exactly is step 7?

Anyways, here is what I had in mind to handle this case:

Currently we have a #install and #uninstall method for every artifact class. In #uninstall the artifact is usually completely removed. To handle a failed upgrade, we should instead make #uninstall do the exact opposite of #install, i.e. if #install moves a file from A to B, #uninstall should move B back to A.

@amyspark
Copy link
Contributor Author

Here:

Behaviour for brew cask upgrade:

Same steps 1–5.

  1. For the ones that differ, run the uninstall and install steps. Uninstalling is necessary to avoid leaving old traces of it when uninstallation procedures change.
  2. If install fails (after uninstall), revert to previous version.

uninstall purges the old installation's metadata, which I need to revert the upgrade.
I will try your approach and rewrite the uninstall step.

@amyspark
Copy link
Contributor Author

amyspark commented Oct 31, 2017

So far, I've added an extra start_upgrade step that uninstalls the app but preserves the outdated Cask's metadata. But I'd need a way to back it up so as to prevent Homebrew from detecting it when the new Cask is installed.
Any ideas?

I've found a way to do so by tapping into purge_versioned_files. Please review!

@amyspark
Copy link
Contributor Author

amyspark commented Nov 3, 2017

/cc for review @reitermarkus

Copy link
Member

@reitermarkus reitermarkus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we have a #install and #uninstall method for every artifact class. In #uninstall the artifact is usually completely removed. To handle a failed upgrade, we should instead make #uninstall do the exact opposite of #install, i.e. if #install moves a file from A to B, #uninstall should move B back to A.

This part hasn't been addressed yet. Probably wasn't clear because I said #uninstall when I meant #uninstall_phase (which is called in #uninstall_artifacts).

end

def run
outdated_casks = casks(alternative: -> { Hbc.installed }).find_all { |cask| cask.outdated?(greedy?) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use select instead of find_all.

outdated_casks = casks(alternative: -> { Hbc.installed }).find_all { |cask| cask.outdated?(greedy?) }

if outdated_casks.empty?
oh1 "No packages to upgrade"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should just return here.

@@ -28,6 +28,7 @@ def initialize(cask, command: SystemCommand, force: false, skip_cask_deps: false
@verbose = verbose
@require_sha = require_sha
@reinstall = false
@upgrade = upgrade
end

attr_predicate :binaries?, :force?, :skip_cask_deps?, :require_sha?, :verbose?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add :upgrade? here and use upgrade? instead of @upgrade in other methods.

@amyspark
Copy link
Contributor Author

amyspark commented Nov 6, 2017

@reitermarkus, I'm afraid I don't yet understand your intended upgrade flow. Could you be more specific?

So far, I came up with this flow:

  1. Once the upgrade package is downloaded, uninstall_artifacts the old Cask (i.e. execute uninstall without purge_versioned_files -- see later why)
  2. install the new Cask.
  3. If successful, finalize_upgrade the old Cask (i.e. execute the remaining uninstall steps). If not successful, reinstall the old Cask.

Since I reuse the reinstall step in revert_upgrade, Cask can fetch again the old package if necessary and restore the old installation. Not deleting the old Cask's metadata is key here, because it must be copied again in save_caskfile.

@reitermarkus
Copy link
Member

reitermarkus commented Nov 6, 2017

If not successful, reinstall the old Cask.

This is what I want to avoid, because often the old version is not available anymore.

We already had it installed before, so we can reinstall from the previous local version. So you are right that we don't want to call purge_versioned_files before the upgrade.

So how do we make it possible to reinstall from the previous local version? Make uninstall_artifacts not delete anything. A simple (hopefully more specific) example:

  • A Cask installs App.app by moving it from the staged_path/App.app to appdir/App.app.
  • When uninstalling this Cask, it will straight away delete appdir/App.app.

We don't want this. Instead, we want the uninstall_phase to move appdir/App.app back to staged_path. And if we want to actually uninstall the whole Cask, purge_versioned_files then removes staged_path.


A more general overview of the flow:

  • install + uninstall
    download -> stage -> install -> uninstall -> delete

    So stage and uninstall will result in the same state, because uninstall is the opposite of install.

  • upgrade
    download -> stage new version & uninstall old version -> install new version
    -> success: delete old version
    -> error: revert partially installed new version -> install old version

Now the artifacts get re-staged, and upon an uninstall/finalize_upgrade
they are deleted by purge_versioned_files instead
@amyspark
Copy link
Contributor Author

amyspark commented Nov 7, 2017

I see now! Thanks for the explanation-- I think it's ready now.
Apologies for the build failure, I forgot to test things in my side. I'm working on the upgrade tests and the message output.

@amyspark
Copy link
Contributor Author

I've fixed all the remaining issues:

  • Updated the reinstall and uninstall tests' text.
  • I added a parameter to the uninstall_artifacts step, clear, so as to let it skip artifacts that have been manually uninstalled (eg. when testing the with-uninstall-script Cask). The move function was updated accordingly.

@commitay
Copy link
Contributor

commitay commented Nov 11, 2017

Apologies in advance if I am misunderstanding this.

Currently this uninstalls pkg Casks before downloading the upgrade, is this correct? It doesn't seem to follow the upgrade flow in #3396 (comment)

$ brew cask upgrade
==> Upgrading 1 outdated package, with result:
vfuse 1.0.6
==> Starting upgrade for Cask vfuse
==> Running uninstall process for vfuse; your password may be necessary
==> Removing launchctl service com.chilcote.vfused
Password:
==> Uninstalling packages:
com.github.vfuse
==> Unlinking Binary '/usr/local/bin/vfuse'.
==> Caveats
Cask vfuse installs files under "/usr/local". The presence of such
files can cause warnings when running "brew doctor", which is considered
to be a bug in Homebrew-Cask.

==> Satisfying dependencies
==> Downloading https://github.com/chilcote/vfuse/releases/download/1.0.6/vfuse-
######################################################################## 100.0%
==> Verifying checksum for Cask vfuse
==> Installing Cask vfuse
==> Running installer for vfuse; your password may be necessary.
==> Package installers may write to any location; options such as --appdir are i
==> installer: Package name is vfuse-1.0.6
==> installer: Installing at base path /
==> installer:PHASE:Preparing for installation…
==> installer:PHASE:Preparing the disk…
==> installer:PHASE:Preparing vfuse-1.0.6…
==> installer:PHASE:Waiting for other installations to complete…
==> installer:PHASE:Configuring the installation…
==> installer:STATUS:
==> installer:%2.229250
==> installer:PHASE:Validating packages…
==> installer:%97.750000
==> installer:STATUS:
==> installer:PHASE:Finishing the Installation…
==> installer:STATUS:
==> installer:%100.000000
==> installer:PHASE:The software was successfully installed.
==> installer: The install was successful.
==> Linking Binary 'vfuse' to '/usr/local/bin/vfuse'.
🍺  vfuse was successfully upgraded!
==> Purging files for version 1.0.5 of Cask vfuse

Edit: Also moves apps before downloading.

brew cask upgrade
==> Upgrading 1 outdated package, with result:
autodmg 1.8
==> Starting upgrade for Cask autodmg
==> Moving App 'AutoDMG.app' to '/usr/local/Caskroom/autodmg/1.7.3/AutoDMG.app'.
==> Satisfying dependencies
==> Downloading https://github.com/MagerValp/AutoDMG/releases/download/v1.8/Auto
######################################################################## 100.0%
==> Verifying checksum for Cask autodmg
==> Installing Cask autodmg
==> Moving App 'AutoDMG.app' to '/Applications/AutoDMG.app'.
🍺  autodmg was successfully upgraded!
==> Purging files for version 1.7.3 of Cask autodmg

@amyspark
Copy link
Contributor Author

@commitay, yes, you're right. I did it that way because I wanted to just install the new Cask (if it fails, it reverts everything on its own).

@amyspark
Copy link
Contributor Author

@commitay, now the flow looks like this:

amalia@Sakura:~$ brew cask upgrade
==> Upgrading 2 outdated packages, with result:
libreoffice 5.4.3, libreoffice-language-pack 5.4.3
==> Satisfying dependencies
==> Downloading https://download.documentfoundation.org/libreoffice/stable/5.4.3
######################################################################## 100,0%
==> Verifying checksum for Cask libreoffice
==> Starting upgrade for Cask libreoffice
==> Moving App 'LibreOffice.app' to '/usr/local/Caskroom/libreoffice/5.4.2/Libre
==> Unlinking Binary '/usr/local/bin/regmerge'.
==> Unlinking Binary '/usr/local/bin/regview'.
==> Unlinking Binary '/usr/local/bin/senddoc'.
==> Unlinking Binary '/usr/local/bin/ui-previewer'.
==> Unlinking Binary '/usr/local/bin/uno'.
==> Unlinking Binary '/usr/local/bin/urelibs'.
==> Unlinking Binary '/usr/local/bin/uri-encode'.
==> Unlinking Binary '/usr/local/bin/xpdfimport'.
==> Unlinking Binary '/usr/local/bin/soffice'.
==> Unlinking Binary '/usr/local/bin/unopkg'.
==> Unlinking Binary '/usr/local/bin/gengal'.
==> Moving App 'LibreOffice.app' to '/Applications/LibreOffice.app'.
==> Linking Binary 'regmerge' to '/usr/local/bin/regmerge'.
==> Linking Binary 'regview' to '/usr/local/bin/regview'.
==> Linking Binary 'senddoc' to '/usr/local/bin/senddoc'.
==> Linking Binary 'ui-previewer' to '/usr/local/bin/ui-previewer'.
==> Linking Binary 'uno' to '/usr/local/bin/uno'.
==> Linking Binary 'urelibs' to '/usr/local/bin/urelibs'.
==> Linking Binary 'uri-encode' to '/usr/local/bin/uri-encode'.
==> Linking Binary 'xpdfimport' to '/usr/local/bin/xpdfimport'.
==> Linking Binary 'soffice.wrapper.sh' to '/usr/local/bin/soffice'.
==> Linking Binary 'unopkg' to '/usr/local/bin/unopkg'.
==> Linking Binary 'gengal' to '/usr/local/bin/gengal'.
==> Purging files for version 5.4.2 of Cask libreoffice
🍺  libreoffice was successfully upgraded!
==> Satisfying dependencies
All Cask dependencies satisfied.
==> Downloading http://download.documentfoundation.org/libreoffice/stable/5.4.3/
######################################################################## 100,0%
==> Verifying checksum for Cask libreoffice-language-pack
==> Starting upgrade for Cask libreoffice-language-pack
==> Purging files for version 5.4.2 of Cask libreoffice-language-pack
🍺  libreoffice-language-pack was successfully upgraded!

@amyspark
Copy link
Contributor Author

Is there anything I can do wrt the java and virtualbox Casks, @commitay?

@commitay
Copy link
Contributor

I've open PRs with fixes for their dependent Casks, thanks anyway for the offer!

@amyspark
Copy link
Contributor Author

I added two more tests to check that failed upgrades are properly reverted (if necessary).

end

unless source.exist?
return if skip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think instead of this, the move(target, source, **options) direction should be completely separate, e.g. move_back or something.

Also, the Utils.path_occupied?(target) part above is not needed for the move_back case, because this will (by which I mean should) never happen.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right here. However, we will still need this var because uninstall tests expect delete's behaviour (skip if artifact not found).

odebug "Started upgrade process for Cask #{old_cask}"
raise CaskNotInstalledError, old_cask unless old_cask.installed? || force?

unless old_cask.installed_caskfile.nil?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should error here if there is no old cask file.

option "--greedy", :greedy, false
option "--quiet", :quiet, false
option "--force", :force, false
option "--force-update", :force_update, false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flag isn't used anywhere.

# If successful, wipe the old Cask from staging
old_cask_installer.finalize_upgrade
rescue CaskError => e
opoo e.message
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead outputting the error message here, re-raise the exception at the end. Otherwise it will exit with 0 when it fails.

@@ -83,7 +84,7 @@ def install
odebug "Hbc::Installer#install"

if @cask.installed? && !force? && !@reinstall
raise CaskAlreadyInstalledError, @cask
raise CaskAlreadyInstalledError, @cask unless upgrade?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add !upgrade? to the if above instead, or move force? || @reinstall down here.

purge_versioned_files
purge_caskroom_path if force?
end

def uninstall_artifacts
def start_upgrade
return unless upgrade?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this, this shouldn't be called anyway if this isn't an upgrade. Same goes for the methods below.

@@ -420,10 +443,10 @@ def purge_versioned_files
end
end
@cask.metadata_versioned_path.rmdir_if_possible
@cask.metadata_master_container_path.rmdir_if_possible
@cask.metadata_master_container_path.rmdir_if_possible unless upgrade?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this check here necessary, since this is not rm_rf but only rmdir?

Copy link
Contributor Author

@amyspark amyspark Nov 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Line 445 clears the Cask's versioned metadata (which is what we need for finalize_upgrade), but the lines below delete the whole metadata folder (which we need for uninstall), and then purges the whole Cask's folder (which will break everything).

(edited for clarity and completeness)

- Split move into a move_back (and clarify when it is used)
- Remove unused flags
- Raise error if installed Caskfile not found
- Error out if an upgrade fails
- Remove some defensive programming checks
@reitermarkus
Copy link
Member

Thank you for your great work here, @amyspark! 👍

🎉

@ghost
Copy link

ghost commented Nov 28, 2017

great to have this natively supported now however what about upgrading auto-updating casks? --greedy calls all "latest" casks which is pretty verbose.

I like a package manager because it provides version information and can act based on that but now the version information of the auto-updatable casks is rendered useless.

(I currently use brew cu -a all the time)

thanks for your efforts!

@toonetown
Copy link
Contributor

@monouser7dig
great to have this natively supported now however what about upgrading auto-updating casks? --greedy calls all "latest" casks which is pretty verbose.

I agree - I'd like to have the ability to upgrade greedily...but without "latest".

@vitorgalvao
Copy link
Member

I like a package manager because it provides version information

We can’t have it every way. Apps will auto-update independently of what we do, and we’re not going to cripple that (nor could we even if we wanted). So versions are going to be auto of sync anyway for casks that auto-update, no way around that.

but now the version information of the auto-updatable casks is rendered useless.

Maybe from an info perspective, but not from a cask file perspective.

I agree - I'd like to have the ability to upgrade greedily...but without "latest".

And what about the casks that are :latest but do not auto-update? You don’t want those to remain outdated.

@koutsenko
Copy link

Hello, how can i test that feature (brew cask upgrade)?

bash-3.2$ brew cask --version
Homebrew-Cask 1.3.8
caskroom/homebrew-cask (git revision ce2cb; last commit 2017-11-28)
bash-3.2$ brew cask upgrade
brew-cask provides a friendly homebrew-style CLI workflow for the
administration of macOS applications distributed as binaries.

Commands:
...
(and just help printed instead of upgrading)  
...

@toonetown
Copy link
Contributor

@vitorgalvao

Yes - those are all points that have been used in the past to argue against having an brew cask upgrade in the first place. I'm fine with whatever is decided - you can do some command line magic to achieve what I'm looking for anyway:

brew cask upgrade $(brew cask outdated --greedy --verbose | grep -v '(latest)' | cut -d' ' -f1)

@ghost
Copy link

ghost commented Nov 28, 2017

  1. I usually run upgrade before the apps auto update themselves so no problem on that side (Or on the rare occasion just reinstall using brew cu -a, on the other hand always upgrading „latest“ is a much! bigger overhead)
  2. I manyually uodate the apps that have „latest“ using brew reinstall or using their build in upgrade mechanism (chrome canary) and this number of apps decreases

Just use brew upgrade for everything that has Version Info so as soon as I run my brew command everything is ready to go. (Or at least as ready as it can be)

@MikeMcQuaid
Copy link
Member

And what about the casks that are :latest but do not auto-update? You don’t want those to remain outdated.

@vitorgalvao Out of interest, is there any metadata that indicates if autoupdate is something a Cask does. If so, perhaps that could be used here?

@vitorgalvao
Copy link
Member

@MikeMcQuaid Yes. It’s a simple stanza we add to casks.

I guess that could work: if :latest and auto_updates true, never include in upgrade.

I’m now running a script to determine how many casks we have in that situation. I’ll wager they’re not many. For reference, we have (across all official repos):

  • 5272 total casks.
  • 1655 casks with latest.

@reitermarkus
Copy link
Member

reitermarkus commented Nov 28, 2017

I would actually be in favour of also upgrading Casks with auto_updates, even without --greedy. Reason being that an app only actually auto-updates if I open it.

@MikeMcQuaid
Copy link
Member

@reitermarkus I think it's a reasonable assumption that people are opening the Casks they've installed, though (at least for a default).

@ghost
Copy link

ghost commented Nov 28, 2017

@mike But you do not open them on a regular basis, yet if you open them you still want them up to date, the main reason to use a package manager

@vitorgalvao
Copy link
Member

vitorgalvao commented Nov 28, 2017

I’m now running a script to determine how many casks we have in that situation.

The verdict is in. From 5272 casks, only 48 are version :latest and auto_updates true:

latest auto_updates casks
  • avira-antivirus
  • basecamp
  • bluesense
  • bookmacster
  • bordertool
  • bordertool2
  • busycal
  • chatwork
  • coccoc
  • domino-cli
  • doxie
  • dropbox
  • expressscribe
  • feed-the-beast
  • google-chrome-canary
  • gpg-suite-nightly
  • huamim
  • ipass
  • iterm2-nightly
  • jandi
  • jdownloader
  • joinme
  • mailspring
  • megasync
  • miniconda
  • miniconda2
  • moxtra
  • myworkspace
  • orfo
  • orfo-plus
  • plantronics-hub
  • pokerstars
  • remote-play
  • riverdesign-sparkle
  • routeconverter
  • sketchpacks
  • soda-player
  • spatial
  • spotify
  • starleaf-breeze
  • steam
  • steamcmd
  • stride
  • vk-messenger
  • xnviewmp
  • xoctave
  • zeplin
  • zoomus-outlook-plugin

I don’t think it’s worth losing sleep over this, then. Especially since ideally the number of latest casks will continue to decrease, and if they have auto_updates there’s a better chance we can get find appcasts for them (meaning they’ll stop being latest).

@MikeMcQuaid
Copy link
Member

@vitorgalvao Seems reasonable to me, thanks for checking ❤️

@tgdnt
Copy link

tgdnt commented Dec 1, 2017

@koutsenko this feature will presumably be in the next stable release of homebrew, that's why we don't have it yet by default. I tested it by creating a new branch within cd $(brew --repository) and merging origin/master into it.

@ev0rtex
Copy link

ev0rtex commented Dec 1, 2017

I just pulled master to try this out and ran into a small issue. On my system I have the virtualbox and virtualbox-extension-pack casks installed. When I tried brew cask upgrade it successfully upgraded Virtualbox but it did not install the virtualbox-extension-pack as expected...and in fact seemed to have removed it. For whatever reason I seem to have broken my copy/paste so here's a screenshot of what happened:

brew_cask_issue

Could this be a side-effect of the upgrade process for virtualbox? It's a minor annoyance but still unexpected behavior.

@reitermarkus
Copy link
Member

@ev0rtex, this seems to be specific to virtualbox, which runs brew cask uninstall virtualbox-extension-pack in uninstall_preflight. Could you open an issue at https://github.com/caskroom/homebrew-cask?

Lorwp pushed a commit to Lorwp/tldr that referenced this pull request Apr 6, 2018
Brew Cask has had a proper upgrade command since Homebrew/brew#3396
@Homebrew Homebrew locked and limited conversation to collaborators May 4, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cask Homebrew Cask
Projects
None yet
Development

Successfully merging this pull request may close these issues.

brew cask upgrade outline
10 participants