Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add spec for package pinning. #1894

Closed
wants to merge 8 commits into from
Closed

Add spec for package pinning. #1894

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
93a2631
Added inital pinned package spec.
jedieaston Feb 2, 2022
141d794
Fixed a boatload of typos :)
jedieaston Feb 2, 2022
1ab51e8
Hey, spelling bot! Most of those aren't typos.
jedieaston Feb 2, 2022
75eacca
Fixed the little things from review by @denelon and @vedantmgoyal2009.
jedieaston Feb 3, 2022
32b5560
Added PowerShell cmdlets/arguments.
jedieaston Feb 3, 2022
60bac9f
Added RequiresExplicitUpgrade discussion among other things.
jedieaston Feb 9, 2022
f71f0dc
Oops, didn't check my markdown...
jedieaston Feb 9, 2022
f6d3c82
Typo
jedieaston Feb 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 19 additions & 11 deletions .github/actions/spelling/expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
abcd
adjacents
activatable
adml
admx
affle
Expand All @@ -10,7 +9,7 @@ aicli
AICLIC
ajor
alreadyinstalled
amd
Amd
amrutha
anonymized
APARTMENTTHREADED
Expand All @@ -19,7 +18,6 @@ apicontract
apiset
appinstallertest
appname
Archs
arget
argumentlist
ARMNT
Expand All @@ -43,6 +41,7 @@ Beigi
bfd
BFirst
bght
billg
bitmask
bkup
blargle
Expand Down Expand Up @@ -72,6 +71,7 @@ chcp
ci
cinq
CLIE
Clippy
cloudapp
clsid
COINIT
Expand All @@ -91,6 +91,7 @@ ctc
Ctx
curated
CYRL
Datacenter
Dbg
debian
deigh
Expand All @@ -100,10 +101,11 @@ dirs
diskfull
dnld
Dobbeleer
dsc
Dsc
dustojnikhummer
dvinns
dw
easton
ecfr
ecfrbrowse
endian
Expand Down Expand Up @@ -140,9 +142,8 @@ GES
GESMBH
GHS
gity
Globals
Google
hackathon
Hackathon
hashtable
helplib
helplibrary
Expand All @@ -151,6 +152,7 @@ HINSTANCE
hkey
hlocal
hmodule
Howto
hre
hresults
htm
Expand All @@ -164,6 +166,7 @@ IHost
IID
IISOn
img
importantsystemdependencies
IMutable
IName
inet
Expand All @@ -186,6 +189,7 @@ IVector
IWeb
IZone
jdk
jedieaston
jfearn
JObject
jp
Expand Down Expand Up @@ -220,13 +224,14 @@ lz
malware
maxvalue
MBH
MDM
mdmp
megamorf
memcpy
middleware
midl
minidump
minexample
minidump
minschema
missingdependency
MMmmbbbb
Expand All @@ -247,7 +252,6 @@ myinstalldir
mylog
mysilent
mysilentwithprogress
mytool
NETFX
netlify
Newtonsoft
Expand All @@ -257,6 +261,8 @@ normer
NOSEPARATOR
NOTAPROPERTY
notmatch
npm
npmjs
npp
nsis
nuffing
Expand All @@ -266,9 +272,8 @@ objbase
objidl
ofile
Outptr
Packagedx
packageinuse
parametermap
PARAMETERMAP
pathparts
pathpaths
Patil
Expand All @@ -278,8 +283,10 @@ PEGI
pfn
pfxpath
Pherson
Pillay
pkgmgr
pkindex
planeteaston
PMS
positionals
powershellgallery
Expand Down Expand Up @@ -366,6 +373,7 @@ tombstoned
tpl
transitioning
trimstart
ubuntu
UCase
ucasemap
UChars
Expand Down Expand Up @@ -404,7 +412,7 @@ Webserver
website
WERSJA
wesome
whatif
Whatif
windir
windowsdeveloper
winerror
Expand Down
180 changes: 180 additions & 0 deletions doc/specs/#476 - Pinned packages.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
---
author: Easton Pillay jedieaston/[email protected]
created on: 2022-01-25
last updated: 2022-02-02
issue id: 476
---

# Pinning a package

For [#476](https://github.com/microsoft/winget-cli/issues/476)

## Abstract

This spec describes the functionality behind "pinning" a package (freezing a package at a certain version so that it will be not be automatically upgraded).
jedieaston marked this conversation as resolved.
Show resolved Hide resolved

## Inspiration

This functionality is inspired by functionality in other package managers, as well as community feedback. Many packages (software development tools and libraries commonly, but other software as well) introduce breaking changes that users may not want integrate into their workflow quite yet.

## Solution Design

A table of packages (by Name, Version and Product Code if available) that are currently pinned will be kept in the local tracking catalog. To be repository independent, these values should be correlated from local sources (Add and Remove Programs, Appx, etc). This also means that the user should be able to pin packages that were not installed via the Windows Package Manager, so that they won't be upgraded via `winget upgrade --all`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the traditional Microsoft Store packages, the Microsoft Store automatically updates programs. Users would not be able to pin those packages. I need to see if there is a way to pin packages from being automatically updated by the Microsoft Store.

Copy link
Contributor Author

@jedieaston jedieaston Feb 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to be as repo neutral as possible, but the Microsoft Store will definitely need special treatment. We need to be able to pin packages even if they are coming from "more normal" REST sources or Intune or whatever.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we define "pinning" as in this spec to "not be automatically upgraded via winget upgrade --all", then anything that upgrades itself (or is upgraded by something else) automatically will be outside of our scope. We can't control most of them anyway, so it is the best thing we can do.

Now if it turns out that Store MSIX upgrades can be put on hold on a per-package basis, we can certainly integrate that control. But that seems like icing rather than cake.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least with chocolatey, I ended up pinning for two reasons:

  • I really want a specific version
  • The application has it's own update mechanism

So the behavior of "the package may still auto update, and is just excluded from winget upgrade --all makes a lot of sense to me.



## UI/UX Design

The main interface for the pinning feature will be a new command, `winget pin`, with a couple of options:


```
winget pin
```

Running `winget pin` with no arguments will show a table of currently pinned packages:

```
Name Id Version
------------------------------------------------------------------------------------------------
Microsoft Bob Microsoft.Bob 2.35.0
iTunes Apple.iTunes 12.0.199.4

```

```
winget pin --set <package>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should cover the intent for the PowerShell cmdlets also. I'd expect several cmdlets. One to show what's pinned, one to pin, and one to "unpin".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through and added sample PowerShell cmdlet names (or arguments to existing cmdlets, where applicable), if you want to take a peek.

```

`<package>` correlates to a query, similar to `winget list`. When running this command, the package information will be stored in the table in the tracking catalog. This command should be able to accept Package Identifers (checked against all repositories) as well as names, tags, etc. If the package does not have a version number locally but is available from a source, then the version number from the source should be used.

If there is no package directly available for software a user wants to pin, but it is [included as part of another package](https://github.com/microsoft/winget-cli/issues/1073), the package which includes said piece of software should be treated as pinned unless a newer version doesn't change the version of the pinned software.

```
winget pin --clear <package>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec should include discussion of how user driven pinning and RequiresExplicitUpgrade interact. I assert that these are effectively the same thing, with the manifest field driving a default pinned state. The user should also be able to clear that default pin if they so desire.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is an opportunity to look at what exactly the RequiresExplicitUpgrade field is supposed to show. If it is intended to show the default pinned state, I would suggest we consider renaming the field to something like Pinning: and make it an enum rather than a boolean so that packages could be marked as unable to be pinned without requiring a separate field.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unable to be pinned

That may not be necessary, since winget can't guarantee that a package won't be upgraded anyway, it can only guarantee that winget upgrade won't upgrade the package without being explicitly asked.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unable to be pinned

That may not be necessary, since winget can't guarantee that a package won't be upgraded anyway, it can only guarantee that winget upgrade won't upgrade the package without being explicitly asked.

True, but it would be an opportunity to keep the user informed if we know of specific packages that auto-upgrade. Just an idea, but probably outside the realm of this spec anyways

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequiresExplicitUpgrade was about apps that update themselves and was something added to the manifest to somewhat declutter winget upgrade for packages that handle upgrades themselves or for ISVs to prefer their own upgrade mechanism. #1163. The description in the schema probably shouldn't use the word "pinned" as pinning is more of a user driven behavior. Although logically it causes apps to be excluded from winget upgrade --all which pinning also does.

Pinning a package is more about a user saying, "I don't want this upgraded". It's often the case for enterprises who have to support specific versions of applications, and developers wanting to have specific versions of packages for their needs.

We may also need to consider the case where packages won't be able to support pinning. Some packages are going to update no matter what "we" want. This could help us to inform users when we can't control the software upgrade mechanism.

```

It functions similarly to `winget pin --set`, but it clears the pin. This should automatically happen on `winget uninstall`, or if it is detected the package is no longer present on the system.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it is detected the package is no longer present on the system.

A nice goal, but difficult for us to achieve. The actual implementation might rely on future installs detecting an abandoned tracking catalog entry and erasing it before installation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe some kind of updating will be necessary of those values regardless, as if a package is pinned because it manages updates itself, whenever the update happens the pinned version in the table will be out of sync with reality.

I'm happy to remove it for now, but it is something to think about.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd put it in future considerations. When the user displays what is pinned or an individual package that is pinned, we could perform some kind of check, but I agree it's likely future work.


Enhancements will also need to be made to other commands in the Windows Package Manager, including:

```
winget upgrade --include-pinned
```

With the new `--include-pinned` argument, Upgrade should show a separate table in addition to the regular table which shows possible upgrades to pinned packages.

```
<...>
Upgrades to pinned packages:

Name Id Version Available Source
-------------------------------------------------------------------------------------------------------------------------
Clippy for Azure 2011 Datacenter Microsoft.Clippy.2011.Datacenter 2.35.0 2.35.1.2 winget
```


```
winget <upgrade/install> --include-pinned <package>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're working towards a slightly different behavior for "install" when a version is already on the system. We've been discussing essentially switching to "upgrade" unless the user does something like winget install <package> --force

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really good idea, but in that case I think the argument should still be present.

Copy link
Contributor

@vedantmgoyal9 vedantmgoyal9 Feb 3, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At line 90, you've written winget install --pin <package> but here, it has been written winget <upgrade/install> --include-pinned <package>.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--pin is to create a new pin, --include-pinned is acknowledging that you want to do something to a package that you've already pinned. Perhaps the distinction isn't relevant.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just standardize the same way the powershell one is and use --pinned

```

Upgrade and Install will now need to use the same argument to bypass the pin for a package temporarily, in which case the pin is updated to pin the new version after the installation completes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this is necessary. The goal is that the standard upgrade --all ignores these, but specifically targeted packages are how it should be done. Placing yet another hurdle seems likely to just trip people rather than actually preventing accidental upgrades.

Copy link
Contributor Author

@jedieaston jedieaston Feb 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in latest commit. I said that it should warn that you are modifying a pinned package, but continue with the usual behavior.


```
winget install --pin <package>
```

Install should also be extended to allow the user to pin a package at install time.




In addition to commands, there should be a new `Pinned` key in exported package lists, allowing users to move their pins to different systems:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have several other enhancements to add to the packages.json schema. We've been discussing supporting essentially all the install switches in the schema so running winget import packages.json would have as much flexibility as though the user was performing discrete installs. The export command likely will be somewhat limited on what it can detect in terms of switches used during an earlier install or upgrade.


```
// ...
{
"Packages":
[
{
"PackageIdentifier": "Microsoft.Bob",
"PackageVersion" : 2.35.0,
"Pinned" : "true"
},
],
}
// ...
```
jedieaston marked this conversation as resolved.
Show resolved Hide resolved

Lastly, there should be good error messaging throughout the Windows Package Manager in order to explain to users when something doesn't go quite right.

As an example, consider the situation where a user has pinned their favorite package, Microsoft Bob:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Microsoft Bob

😎


```
PS C:\Users\billg> winget install --pin Microsoft.Bob
Found Microsoft Bob [Microsoft.Bob] Version 2.32.0
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://aka.ms/importantsystemdependencies/bob.exe
██████████████████████████████ 2.08 MB / 2.08 MB
Successfully verified installer hash
Starting package install...
Successfully installed
Microsoft.Bob has been pinned at version 2.32.0.
```

Then, later, that user tries to install a package that is dependent on a newer version of Microsoft Bob. The Windows Package Manager should not upgrade Microsoft Bob automatically, but instead should notify the user and give them information on how to continue:

```
PS C:\Users\billg> winget install --pin Microsoft.RoverTheDog
Found Rover the Dog [Microsoft.RoverTheDog] Version 5.1
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
This package requires the following dependencies:
- Packages
Microsoft.Bob, version 2.39.0.

Microsoft Bob is currently pinned at version 2.32.0, but Rover the Dog 5.1 requires at least version 2.39.0.
To upgrade Microsoft Bob and other pinned dependencies, add the argument --include-pinned to your previous command.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gone back and forth between just "informing" users and giving an interactive [Y/N] sort of dialog to see if the user wants to perform suggested actions.

Copy link
Contributor Author

@jedieaston jedieaston Feb 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A [Y/N] prompt would be more user-friendly, but it would make it harder to use winget in automation scenarios. Is there a way on Windows to detect if a user is interactively using a shell vs running a script? I know the way to tell apt on linux not to ask questions is via an environment variable, but that seems kludgy.

I guess we could also say that the way forward for automating winget is via the PowerShell library or directly using COM, but since that's not complete yet...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any of the [Y/N] style prompts are likely suitable for a setting to specify a default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the way to tell apt on linux not to ask questions is via an environment variable, but that seems kludgy.

Sorry for being a little off topic. Just want to make sure you're talking about export DEBIAN_FRONTEND=noninteractive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You got it :D

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've requested that @lechacon do more extensive interactivity controls for another review. The goal would be to have a common option that disables interactivity, as well as a setting. Interactivity in general is not a good way to handle these cases, because the COM (and thus PowerShell eventually) entrypoints cannot use that pattern. But with a more generic and extensive set of controls and thoughts about non-interactive defaults, we can use them more (within reason).

```

## Capabilities

### Accessibility

Accessibility should not be impacted by this change. There will be a few more tables printed to the terminal in certain cases, but they should use the current table implementation used by `winget upgrade` and `winget list`.

### Security

Security of the Windows Package Manager should not be impacted by this change. However, security of user's software may be, as if they pin a insecure version of a package they currently will not be notified of important security updates within the Windows Package Manager.

### Reliability

The change will improve reliability, as users will be able to have fine grained control of the Windows Package Manager's upgrade functionality to ensure their workflow is not disrupted.

### Compatibility

There should not be any breaking changes to the code. Although there could be a mild breaking change to the behavior of `upgrade --all` (not all packages are upgraded anymore since pinned ones are skipped), this is purely opt-in from the user's perspective at this time (if they do not pin software, there should not be a change).

### Performance, Power, and Efficiency

There should not be any notable performance changes.


## Potential Issues

Outside of missing security updates and having possible dependency resolution failures due to pins, there should not be any other impacts. Considering the opt-in nature of pinning, it should not affect users unless they want to use it.

## Future considerations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another consideration. As verified publishers start adding their packages, they may want to force override pins / require an update. Do we need a provision for this?

Copy link
Contributor Author

@jedieaston jedieaston Feb 3, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My opinion would be that publishers shouldn't be able to override pins via the manifests, as the main reason a user would want to pin a package is because they disagree with the ISV's patching schedule. If they wanted to get around it they could add an automatic updater to their piece of software (as many have chosen to do).


- Group Policy or MDM control of pinned packages would be a natural addition to the defined functionality. Administrators may want to restrict upgrades for certain pieces of conflicting software if it will cause issues for some or all of their users.

- Packages may want to define pins for their dependencies, if they can't use anything above a certain version of a dependency. These pins could be merged into the user's current pins, making sure that the packages currently installed keep working.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great call out. We've been discussing this scenario with the dependency section. I'll be sure to add this kind of a concept. We've got the concept of a "minimum supported version" but were going back and forth on a "maximum supported version".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be in favor of a maximum version, since if a package requires a specific version then the minimum and maximum could be set to the same thing. This isn't a rare scenario, but it certainly isn't as common as a minimum version.


## Resources

- [Brew - How do I stop certain formulae from being upgraded?](https://docs.brew.sh/FAQ#how-do-i-stop-certain-formulae-from-being-updated)

- [NPM - package.json dependencies](https://docs.npmjs.com/cli/v7/configuring-npm/package-json#dependencies)

- [APT - Introduction to Holding Packages](https://help.ubuntu.com/community/PinningHowto#Introduction_to_Holding_Packages)