Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spec for package pinning #2611

Merged
merged 10 commits into from
Feb 9, 2023
Merged

Spec for package pinning #2611

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
64cc0c3
Create #476 - Package Pinning.md
yao-msft Oct 13, 2022
356284c
Update #476 - Package Pinning.md
yao-msft Oct 13, 2022
7c66d6c
Update #476 - Package Pinning.md
yao-msft Oct 13, 2022
7cf4815
Update #476 - Package Pinning.md
yao-msft Oct 18, 2022
d0f97eb
Update #476 - Package Pinning.md
yao-msft Oct 18, 2022
2d7ca4d
Update #476 - Package Pinning.md
yao-msft Oct 18, 2022
1d8b3bf
Update #476 - Package Pinning.md
yao-msft Oct 18, 2022
d3b7f75
Update #476 - Package Pinning.md
yao-msft Oct 18, 2022
2b0c15f
Update #476 - Package Pinning.md
yao-msft Oct 19, 2022
459cb45
Update #476 - Package Pinning.md
yao-msft Oct 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 205 additions & 0 deletions doc/specs/#476 - Package Pinning.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
---
author: Yao Sun @yao-msft
created on: 2022-10-12
last updated: 2022-10-12
issue id: 476
---

# Package Pinning

For [#476](https://github.com/microsoft/winget-cli/issues/476)

## Abstract

This spec describes the functionality and high level implementation design of Package Pinning feature.

## Inspiration

This is inspired by functionalities in other package managers, as well as community feedback.
- Packages may introduce breaking changes that users may not want integrate into their workflow quite yet.
- Packages may update themselves so that it will be duplicate effort for winget to try to update them.
- User may want to maintain some of the packages through other channels outside of winget, or prefer one source over others within winget.
- User may want some of the packages to stay in some major versions but allow minor version changes during upgrade.

## Solution Design

#### Package Pinning types
To achieve goals listed above, winget will support 3 types of Package Pinning:
- **Blocking:** The package is blocked from `winget upgrade --all` or `winget upgrade <specific package>`, user has to unblock the package to let winget perform upgrade.
- **Pinning:** The package is excluded from `winget upgrade --all` but allowed in `winget upgrade <specific package>`, a new argument `--include-pinned` will be introduced to let `winget upgrade --all` to include pinned packages.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: extra "to"

Suggested change
- **Pinning:** The package is excluded from `winget upgrade --all` but allowed in `winget upgrade <specific package>`, a new argument `--include-pinned` will be introduced to let `winget upgrade --all` to include pinned packages.
- **Pinning:** The package is excluded from `winget upgrade --all` but allowed in `winget upgrade <specific package>`, a new argument `--include-pinned` will be introduced to let `winget upgrade --all` include pinned packages.

- **Gating:** The package is pinned to specific version(s). For example, if a package is pinned to version `1.2.*`, any version between `1.2.0` to `1.2.<anything>` is considered valid.
Comment on lines +27 to +30
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to define how each type of pin interacts with RequireExplicitUpgrade. I'm guessing something like this:

  • Pinning pins and RequireExplicitUpgrade behave the same way. Even though they do the same things, we allow pinning in case the manifest gets updated to remove the field or the package updates to a version without that field.
  • Blocking pins override RequireExplicitUpgrade, so we won't update even if the include flag is added. We have to both remove the blocking pin and pass the flag when updating.
  • Gating pins block from updating beyond the gated version, even with the special flag for RequireExplicitUpgrade, and updates within that version range still require the special flag.

Does that sound good?


To allow user override, `--force` can be used with `winget upgrade <specific package>` to override some of the pinning created above.
florelis marked this conversation as resolved.
Show resolved Hide resolved

#### Package Pinning Configuration Storage

A separate sqlite db (other than the existing tracking catalog) will be created to store the package pinning configurations from user.
```text
PackageIdentifier SourceIdentifier Version PinningType
----------------------------------------------------------------------------
Microsoft.TestApp winget 1.2.* Gating
```

**Notes:** For this iteration, winget will only support pinning packages that are locally installed and correlatable with at least one of the remote sources. Winget will record a pinned package by the PackageIdentifier and SourceIdentifier. There can only be one pinning configuration for a specific package. In the future, winget may consider pinning packages from installed packages (upon improving the installed package's PackageIdentifier logic).

## UI/UX Design
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the powershell cmdlets considered here? I'm assuming the commands will be something like Add-WingetPackagePin and Remove-WingetPackagePin, but didn't see those in here

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verbs Add and Remove are usually used for resources that support having multiple things added to them, such as with Add-Content appending to a file or Add-ADGroupMember adding one or more members to a group which may countain N amount of members total.

Since a package would probably only ever have one pin state (yes with options or no) at a time, I would maybe suggest the verbs Suspend/Resume (updates), Get/Set/Reset (a pin) or Lock/Unlock (a version),

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok the:

There can only be one pinning configuration for a specific package.

threw me off here, I can see from further down the document now that a package can have multiple co-existing pins for different sources. That makes sense, so Get/Add/Remove/Reset (or Remove without parameters = Reset) makes sense

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking along the lines that each pin is an item in a collection of pins (even if the package identifier is different, they are still part of the collection), so something like Get-WingetPackagePin would show all pins (Similar to how Get-AppxPackage shows all AppxPackages installed) since Add and Remove are referring to the collection of pins, not the individual package pin

Copy link
Contributor Author

@yao-msft yao-msft Oct 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Powershell support is not considered in this spec yet :(. I'll add it to future list and we'll probably get back to it after core feature is done.

edit: winget team is moving Powershell cmdlets to call Com Apis to perform winget operations, that means Com Api needs to support the pinning feature first before Powershell support could be enabled. More work is needed..


#### winget pin commands

A new `winget pin` command with 3 sub-commands will be introduced.
- Add package pinning configuration:

`winget pin add <package> [--version <optional gated version>] [--source <source>] [--force] [--blocking]`

This comment was marked as resolved.

Sign in to view

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, they'll be same

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that also go for the package query arguments this will use? So, having --query, --id, --name, --moniker, --tag, --command? (I'm already excluding --manifest as that is not linked to a source.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, any args for finding a package and applicable for search a package to pin.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify the behavior when a pin already exists for a package? Will a user have to remove the existing pin and re-pin? A winget pin update command? Or is this what the --force parameter is for, so that a warning can be displayed that existing pins would be overwritten and they must force the addition to override it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I will add more clarification and example for it.
Basically, if a pin configuration already exists, we'll show a warning and exit. User will remove the pin and add a new one, or use --force to override

florelis marked this conversation as resolved.
Show resolved Hide resolved

- Remove package pinning configuration:

`winget pin remove <package> [--source <source>] [--force]`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add aliases for the subcommands? I see we are using source rm for source remove, rm for uninstall, ls for list. Personally, I'd like to have consistency and add pin rm, pin ls (and source ls).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, consistency is better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, and I bet @Trenly would agree :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

;)


- List package pinning configuration:

`winget pin list <package> [--source <source>]` for a specific package or `winget pin list` to list all
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might also suggest winget pin remove --all --force or winget pin reset --force to fully reset the pinning state, similar to a source reset, where the force argument would be required

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add winget pin reset --force


#### Blocking
To block a package from upgrade, use `winget pin add <package> --blocking`
```text
cmd> winget pin Microsoft.TestApp --blocking
```
Now the pinning configuration is recorded as
```text
PackageIdentifier SourceIdentifier Version PinningType
----------------------------------------------------------------------------
Microsoft.TestApp winget Blocking
Microsoft.TestAppStore msstore Blocking
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users might not be able to pin packages from the msstore source unless, as @JohnMcPMS pointed out, it is made very clear that pinning is not a guarantee the package won't be upgraded outside of winget, it's just saying that it won't be upgraded as part of winget upgrade. Either documentation around this needs to be considered, or provisions should be made to not allow pinning of packages installed from msstore.

With the traditional Microsoft Store packages, the Microsoft Store automatically updates programs. Users would not be able to pin those packages. I need to see if there is a way to pin packages from being automatically updated by the Microsoft Store.

Originally posted by @denelon in #1894 (comment)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the pins are recorded for each source present at the time of adding the pin then if a user later adds another source the package would be upgraded from there right? I think from a UX perspective if I ran winget pin add <package> --blocking I would expect the package to be pinned indefinitely for all current and future sources. So I would prefer if pinning a package without explicitly listing individual sources would result in a different, bool SQLite column being set like AllSources rather than multiple entries with arbitrary SourceIdentifiers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re @Trenly , all these pin configurations only apply to winget behaviors, we cannot control upgrades outside winget. Blocking simply blocks winget from doing anything to the blocked package. In the original issue, some community member wanted winget to block upgrade for some specific package, because they want to perform upgrades outside winget through other channel.

Re @jantari , yes, it would be ideal to pin an installed package for all future sources. Due to current limitation on the package identifier generation for installed packages, pinning from installed source is put into future considerations. Currently, a package identifier for an installed package is its produce code, if user upgrade the package, the pin is no longer in effect since most likely product code will change after an upgrade. Regarding AllSources in sqlite, it's trivial to record it, but we don't have efficient cross reference for remote sources yet, so it's unfortunately moved as future considerations.

```
**Note:** by default packages correlated from all sources are blocked, user can pass in `--source` to block for a specific source

Corresponding upgrade behavior
```text
cmd> winget upgrade -all
Microsoft TestApp is blocked from upgrade and skipped

cmd> winget upgrade Microsoft.TestApp
Microsoft TestApp is blocked from upgrade

cmd> winget upgrade Microsoft.TestApp --force
Success
```

#### Pinning
To pin a package from `winget upgrade --all`, use `winget pin add <package>`
```text
cmd> winget pin Microsoft.TestApp
```
Now the pinning configuration is recorded as
```text
PackageIdentifier SourceIdentifier Version PinningType
----------------------------------------------------------------------------
Microsoft.TestApp winget Pinning
Microsoft.TestAppStore msstore Pinning
```
**Note:** by default packages correlated from all sources are pinned, user can pass in `--source` to pin for a specific source

Corresponding upgrade behavior
```text
cmd> winget upgrade -all
Microsoft TestApp is pinned from upgrade and skipped

cmd> winget upgrade Microsoft.TestApp
Success
```

#### Gating
To gate a package to some specific version, use `winget pin add <package> --version <gated version>`
```text
cmd> winget pin Microsoft.TestApp --version 1.2.*
```
Now the pinning configuration is recorded as
```text
PackageIdentifier SourceIdentifier Version PinningType
----------------------------------------------------------------------------
Microsoft.TestApp winget 1.2.* Gating
Microsoft.TestAppStore msstore 1.2.* Gating
```
**Note:** by default packages correlated from all sources are gated, user can pass in `--source` to gate for a specific source

Corresponding upgrade behavior
```text
cmd> winget upgrade -all
Success // If the available versions for upgrade are: 1.2.3 and 1.3.0, the selected version for upgrade is 1.2.3

cmd> winget upgrade Microsoft.TestApp
Success // If the available versions for upgrade are: 1.2.3 and 1.3.0, the selected version for upgrade is 1.2.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the package gets updated to 1.3.0 outside of winget? Do we show a warning, offer a way to re-install the pinned version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've commented about that situation before. I think it would be good for WinGet to show when a "pin" has been violated in cases other than "blocking". I think we would also want to make sure that if a user applied "--force" or some other formal override, that the pin is either removed or ideally updated and the user is informed.


cmd> winget upgrade Microsoft.TestApp --version 1.3.0
Microsoft TestApp is gated to version 1.2.* Override with --force

cmd> winget upgrade Microsoft.TestApp --version 1.3.0 --force
Success
```

**Note:** Regarding gated version syntax, it will be mostly same as what current winget version supports, except with special `.*` in the end as wild card matching any remaining version parts if there are any.

Example:
When `.*` in the end is detected:
Gate version `1.0.*` matches Version `1.0.1`
Gate version `1.0.*` matches Version `1.0`
Gate version `1.0.*` matches Version `1`
Gate version `1.0.*` matches Version `1.0.alpha`
Gate version `1.0.*` matches Version `1.0.1.2.3`
Gate version `1.0.*` matches Version `1.0.*`
Gate version `1.0.*` does not match Version `1.1.1`

In rare cases where `*` is actually part of a version, only the last `.*` is considered wild card:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How likely is this case to truly exist? A quick check of winget-pkgs shows that no packages there currently have a version which includes *. I fear this may be limiting functionality for an edge case that is almost non-existent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlikely, I just wanted to clarify that for any gate version, only the .* in the end is considered. If there is a need to match1.*.*, user should probably use 1.* instead. This is just for simpler logic for parsing and matching gate version, to be less error-prone.

But if there's a need to match gate version like 1.*.2 where 1.1.2 matches and 1.0.3 does not match, I can make a version part with only * as a wild card, that's also fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it really matters too much. Thank you for the explanation on the reasoning!

Gate version `1.*.*` matches Version `1.*.1`
Gate version `1.*.*` matches Version `1.*.*`
Gate version `1.*.*` does not match Version `1.1.1`

If no `.*` in the end is detected, the gate version gates to the specific version:
Gate version `1.0.1` matches Version `1.0.1`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarification would be useful around 1.0.1a or 1.0.1b

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also things like gate version 1.0.1 (no .*) and version 1.0.1.2.

Gate version `1.0.1` does not match Version `1.1.1`

## Capabilities

### Accessibility

Accessibility should not be impacted by this change. There will be a few more tables printed to the terminal in certain cases, but they should use the current table implementation used by `winget upgrade` and `winget list`.

### Security

Security of the Windows Package Manager should not be impacted by this change. However, security of user's software may be, as if they pin a insecure version of a package, it will not be upgraded by Winget unless explicitly requested by user.

### Reliability

The change will improve reliability, as users will be able to have fine grained control of the Windows Package Manager's upgrade functionality to ensure their workflow is not disrupted.

### Compatibility

There should not be any breaking changes to the code. Although there could be a mild breaking change to the behavior of `upgrade --all` (not all packages are upgraded anymore since pinned ones are skipped), this is purely opt-in from the user's perspective at this time (if they do not pin software, there should not be a change).

### Performance, Power, and Efficiency

There should not be any notable performance changes.

## Potential Issues

- Installation/Upgrades from Com Apis may be impacted by user's package pinning configuration. It could be mitigated by returning a specific error code and the caller retrying with Force option.
- Package dependencies resolution may be impacted by user's package pinning configuration.
- Package imports may be impacted by user's package pinning configuration.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Package imports may be impacted by user's package pinning configuration.
- Package imports may be impacted by user's package pinning configuration.
- Package exports may not consider user's package pinning configuration


## Future considerations

- Implementation in this spec only supports pinning from remote sources, so all installed versions from same package share the same pinning configuration. Winget could better support side by side installations by introducing package pinning from installed source.
- Package pinning from user and from manifest are stored separately, we may integrate the `winget pin` commands to control package pinning from manifests.
- A couple UI integrations can be made to `winget upgrade` and `winget list` to show pinned status during listing.
- Dependencies flow can be improved to first check pinned status of each dependant package before trying to install all dependencies.
- Support setting pinned state right after installation/upgrades like `winget install foo --pin`.
- Improvements to import export commands to work seamlessly with existing package pinning configurations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Improvements to import export commands to work seamlessly with existing package pinning configurations.
- Improvements to import export commands to work seamlessly with existing package pinning configurations.
- Pinning multiple versions of a single package installed side by side


## Resources

- [Brew - How do I stop certain formulae from being upgraded?](https://docs.brew.sh/FAQ#how-do-i-stop-certain-formulae-from-being-updated)
- [NPM - package.json dependencies](https://docs.npmjs.com/cli/v7/configuring-npm/package-json#dependencies)
- [APT - Introduction to Holding Packages](https://help.ubuntu.com/community/PinningHowto#Introduction_to_Holding_Packages)
- [Chocolatey - pin a package](https://docs.chocolatey.org/en-us/choco/commands/pin)

Special thanks to [@jedieaston](https://github.com/jedieaston) for coming up with the initial draft of Package Pinning spec at [#1894](https://github.com/microsoft/winget-cli/pull/1894/). A lot has been discussed and this spec is much inspired from the draft.