pacman info (Provide) owns & list executable files/commands

[goyalyashpal]

Background and Quotations

there are many things:

• files (some can be executables)
• packages (collection of files)
• groups (collection of packages)

Assumption: -i shows same info under "Provides" for both -Q and -S

  -i, --info           view package information (-ii for backup files)
  -i, --info           view package information (-ii for extended information)

• a package often provides multiple executable files:
• -Q --list <pkg-name> list files owned by queried package
• -Q --owns <file> query package that owns file
• --info <package> view package info in various fields including "Provides"

Online counterparts:

• -Qi <pkg>: -Si <pkg>
• -Ql <pkg>: -Fl <pkg> or "Files" section at packages.msys2.org/package/<pkg> (ref: VLC mpv huge package size #3587 (reply in thread))

Expectations

My naive understanding was that:

• -Qi "Provides" field ...
  would show the executable files from that package
• ... i.e. smth on likes of -Qi <pkg> | grep -E */bin

Resolved

• many don't have .exe extension, so -Qi <pkg> | grep --extended-regexp *.exe won't work. grep usr/bin is not generic enough
• update: it is consistent currently too. The ".exe" was significant here, which i missed.
• -Ql <pkg> and -Qo <file> ...
  would be consistent i.e. file in output from l when fed to o will result in same pkg

Reality & Question

1. "Provides" field says "none" very frequently despite having the package "owning" various executable files/commands
   So, what is it that "Provides" field actually lists??

2. Why do there exist 2 files for same software? i.e. one with .exe and other without? and they also behave different in respect to whom they are owned by (refer logs below)?
   as in more vs more.exe and cksum vs cksum.exe and so on!

Logs

Failures regarding "Provides":

$ # Failure: pacboy not listed
$ pacman -Qi pactoys | grep Provides
Provides        : pactoys-git  repman-git  updpkgver-git

$ pacman -Ql pactoys | grep -E */bin
pactoys /usr/bin/pacboy
   ...

$ pacman -Qo pacboy
/usr/bin/pacboy is owned by pactoys r55.8860e1f-1

$ pacman -Qi coreutils | grep Provides
Provides        : None

$ pacman -Ql coreutils | grep -E */bin
coreutils /usr/bin/cksum.exe
coreutils /usr/bin/sha256sum.exe
   ...

$ pacman -Qo sha256sum cksum
error: No package owns /usr/bin/sha256sum
error: No package owns /usr/bin/cksum

$ # Waittt whatttt 😨 ?! these are different?
$ pacman -Qo sha256sum.exe cksum.exe
/usr/bin/sha256sum.exe is owned by coreutils 8.32-5
/usr/bin/cksum.exe is owned by coreutils 8.32-5

$ pacboy debug -Si clang-tools-extra:c | grep Provides
Provides        : None

$ # From online counterpart to -Ql i.e. in "Files" section of ...
$ # https://packages.msys2.org/package/mingw-w64-clang-x86_64-clang-tools-extra

/clang64/bin/clang-apply-replacements.exe
/clang64/bin/clang-change-namespace.exe
/clang64/bin/clang-doc.exe
/clang64/bin/clang-include-fixer.exe
/clang64/bin/clang-move.exe
/clang64/bin/clang-pseudo.exe
/clang64/bin/clang-query.exe
/clang64/bin/clang-reorder-fields.exe
/clang64/bin/clang-tidy.exe
/clang64/bin/clangd.exe
/clang64/bin/find-all-symbols.exe
/clang64/bin/modularize.exe
/clang64/bin/pp-trace.exe
/clang64/bin/run-clang-tidy

$ pacman -Qi inetutils | grep Provides
Provides        : None

$ pacman -Ql inetutils | grep -E */bin
inetutils /usr/bin/
inetutils /usr/bin/dnsdomainname.exe
inetutils /usr/bin/ftp.exe
inetutils /usr/bin/rcp.exe
inetutils /usr/bin/rlogin.exe
inetutils /usr/bin/rsh.exe
inetutils /usr/bin/talk.exe
inetutils /usr/bin/telnet.exe

[Biswa96]

This documentation may help https://wiki.archlinux.org/title/PKGBUILD#provides

[sskras]

@yashpalgoyal1304, now I see where you get the extensionless path format of the command from – it's the output of pacman -Qo:

$ pacman -Ql coreutils | grep -E */bin
coreutils /usr/bin/cksum.exe
coreutils /usr/bin/sha256sum.exe
   ...

$ pacman -Qo sha256sum cksum
error: No package owns /usr/bin/sha256sum
error: No package owns /usr/bin/cksum

To developers: perhaps pacman -Qo ... should add .exe automatically when the arguments are provided without a path (and when the found path is added to the file automatically), umm?

[goyalyashpal]

perhaps pacman -Qo ... should add .exe automatically when the arguments are provided without a path

yes, that would be awesome, i guess 😅 . that is, if its not considered bloat 😆 by others

[Biswa96]

I think the behavior is expected. /usr/bin/sha256sum really does not exist. So, pacman is right that /usr/bin/sha256sum is not owned by any package. If you want to search file use pacman -F command.

should add .exe automatically when the arguments are provided without a path

Adding .exe automatically would create issues for the packages which contains both foo and foo.exe files. For example, mingw-w64-mpv

$ pacman -Qo /ucrt64/bin/mpv
/ucrt64/bin/mpv is owned by mingw-w64-ucrt-x86_64-mpv 0.35.1-3
$ pacman -Qo /ucrt64/bin/mpv.exe
/ucrt64/bin/mpv.exe is owned by mingw-w64-ucrt-x86_64-mpv 0.35.1-3

[goyalyashpal]

Adding .exe automatically would create issues for the packages which contains both foo and foo.exe files. For example, mingw-w64-mpv

thats why i was thinking of some way to supply both names at once via some external means (possibly using regex)

[sskras]

@Biswa96 wrote:

packages which contains both foo and foo.exe files. For example, mingw-w64-mpv

$ pacman -Qo /ucrt64/bin/mpv
/ucrt64/bin/mpv is owned by mingw-w64-ucrt-x86_64-mpv 0.35.1-3
$ pacman -Qo /ucrt64/bin/mpv.exe
/ucrt64/bin/mpv.exe is owned by mingw-w64-ucrt-x86_64-mpv 0.35.1-3

What is the reason to add two executables along each other?

Heck, this is an ugly hack!
msys2/MINGW-packages@6f25049#diff-c433d4797ff26bbf54e02123bb2b5e1df3e652bdcb811865bbf4e9188a0049ddR92

And it's not going away anytime soon, I guess:

package() {
  ...
  # mpv needs winpty for key bindings to work on the terminal
  mv "${pkgdir}${MINGW_PREFIX}/bin/mpv.exe" "${pkgdir}${MINGW_PREFIX}/bin/mpv_exe"
  _exename=mpv
  ...
  echo '/usr/bin/winpty ... >> "${pkgdir}${MINGW_PREFIX}/bin/${_exename}"
  mv "${pkgdir}${MINGW_PREFIX}/bin/mpv_exe" "${pkgdir}${MINGW_PREFIX}/bin/mpv.exe"
}

That's not a clean (and even ugly) way to do the integration.

---

Another good point to consider while developing the @midipix (which doesn't use .exe extensions).

[sskras]

@yashpalgoyal1304 wrote 2 hours ago:

thats why i was thinking of some way to supply both names at once via some external means (possibly using regex)

I still don't understand your primary aim. Do you wan to extract all the executable names (and skip other thing like DLLs)?

In the OP you use this following syntax:

... | grep -E */bin

The leading * is useless. It's the operator meaning "the preceding symbol occurs zero or more times". And there you give no preceding symbols here. So it becomes just grep /bin:

$ pacman -Ql mingw-w64-x86_64-mpv | grep /bin
mingw-w64-x86_64-mpv /mingw64/bin/
mingw-w64-x86_64-mpv /mingw64/bin/libmpv-2.dll
mingw-w64-x86_64-mpv /mingw64/bin/mpv
mingw-w64-x86_64-mpv /mingw64/bin/mpv.com
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

Now alongside bin there might be binaries in sbin subdir. Name of both subdirs starts differently but ends the same.
So it needs the slash moved to the end of ...bin instead, hence grep bin/, and in our case nothing changes:

$ pacman -Ql mingw-w64-x86_64-mpv | grep bin/
mingw-w64-x86_64-mpv /mingw64/bin/
mingw-w64-x86_64-mpv /mingw64/bin/libmpv-2.dll
mingw-w64-x86_64-mpv /mingw64/bin/mpv
mingw-w64-x86_64-mpv /mingw64/bin/mpv.com
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

Now name of the extensionless binary can start by any char except for the . symbol, right?, hence [^.]:

$ pacman -Ql mingw-w64-x86_64-mpv | grep bin/[^.]
mingw-w64-x86_64-mpv /mingw64/bin/libmpv-2.dll
mingw-w64-x86_64-mpv /mingw64/bin/mpv
mingw-w64-x86_64-mpv /mingw64/bin/mpv.com
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

And there should be at least one such char before the end of the string (the extensionless name), which in RegExps is denoted by the $.
This implies the operator + in the extended RE syntax, so we return -E switch back:

$ pacman -Ql mingw-w64-x86_64-mpv | grep -E bin/[^.]+$
mingw-w64-x86_64-mpv /mingw64/bin/mpv

And finally we filter out only the wanted extension that we count as executable, I suppose the that's .exe.
For that we should escape the . symbol to become \. (otherwise it means any possible symbol in the string).
So the part or RexExp to match this extension becomes \.exe, which we expect just before the end of the line, the $:

$ pacman -Ql mingw-w64-x86_64-mpv | grep -E 'bin/[^.]+\.exe$'
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

Now we might make matching the file extension optional by embracing it using braces, (\.exe), and putting the ? operator after it which means "occurs zero or one time":

$ pacman -Ql mingw-w64-x86_64-mpv | grep -E 'bin/[^.]+(\.exe)?$'
mingw-w64-x86_64-mpv /mingw64/bin/mpv
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

And if you would like to include some other file extension, eg. .com, you can use the RegExp OR-operator denoted by | symbol so that part becomes (\.com|\.exe):

$ pacman -Ql mingw-w64-x86_64-mpv | grep -E 'bin/[^.]+(\.exe|\.com)?$'
mingw-w64-x86_64-mpv /mingw64/bin/mpv
mingw-w64-x86_64-mpv /mingw64/bin/mpv.com
mingw-w64-x86_64-mpv /mingw64/bin/mpv.exe

Is that something like what you wanted?

[sskras]

@goyalyashpal wrote:

one command; be it longer; if it covers more cases (i.e. hence is more fool proof for me), works much better for me.

Thanks to you I now have additional function in my Bash environment:

$ type realbins
realbins is a function
realbins ()
{
    CMD_ALIASES=`echo $* | xargs -n1 | while read; do echo $REPLY{.exe,}; done | xargs`;
    command -v $CMD_ALIASES | pacman -Qo - 2>&1 | awk '/^\// {print $1}'
}

$ realbins mpv
/mingw64/bin/mpv.exe
/mingw64/bin/mpv

$ realbins cksum sha256sum
/usr/bin/cksum.exe
/usr/bin/sha256sum.exe

[goyalyashpal]

THIS COMMENT IS FOR MY OWN
FEEL FREE TO IGNORE :)

---

... |& grep ...
- Abridged from @ sskras at #3593 (reply in thread)

The |&¹ redirection operator sends both stdout & stderr

Ahw:

• -Qo gives errors to stderr (i.e. already sorted, nice)
• plain pipe | redirects just stdout

• This explains earlier experiment resuls,
  why ... | grep ... "sorted" results: errors outputted, stdout redirected

So:

• pacman -Qo - <<< e {cksum,sha256sum}{.exe,} |& grep by works too, although is hakish
• some way to "discard" or suppress-stderr stream will be suffice too (no need to grep)

More experimentations

$ # Old Proper way
$ command -v {cksum,sha256sum}{.exe,} | pacman -Qo - |& grep -v ^error

$ # Old hack, modified
$ pacman -Qo - <<< e {cksum,sha256sum}{.exe,} |& grep by

$ # New proper way: suppress stderr of -Qo
$ pacman -Qo - <<< $(command -v {cksum,sha256sum}{.exe,}) 2> /dev/null
$ # Yes, worked! This even maintains colors lol :)

$ # Using part of old hack to shorten a bit
$ pacman -Qo - <<< e {cksum,sha256sum}{.exe,} 2> /dev/null

$ # Let's try removing input redirection 😋 
$ pacman -Qo {cksum,sha256sum}{.exe,} 2> /dev/null
/usr/bin/cksum.exe is owned by coreutils 8.32-5
/usr/bin/sha256sum.exe is owned by coreutils 8.32-5

$ # Woww! Short, Colourful, Nice :)

---

Update: Following not needed anymore
Other is that, rearranging the input strings part towards right will help in reusing it for different input with up arrows.

<filtering stderr> <<< pacman -Qo - <<< e {cksum,sha256sum}{.exe,}

Footnotes

1. Article linked at what-is-symbol-and-in-unix-linux <-- ddg 🔍: |& in bash ↩

[sskras]

@goyalyashpal:

With all due respect, at least half of your text is quite unrelated to MSYS2.
Maybe you should start your blog about learning basics of *nix CLI?

You could even host it here, on GitHub – no need to buy a domain name: https://pages.github.com/

[goyalyashpal]

Maybe you should start your blog about learning basics of *nix CLI?

u r exactly on point. i was thinking about that from long time, but couldn't get time.

[goyalyashpal]

$ pacman -F more.exe
- @ Biswa96 at #3593 (reply in thread)

$  pacman -F more{,.exe}
msys/util-linux 2.35.2-3 [installed]
    usr/share/bash-completion/completions/more
msys/util-linux 2.35.2-3 [installed]
    usr/bin/more.exe

Thanks for sharing that.

• it ain't better for querying local packages ('ts slow, and overkill)
• But it's a must-have companion to -Ss for finding remote package name
  when the file name is not same as package name.
• I was always searching for smth like this (e.g. when i had to find pacboy, & clangd)

To query the database to know which remote package a file belongs to:
- https://wiki.archlinux.org/title/Pacman

So:

$ pacman -Ss pacboy

$ pacman -F pacboy{,.exe}
msys/pactoys r55.8860e1f-1 [installed]
    usr/bin/pacboy
    usr/share/bash-completion/completions/pacboy

$ pacboy -Ss clangd

$ pacman -F clangd{,.exe}
clangarm64/mingw-w64-clang-aarch64-clang-tools-extra 15.0.7-3 (mingw-w64-clang-aarch64-toolchain)
    clangarm64/bin/clangd.exe
   ...

---

Here's how to remove package database. I verified it with doublecmd open at that path while doing -Fy.

-Fy logs

$ pacman -F clangd
warning: database file for 'clangarm64' does not exist (use '-Fy' to download)
   ...

$ realpath $(pacman-conf dbpath)/sync/*.files
/var/lib/pacman/sync/*.files

$ realpath --help
Usage: realpath [OPTION]... FILE...
Print the resolved absolute file name;
all but the last component must exist
   ...

$ pacman -Fy
:: Synchronizing package databases...
   ...

$ realpath $(pacman-conf dbpath)/sync/*.files
/var/lib/pacman/sync/clang32.files
   ...