Skip to content
Gabor Szarnyas edited this page Dec 14, 2023 · 270 revisions

Related pages:

A superb cheat sheet is available at https://github.com/kevinthew/linuxgems/blob/master/cheat_sheet.org.sh

Keys

  • Alt + mouse wheel up/down: zoom / magnifier (configurable in some window managers, e.g. Xfce and Cinnamon).

(These only work when using a Hungarian keyboard layout.)

  • Alt Gr + 2 + number: subscript, e.g. ₀₁₂₃₄₅₆₇₈₉
  • Shift + Alt Gr + 3 + number: superscript, e.g. ⁰¹²³⁴⁵⁶⁷⁸⁹

Bash commands

Minimal safe Bash script template

See Minimal safe Bash script template and Safer bash scripts with 'set -euxo pipefail'.

#!/usr/bin/env bash

set -euo pipefail

Create a bootable pendrive (USB stick)

sudo dd if=/path/to/installer.iso of=/dev/sdX oflag=direct bs=1048576 && sync

sdX should be set to the address of the device (e.g. sdc) and not the address of the partition (e.g. sdc1). Else you may get one of the following error messages:

  • Operating system load error.
  • isolinux.bin missing or corrupt.

You can list the devices with sudo fdisk -l.

Navigate to the directory the Bash script is stored in

cd "$( cd "$( dirname "$0" )" && pwd )"

Source: http://stackoverflow.com/questions/59895/can-a-bash-script-tell-what-directory-its-stored-in

Fix broken Unicode zip filenames

We had a zip file containing filenames with broken accented characters. This is an incomplete set of commands to fix them.

rename "s/µ/Á/g" *
rename "s/é/Ú/g" *
rename "s/à/Ó/g" *
rename "s/Ö/Í/g" *
rename "s/$(echo -ne '\u0082')/é/g" *
rename "s/$(echo -ne '\u0099')/Ö/g" *
rename "s/$(echo -ne '\u008A')/Ő/g" *
rename "s/$(echo -ne '\u009A')/Ü/g" *

Compress a directory with tar

tar -zcvf prog-1-jan-2005.tar.gz /home/jerry/prog

Pay attention that that order of the zcvf arguments matter, z has to come first.

Copy the current working directory to the clipboard

Use xsel:

alias pw='pwd | head -c -1 | xsel -b'

[szarnyasg] However, this does not work with some applications (that make up half my daily workflow): IntelliJ IDEA, yEd, etc. So, a better alternative is to use xclip and Parcellite:

alias pw='pwd | head -c -1 | xclip'

On Wayland, wl-copy works reliably with the applications mentioned above:

alias pw='pwd | head -c -1 | wl-copy'

Go to the Properties dialog in Parcellite and check Use Primary (Selection) and Synchronize clipboards.

Using arrays in Bash

Updating Apt repositories in Ubuntu

The following command changes the Apt repositories form 12.04 (Precise Pangolin) to 14.10 (Utopic Unicorn).

sudo sed -i "s/precise/utopic/" /etc/apt/sources.list
sudo apt-get update

No password for sudo commands

Edit the sudoers file with the following command:

sudo visudo

Add a line at the end of the file for your user, e.g.:

myuser ALL=(ALL) NOPASSWD: ALL

Add a trap for exiting from a Bash script upon Ctrl+C

Bash will not cancel the script if you are inside a loop, but instead, carry on with the next iteration.

Solution:

cleanup()
# example cleanup function
{
  rm -f /tmp/tempfile
  return $?
}

control_c()
# run if user hits control-c
{
  echo -en "\n*** Ouch! Exiting ***\n"
  cleanup
  exit $?
}

# trap keyboard interrupt (control-c)
trap control_c SIGINT

Source: http://hacktux.com/bash/control/c

Halting an Ubuntu server machine

In some virtual machines, sudo halt will require you to stop the VM manually. Use the following command instead:

sudo shutdown -h now

Source: http://ubuntuforums.org/showthread.php?t=1968626

Copying a file from the file manager to the working directory of a terminal emulator

If you copy a file from the file manager application (e.g. Nautilus, Nemo), you can paste it in the terminal by issuing cp <path> ., where you can obtain the <path> by right clicking and choosing Paste (Shift+Insert will not work).

cp file:///home/szarnyasg/eclipse/about.html .

Mount NTFS partition

Add the following line to the /etc/fstab file:

/dev/sdb1 /home/szarnyasg/hdd ntfs-3g defaults,locale=en_US.utf8 0 0

Don't forget to test with

sudo mount -a

Send a file back over and SSH connection

Install zssh on both the client and the server. Login to the server using the zssh user@host command. Use sz <filename> to send the file, then go to file transfer mode with Ctrl+Space and use rz to receive the file.

Source: http://askubuntu.com/questions/13382/download-a-file-over-an-active-ssh-session

Tmux: save scroll-back buffer to file

  • prefix, :capture-pane -S -3000 to save 3000 lines
  • prefix, :save-buffer /home/ubuntu/filename.txt

Source: https://unix.stackexchange.com/a/236845

See also: tmuxcheatsheet.com

File operations

Read a file line by line
while read line; do echo $line; done < filename
Calculate SHA1 checksums for each file in a directory

The -print0, -z, and -0 switches are required for files with spaces in the filename.

find . -type f -print0 | sort -z | xargs -0 sha1sum -b > checksum.sha
# check
sha1sum -c checksum.sha
String in-place replacement in all files found in a directory and in its subfolders

I used this command when refactored a large codebase without an IDE.

find ./ -type f -exec sed -i -e 's/string-to-replace-regex/replacement-string/g' {} \;

Alternatively, you can use ag -l to find files that contain a given string, then execute sed. See this answer here.

Set the modification date of all files with a given extension

Again, the find command can be combined with touch.

find . -iname "*.ext" -exec touch {} \;

This will touch all files with the given extension under the current directory (transitively).

comm - compare two sorted files line by line
comm -3 file1.txt file2.txt
comm -3 <(sort file1.txt) <(sort file2.txt)

-3 = suppress column 3 (lines that appear in both files)

Source: https://unix.stackexchange.com/a/377667

Intersect two files

comm -12 <(cat file1.txt) <(cat file2.txt)
Concatenate multiple files but include filename as section headers
tail -n +1 *.txt

(source)

Piping and redirection

Redirect or pipe both stderr and stdout
(time ls) &> results.txt
(time ls) |& tee results.txt
# compatible with Bash <4
(time ls) 2>&1 | tee results.txt

Source: https://stackoverflow.com/a/363239

Process substitution
diff <(transform <file1) <(transform <file2)
tee >(transform1 >out1) >(transform2 >out2)

Source: https://superuser.com/a/184412

Compression

Zip files in a directory separately
for i in *; do zip "$i.zip" "$i"; done
Unzip multiple zip files

The trick is the apostrophe:

unzip '*.zip'
Convert subtitles to UTF-8

Most Hungarian subtitles use Latin-2 encoding, which results in malformed õûÕÛ characters. To fix them, run this script:

for f in *.srt; do g=${f%.*}; mv $g.srt $g-original.srt; iconv -f ISO-8859-2 -t UTF-8 $g-original.srt > $g.srt; done

Resize partition with data

Guide for growing or shrinking a partition: https://www.howtoforge.com/linux_resizing_ext3_partitions#-shrinking-an-ext-partition

Delete history from command line

cat /dev/null > ~/.bash_history && history -c && exit

Source: https://askubuntu.com/questions/191999/how-to-clear-bash-history-completely#192001

Remove an entry from the PATH environment variable

To remove the /home/user/bin from your path, use

PATH=$(echo $PATH | sed -e 's;:\?/home/user/bin;;' -e 's;/home/user/bin:\?;;')

Source: https://www.linuxjournal.com/content/remove-path-your-path-variable-0

Troubleshooting

Terminal frozen

You might have accidentally pressed Ctrl + S which is bound to xoff (turning the terminal off). Press Ctrl + Q for xon.

Linux Mint gets stuck in a boot loop

My Nvidia card crashed the windowing system. I resolved this by removing all nvidia-* packages, but then my login screen got stuck in a boot loop. I managed to resolve it as suggested in the Mint Forums:

rm .Xauthority

Spotify is stuck in fullscreen

Solution:

Close Spotify, remove ~/.config/spotify/window_position.prefs file and start Spotify again.

For lazies:

rm ~/.config/spotify/window_position.prefs

Cannot connect to MySQL 5.7 Ubuntu 16.04 / Linux Mint 18

You have to reset the password as described here: http://askubuntu.com/questions/766900/mysql-dont-ask-for-root-password-when-installing

Change Ubuntu Bash language

Source: http://askubuntu.com/a/505424/415610

sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8

Windows VM in VirtualBox does not have an internet connection

Source: http://askubuntu.com/questions/363003/no-internet-connection-on-virtualbox-windows-7-as-guest-ubuntu-13-04-as-host

In the Network Settings, use Bridged Adapter with Promiscuous Mode for Allow VMs (not All). You might have to restart the VM.

HDMI output does not work

My laptop is a Dell Latitude E6520, equipped with an NVS4200 video card.

Ubuntu 15.10 (with Unity) is fine, but on Linux Mint 17.3 (with Cinnamon) the HDMI output is laggy, the screen is redrawn in a "lazy manner", i.e. just around the mouse cursor. To fix this, go to the Driver Manager and change the open-source xserver-xorg-video-nouveau driver to the closed source nVidia driver (e.g. nvidia-352). Unfortunately, this one does not work well with the VGA output, so I usually have reinstall the driver (and reboot) if I have to change from HDMI to VGA or vica versa.

Clock time in Windows is off in dual boot

http://askubuntu.com/questions/169376/clock-time-is-off-on-dual-boot

The simplest solution is to fix this in Linux:

Tell Linux to use 'local' time:

  1. edit /etc/default/rcS
  2. add or change the following section
# Set UTC=yes if your hardware clock is set to UTC (GMT)
UTC=no

Eclipse UI behaves weirdly

The GTK3 is not working properly with Eclipse. To fix this bug, you have to force Eclipse to use GTK2.

If you run Eclipse from command line, add the following line to your ~/.profile file:

export SWT_GTK3=0

If you want to run Eclipse from a file manager or any other way, add this to eclipse.ini:

--launcher.GTK_version
2

after the following lines:

-startup
[...].jar

If there are huge toolbars and titlebars, install jeeeyul/eclipse-themes and follow the Linux User Guide section of its wiki!

Pairing devices using the Logitech Unifying Receiver

See https://wiki.archlinux.org/index.php/Logitech_Unifying_Receiver

Usage:

$ ls -l /sys/class/hidraw/hidraw*/device/driver | awk -F/ '/receiver/{print $5}'

This will show the names of your receiver, for example hidraw0. Now switch off the device that you want to pair (if it was on) and execute your compiled program with the appropriate device as argument:

$ sudo ./pairing_tool /dev/hidraw0
The receiver is ready to pair a new device.
Switch your device on to pair it (you have thirty seconds to do so).

Perl/Python cannot find the appropriate locale

Problem: The following error message appears:

  • Perl:

    perl: warning: Setting locale failed.
    perl: warning: Please check that your locale settings:
    	LANGUAGE = "en_US:en",
    	LC_ALL = (unset),
    	LC_PAPER = "hu_HU.UTF-8",
    	LC_ADDRESS = "hu_HU.UTF-8",
    	LC_MONETARY = "hu_HU.UTF-8",
    	LC_NUMERIC = "hu_HU.UTF-8",
    	LC_TELEPHONE = "hu_HU.UTF-8",
    	LC_IDENTIFICATION = "hu_HU.UTF-8",
    	LC_MEASUREMENT = "hu_HU.UTF-8",
    	LC_TIME = "hu_HU.UTF-8",
    	LC_NAME = "hu_HU.UTF-8",
    	LANG = "en_US.UTF-8"
        are supported and installed on your system.
    perl: warning: Falling back to the standard locale ("C").
    
  • Python. Pretty much everything related to Python (pip, scripts) throws the following error:

      File "/usr/lib/python3.2/locale.py", line 541, in setlocale
        return _setlocale(category, locale)
    locale.Error: unsupported locale setting
    

Solution (based on http://askubuntu.com/questions/144235/locale-variables-have-no-effect-in-remote-shell-perl-warning-setting-locale-f): comment out the SendEnv LANG LC_* line in the client's /etc/ssh/ssh_config and (or?) the server's /etc/ssh/sshd_config file and reload the ssh configuration on the server (sudo service ssh reload). (See the comments in http://stackoverflow.com/a/2510548/3580502).

Another solution (based on https://askubuntu.com/questions/351905/redis-install-unsupported-locale-setting/351906):

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales

Shell

Change the default shell to Bash (the dpkg-reconfigure command is necessary to avoid the error message let: not found)

chsh -s /bin/bash
sudo dpkg-reconfigure bash
sudo ln -sf bash /bin/sh
Debian: use Bash instead of default Dash
# make /bin/sh symlink to bash instead of dash:
echo "dash dash/sh boolean false" | debconf-set-selections
DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash

(source)

Maximum argument length in Bash

The maximum length of the arguments if 2 MB for most Ubuntu-based Linux systems.

$ getconf ARG_MAX
2097152

"Also as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072)." (http://www.in-ulm.de/~mascheck/various/argmax/)

Ubuntu -- switch from Unity to GNOME Classic

sudo apt-get upgrade
sudo apt-get install -y gnome-session-fallback

(Source: http://askubuntu.com/questions/450294/how-to-switch-from-unity-to-gnome)

Ubuntu GNOME search

Disable automatic search and return to normal (Nemo- and traditional Nautilus-like) behaviour:

gsettings set org.gnome.nautilus.preferences enable-interactive-search true

Networking

Keyless SSH

Generate keys for keyless/passwordless SSH:

ssh-keygen -t rsa

To copy the public key (~/.ssh/id_rsa.pub) to server:

ssh-copy-id user@host

See also http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html.

Disable IPv6

Source: http://www.noobslab.com/2012/05/disable-ipv6-if-your-internet-is.html

Add the following lines to the /etc/sysctl.conf file:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

Run sudo sysctl -p or reboot.

Set static IP address

Edit the /etc/network/interfaces file:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.203.101
netmask 255.255.255.0
gateway 192.168.203.2
dns-nameservers 208.67.222.222 208.67.220.220

Run the following commands:

sudo ifdown eth0
sudo ifup eth0

If you are working in the Budapest University of Technology and Economics, you may want to use the local DNS servers instead of the OpenDNS ones:

dns-nameservers 152.66.208.1 152.66.208.7

Copy files over the network using terminal

The simplest command may be

scp -r /path/to/local/folder/to/copy [email protected]:/path/to/destination/on/remote/computer

Source: https://kx.cloudingenium.com/uncategorized/howto-copy-files-network-using-ubuntu-command-line/

Pro tips and tools for Linux users

Powerline

Powerline supports version control systems (e.g. SVN, Git) from the command line.

Install the fonts using the ./install.sh script. (If you would like to install the fonts manually, copy the selected fonts to the ~/.fonts directory and update the font cache wtih the fc-cache -vf ~/.fonts command).

Personally, I recommend the "DejaVu Sans Mono for Powerline" and the "Inconsolata for Powerline Medium" fonts.

Powerline configuration

The default appearance of powerline can be customized in the file ~/.config/powerline-shell/config.json. Example contents:

{
  "segments": [
    "newline",
    "virtual_env",
    "hostname",
    "ssh",
    "cwd",
    "git",
    "hg",
    "jobs",
    "newline",
    "root"
  ],
  "mode": "patched",
  "cwd": {
    "max_depth": 5
  },
  "vcs": {
    "show_symbol": "true"
  }
}

Custom color theme can be applied by adding the following line:

  "theme": "/path/to/theme.py",

Typing Unicode characters

Ctrl+Shift+U, type the character code and press Space. Character codes:

  • en dash (–): u2013
  • ellipsis (…): u2026
  • double quotes for Hungarian texts („”): u201e, u201d

See also http://en.wikipedia.org/wiki/International_variation_in_quotation_marks.

Ubuntu missing icons from context menu (e.g. Eclipse)

Clipboard manager (best Ditto alternative) CopyQ

Powerful Console alternative: Zsh

Cut videos with ffmpeg

Source: http://stackoverflow.com/questions/18444194/cutting-the-videos-based-on-start-and-end-time-using-ffmpeg

ffmpeg -i video.mp4 -ss 00:00:03 -t 00:00:08 -c copy -async 1 cut.mp4

⚠️ Note that the -c copy switch performs a quick but possibly inaccurate cut, remove it to perform full reeconding (which is much slower, but will be accurate).

Map audio/video tracks:

ffmpeg -i video.mkv -ss 00:59:35 -t 00:00:35 -map 0:v:0 -map 0:a:2 cut.mkv

Alternative with GUI: LosslessCut

Recode video so Facebook Messenger can play it:

ffmpeg -i video.mkv -filter:v scale=-2:720 out.mp4

Web UI for creating ffmpeg commands (requires registration as of Oct 2022): https://ffmpeg.guide/

Linux goodies

Videos

  • Linux sucks: a very good overview of the key strengths and weaknesses of the Linux ecosystem.

Mailing

To send e-mails from command line using an external SMTP server SWAKS was proven to be suitable for me. Requires no configuration, see the second (final) section of the post here for an example command.

Using Bose QuietControl (QC30) on Ubuntu

  • Edit the Bluetooth configuration file: sudo vim /etc/bluetooth/main.conf and set the ControllerMode key to ControllerMode = bredr.
  • Restart the Bluetooth controller sudo systemctl restart bluetooth.
  • On the earphones, hold the button for approx. 5 seconds to start pairing.
  • Start the Bluetooth application and pair the device.
  • Go to Sound, Output, Headphone and select High Fidelity Playback (A2DP Sink).
  • You'll likely to have to increase the volume a bit.

Tutorials for QC35 might also help: https://askubuntu.com/questions/833322/pair-bose-quietcomfort-35-with-ubuntu-over-bluetooth

Covnert from epoch to timestamps

On Linux, use date. On MacOS, use gdate.

Convert from epoch timestamps to GMT

(szarnyasg) I use this for the LDBC SNB implementations to convert from epoch milliseconds timestamps:

EPOCH_MILLI=1262300400000 EPOCH_SEC=$(($EPOCH_MILLI / 1000)) TZ=GMT date +%Y%m%d%H%M%S000 --date "@${EPOCH_SEC}" | head -c -1 | xsel -b

This copies the result (20091231230000000) to the clipboard. There is also a Bash function that does this:

function epoch () {
  EPOCH_MILLI=$1
  EPOCH_SEC=$(($EPOCH_MILLI / 1000))
  export TZ=GMT
  TIMESTAMP=`date +%Y%m%d%H%M%S000 --date @$EPOCH_SEC`
  echo $TIMESTAMP | head -c -1 | xsel -b # use wl-copy on Wayland
  echo $TIMESTAMP copied to clipboard
}

PostgreSQL-style timestamps

function epoch () {
  EPOCH_MILLI=$1
  EPOCH_SEC=$(($EPOCH_MILLI / 1000))
  export TZ=GMT
  TIMESTAMP=`date +"'%Y-%m-%dT%H:%M:%S.000+00:00'::timestamp" --date @$EPOCH_SEC`
  echo $TIMESTAMP | head -c -1 | xsel -b
  echo $TIMESTAMP copied to clipboard
}

SDKMAN troubleshooting

Problem: Other than vanilla sdk (which lists the available commands), every other command displays __sdk_: command not found:

$ sdk version
__sdk_: command not found

Solution: make sure that you are not running a login shell. Also make sure that the tr command points to the "translate" UNIX tool and is not overriden by an alias.

Generating thumbnails for PDFs

Make sure the pdftk and imagemagick packages are installed.

for f in *.pdf; do pdftk $f cat 1 output $f-p1 && convert -density 150 $f-p1 -quality 90 $f.png && rm $f-p1; done

If ImageMagick (convert) does not work, check out this StackOverflow answer.

Turn off screen from command line

xset dpms force off

Show Git branch in prompt

As suggested at Coderwall, add the following to your ~/.bashrc file:

parse_git_branch() {
     git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
}
export PS1="\u@\h \[\033[32m\]\w\[\033[33m\]\$(parse_git_branch)\[\033[00m\] $ "

Thunderbolt USB-C dock

Cable Matters only offers Windows and Mac drivers for its [Certified] Aluminum Thunderbolt 3 Docking Station with Dual 4K 60Hz Video and 60W Power Delivery hardware. However, there is no need for a specific driver on Linux: simply use Thunderbolt Tools.

$ sudo apt install thunderbolt-tools
...
$ tbtadm devices
0-1    Cable Matters Inc.    Thunderbolt 3 Docking Station    non-authorized    not in ACL
$ sudo tbtadm approve-all 
Found domain "/sys/bus/thunderbolt/devices/domain0"
Found child "/sys/bus/thunderbolt/devices/domain0/0-0/0-1"
Authorizing "/sys/bus/thunderbolt/devices/domain0/0-0/0-1"
Added to ACL
Authorized
$ tbtadm devices
0-1	Cable Matters Inc.	Thunderbolt 3 Docking Station	authorized	in ACL

Logitech MX Master 2 lags

As per the Arch wiki:

$ hcitool con
get handle
$ sudo hcitool lecup --handle $HANDLE --latency 0 --min 6 --max 8

VMware

VMware Workstation does not work

Use the latest version: on Ubuntu 16.04+, make sure you install VMware Workstation 12.5+.

VMware Workstation throws the following message: Before you can run VMware, several modules must be compiled and loaded into the running kernel. However, it throws the following error: Unable to start services. See log file .... The solution is to patch the kernel module of VMware, see this site for details: http://askubuntu.com/questions/449629/error-installing-vmware-workstation-on-ubuntu/449630#449630

For Ubuntu 18.04+ systems, the solution is described at https://communities.vmware.com/thread/609330. The solution here is based on the resources at https://askubuntu.com/a/1052104/415610 and https://github.com/mkubecek/vmware-host-modules.

VMware Tools prerequisites

If you are running Mint as a VMware virtual machine, the linux-headers packages are required to install VMware Tools.

sudo apt-get install -y gcc make
sudo apt-get install -y linux-headers-$(uname -r) linux-headers-generic
cd /lib/modules/$(uname -r)/build/include/linux
sudo ln -s ../generated/utsrelease.h
sudo ln -s ../generated/autoconf.h
sudo ln -s ../generated/uapi/linux/version.h

Install VMware Tools

cd ~
tar xf /media/szarnyasg/VMware\ Tools/VMwareTools-*.tar.gz
cd vmware-tools-distrib/
sudo ./vmware-install.pl

VMware Tools troubleshooting

  • Problem: "VMware Tools installation cannot be started manually while Easy Install is in progress"
  • Solution: go to the virtual machine settings, Hardware tab, CD/DVD and Floppy devices and set them to Use a pyhsical drive.

Windows scroll stuck

Windows 7/8.1 machine running in VMware Workstation 9.0/10.0 scrolls left endlessly: press Ctrl, Alt and Shift at once a couple of times, this will cancel the scroll. Unfortunately, this solution does not seem to work now (Workstation 12.0).

Using dual monitors

In theory, VMware Workstation is capable of using multiple displays. Shut down the virtual machine, go to the Virtual Machine Settings, Displays menu and choose Specify settings for monitors. Be very careful with this option as it may not work with any outer displays (e.g. projectors).

If it works, you can use PowerPoint's presentation view in VMware's Unity mode.

User interface configurations

Change keyboard layout

Depending on the window manager:

  • Cinnamon: Preferences | Region & Language | Layouts
  • Xfce: Right click on the taskbar, go to Panel | Add new items | Keyboard layouts. You can change the hotkeys by right clicking the keyboard layout flag and choosing Properties.

Compress with zstd

Older tar versions such as the one in Ubuntu 18.04 do not recognize the ZSTD format -- you have to provide either the zstd or the zstdmt (multi-threaded) binary as the compress program:

tar --use-compress-program=zstdmt -cf archive.tar.zst path/to/target/files/or/directories/

An alternative syntax, available in distributions like Ubuntu 20.04+ and Fedora, is:

tar --zstd -cf archive.tar.zst path/to/target/files/or/directories/

For multi-threaded execution, use the ZSTD_NBTHREADS environment variable:

export ZSTD_NBTHREADS=`nproc`

Decompress zstd tar archives

Older tar versions such as the one in Ubuntu 18.04 do not recognize the ZSTD format -- you have to provide the zstd binary:

tar -I zstd -xf <filename.zst>

History handling

You might want to put this in your ~/.bashrc:

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=100000
HISTFILESIZE=200000

Close unresponsive SSH connection

You can use tilde dot to exit from an unresponsive SSH connection. Hit Enter, ~, ..

Debian/Ubuntu 20.04+: apt install prompts

Problem: apt install prompts for timezone data and then times out.

debconf: falling back to frontend: Readline
Configuring tzdata
------------------
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
  1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
  2. America     5. Arctic     8. Europe    11. SystemV
  3. Antarctica  6. Asia       9. Indian    12. US

Solution:

export DEBIAN_FRONTEND=noninteractive

Ubuntu 22.04.2+: apt prompts to restart, vol 1: services

Problem: apt upgrade and apt install may prompt for restarting services, causing it to hang in non-interactive environments.

Scanning processes...
Scanning candidates...
Scanning linux images...

Running kernel seems to be up-to-date.

Restarting services...

Solution:

Set the following environment variable:

export NEEDRESTART_SUSPEND=a

If this does not work for some reason, a less elegant solution is to remove the needrestart package. (Obviously this should only be done in Docker containers, CI and test machines, etc.)

sudo apt purge -y needrestart

Ubuntu 22.04+: apt prompts to restart, vol 2: services again

Ubuntu 22.04 wants to restart libraries due to "Daemons using outdated libraries". Setting the NEEDRESTART_SUSPEND environment variable does not help.

Run the following... ⚠️ Be careful if you are running this on your daily driver machine:

export DEBIAN_FRONTEND=noninteractive
sudo sed -i 's/#\$nrconf{restart} = '"'"'i'"'"';/\$nrconf{restart} = '"'"'a'"'"';/g' /etc/needrestart/needrestart.conf
sudo apt-get update
sudo apt-get upgrade -y

(Related posts on StackOverflow and AskUbuntu)

Ubuntu 22.04+: apt prompts to restart, vol 3: due to kernel upgrades

sudo sed -i "s/#\$nrconf{kernelhints} = -1;/\$nrconf{kernelhints} = -1;/g" /etc/needrestart/needrestart.conf

Splitting files

Splitting e.g. a .tar.zst file into 32GiB chunks:

export FILE=graph500-30.tar.zst
split -d -a3 -b 32768m ${FILE} ${FILE}.

A concrete example:

split -d -a3 -b 32768m graph500-30.tar.zst graph500-30.tar.zst.

To recombine & decompress them, run:

export FILE=graph500-30.tar.zst
cat ${FILE}.* | tar -xv --use-compress-program=unzstd

feymr10 missing

  • Error:

    kpathsea: Running mktexmf feymr10
    I can't find file `feymr10'.
  • Solution: Install the texlive-collection-fontsrecommended package.

Bash workflow improvements

There is a nice Bash workflow outline on Hacker News (Web Archive link).

Find files with a missing newline ('\n') at the end of the file

find . -type f -name '*.md' | xargs pcregrep -LMr '\n\Z'
Clone this wiki locally