Startup performance is noticeably slow

[maximbaz]

Hey @romkatv,

You have sparkled my interest in spaceship-prompt/spaceship-prompt#734, so first of all thanks for that 😉

I'm very interested to give this prompt a try, so I installed it using antigen and followed the initial wizard to configure the prompt.

While the experience in general feels smooth, I noticed that the startup time is noticeably slower than what I'm used to with my async version of spaceship. I can't give you any numbers, and to be honest I didn't dig deeply into powerlevel configuration to see what can be optimized (I only tried to disable a bunch of segments to no avail), but I did ask a friend for his perception (who I know also uses my async fork) and he confirms that the startup feels slower.

To clarify, by startup I mean the period of time between when I launch a new terminal instance and when the prompt is rendered so I can start typing my commands.

So maybe to start the discussion with something, have you heard a similar feedback before? Do you have any idea what might be the cause? Have you specifically attempted to optimize the startup time, or it hasn't been your focus so far?

I'm happy to assist with whatever I can, running experiments, performing benchmarks (how?), etc. At this point I dont have a powerlevel configuration I like (so we can start with some default and use it as an example).

I'm on Arch Linux using kitty terminal, zsh 5.7.1.

To me personally it's very important to get startup time to a minimum, I launch new terminal windows all the time and it feels annoying when I can notice the "loading". This is a lot more annoying than watching slow sections load (as long as they are asynchronous and not blocking my typing), so if I had to choose, I'd rather have slower async sections but faster startup.

🙂

[romkatv]

See https://github.com/romkatv/powerlevel10k/blob/master/README.md#is-powerlevel10k-fast-to-load.

I've done nothing to optimize startup time because I personally don't care. My workflow is unaffected by zsh startup latency. It may be possible to significantly reduce startup time but it's not something I want to do. If you send a PR that improves startup latency without too many adverse side effects, I'll be happy to incorporate it.

[maximbaz]

Oh, can't believe I missed that section. Thanks for the fast reply :)

[romkatv]

To measure startup latency with your own config:

time (repeat 100 zsh -dfis <<< 'source ~/.p10k.zsh; source ~/powerlevel10k/powerlevel10k.zsh-theme')

Note that latency depends on the environment in which you measure it. E.g., startup latency is higher in a git repo. If you have nvm or kubernetes installed, it'll be higher still. Powerlevel10k can perform a lot of computation during initialization in order to save on prompt latency in the long run. I won't be surprised if you can get startup latency over 1s if you install everything that Powerlevel10k can possibly use. Interactive prompt will still be fast though.

[maximbaz]

I see, in my case I was comparing a relative startup time difference of both prompts launching in the home dir, with the same set of tools installed.

But it makes sense now, powerlevel10k is targeting to optimize the performance of long-running sessions by sacrificing a bit of startup time, and async spaceship has the opposite goal.

Thanks again 🙂

[romkatv]

But it makes sense now, powerlevel10k is targeting to optimize the performance of long-running sessions by sacrificing a bit of startup time, and async spaceship has the opposite goal.

I don't think there is an unavoidable trade off between these goals. Maybe sometimes you have to choose one over the other but in many cases you can have both. Powerlevel10k is slow to start and fast to use simply because I optimized for the latter without caring for the former. Things you don't care about tend to be slow but it doesn't mean they cannot be made fast.

[eddyg]

Out of curiosity, any idea if automatically running zcompile on all the stuff in the internal directory (to generate .zwc files) help startup times?

[maximbaz]

I don't think there is an unavoidable trade off between these goals. Maybe sometimes you have to choose one over the other but in many cases you can have both. Powerlevel10k is slow to start and fast to use simply because I optimized for the latter without caring for the former. Things you don't care about tend to be slow but it doesn't mean they cannot be made fast.

Agreed. If you want, we can re-open the issue so that it can attract others interested in this 😉

[maximbaz]

Out of curiosity, any idea if automatically running zcompile on all the stuff in the internal directory (to generate .zwc files) help startup times?

Good point, although in my specific case, I believe antigen is doing that already.

[romkatv]

Out of curiosity, any idea if automatically running zcompile on all the stuff in the internal directory (to generate .zwc files) help startup times?

Last time I checked, it had no effect. There isn't that much source code and parsing is relatively fast.

If I recall correctly, about 60ms is spent on starting gitstatusd and 70ms on converting POWERLEVEL9K configuration options into chunks of precomputed code. The rest of the time is spent calling external programs such as kubectl and dotnet.

Note that p10k is also async. This fact isn't advertised on the homepage because it's almost never relevant. However, if your machine is very slow and you end up in a very large git repo, you'll notice it. The async Git UI is superb. It's a shame almost no p10k users have ever seen it.

[romkatv]

Agreed. If you want, we can re-open the issue so that it can attract others interested in this

I've an idea that might speed up startup quite a bit without too many code changes. I'll give it a shot some time next week. Will report here whether it works or not.

[sinetoami]

Interesting seeing this thread here. I feel the same thing as @maximbaz feels. Actually that's the one thing which tiny bothering me. Zinc prompt, by robobenklein, have a great startup performance and it's fast like powerlevel10k. But haven't some features that a like... and powerlevel10k is waaay mature and stable. For someone that constantly launch new terminal instances or tmux panes(and have a slow machine) i think this point make difference.

[romkatv]

I've made p10k startup about 2.5-3 times faster. It's still not super fast but certainly better than before. Please give it a try and let me know if you see an improvement and whether startup latency is good enough now.

Here are some benchmarks. All runs are from my desktop running Ubuntu. Current directory is powerlevel10k Git repo. 100 runs per benchmarks. Reported numbers are for a single run.

Baseline: 5.01ms.

time (repeat 100 zsh -dfis <<<'true')

Powerlevel9k with default settings: 171ms

time (repeat 100 zsh -dfis <<< 'source ~/powerlevel9k/powerlevel9k.zsh-theme')

Powerlevel10k with default settings: 59.7ms

With these settings p10k looks the same as p9k.

time (repeat 100 zsh -dfis <<< 'source ~/powerlevel10k/powerlevel10k.zsh-theme')

Powerlevel10k with ~/.p10k.zsh: 78.6ms

~/.p10k.zsh was generated by p10k configure. The choices don't seem to have much of an effect on startup performance. The reported number is from what I suppose is the slowest configuration with the maximum number of bells and whistles. Here's the config's header:

# Generated by Powerlevel10k configuration wizard on 2019-10-02 at 15:42 CEST.
# Based on romkatv/powerlevel10k/config/p10k-classic.zsh, checksum 47547.
# Wizard options: nerdfont-complete + powerline, small icons, classic, dark, time,
# slanted separators, blurred heads, blurred tails, 2 lines, dotted, right frame,
# sparse, many icons, fluent.

time (repeat 100 zsh -dfis <<< 'source ~/.p10k.zsh; source ~/powerlevel10k/powerlevel10k.zsh-theme') 

I believe the last benchmark is the most important. For comparison, the same benchmark before my optimizations reports 231ms startup latency. Now it's 2.9 times faster.

[romkatv]

P.S.

After zcompiling all sources the last benchmark has gotten 15% faster (68.1 ms). Here's the command I used to zcompile.

for f in ~/powerlevel10k/{*.zsh-theme,**/*.zsh} ~/.p10k.zsh; do zcompile $f; done

If you care about startup latency, update powerlevel10k and zcompile its sources and your ~/.p10k.zsh. It'll be over 3x faster to load than before.

[eddyg]

Thanks for the continued work on making powerlevel10k awesome, Roman!

Any reason not to just have the configuration wizard run your command above to compile everything? According to the zsh docs:

If a file named file.zwc is found, is newer than file, and is the compiled form (created with the zcompile builtin) of file, then commands are read from that file instead of file.

So even if somebody later modifies a file that the wizard compiled, the compiled version would be properly ignored.

[romkatv]

Any reason not to just have the configuration wizard run your command above to compile everything?

I was thinking the same thing.

It's certainly possible. One non-trivial aspect is handling different combinations of permissions and file/directory ownership. Not too difficult. The thing that is stopping me from implementing this right now is potential interaction with plugin managers.

I'll wait for the fallout from my recent optimizations to subside and then add zcompile once bug reports stop pouring in.

[romkatv]

FYI: I've reverted my optimizations as they were causing some issues and I don't have the time to debug them right now. Will debug and roll forward later this week.

[romkatv]

I've fixed bugs in my previous implementation and rolled it forward. I had to disable some optimizations for zsh < 5.4 due to bugs in quoting that I don't want to write workarounds for. I implemented a few extra optimizations.

Now p10k loads in 37 ms on my machine (6x improvement). Not instant but decently fast. This time includes starting zsh, sourcing ~/p10k.zsh, loading the theme and rendering the first prompt. Note that the first prompt is complete, unlike with some other themes that will give you just the current directory and then repaint prompt asynchronously when extra data becomes available.

I think it's possible to shave off another 10 ms or so but more than that will be difficult. Unless someone complains loudly enough, I'll leave the code be.

I also might have to revert the change if I broke something again. The changes are quite complex, so there is non-trivial risk of breakage.

I would appreciate if someone on this issue could update powerlevel10k and verify that loading has gotten faster.

[sinetoami]

@romkatv, let me ask you a newbie question (sorry for my bad english, btw). If you split up all segments in their own files, and then load only these which want to use (in this case, not that all them)... this would improve performance at some level? Or would it be the same as already is?

[romkatv]

@sinetoami Yes, this kind of refactoring can improve performance. By my estimate you can save about 0.5 ms for the typical config if you make the right decisions about what to factor out and what to keep in the main file.

By the way, this sort of guessing game is a terribly inefficient way to optimize code. Here's a better way:

1. Write a benchmark.
2. Run the benchmark. If it's fast enough, you are done.
3. Profile the code.
4. Optimize the main bottleneck.
5. Go to 2.

[sinetoami]

I would appreciate if someone on this issue could update powerlevel10k and verify that loading has gotten faster.

Yes, for me it's got faster! 👍

@sinetoami Yes, this kind of refactoring can improve performance. By my estimate you can save about 0.5 ms for the typical config if you make the right decisions about what to factor out and what to keep in the main file.

I want to give a try to this in a fork. Probably I will have some headaches trying to call the parts of segments from a main file... But thanks for your advice about this 👍 .

EDIT
I notice you are a way technical than me. But, I don't know if you will "feel" how fast zinc prompt is on load, but I've made a video intend to show you. Ignore the issues, please.

[romkatv]

Yes, for me it's got faster! 👍

Thanks for the confirmation!

@sinetoami Yes, this kind of refactoring can improve performance. By my estimate you can save about 0.5 ms for the typical config if you make the right decisions about what to factor out and what to keep in the main file.

I want to give a try to this in a fork. Probably I will have some headaches trying to call the parts of segments from a main file... But thanks for your advice about this 👍 .

Didn't expect you to become enthusiastic about potentially reducing zsh startup latency by 0.5 ms. But what the hell, if that's the sort of thing you want to do, who am I to judge?

EDIT
I notice you are a way technical than me. But, I don't know if you will "feel" how fast zinc prompt is on load, but I've made a video intend to show you. Ignore the issues, please.

I don't understand what's going on in that screencast. If you are saying that loading p10k is noticeably slower than loading zinc on your machine, it would be more useful if you describe your machine (OS, CPU, zsh version) and attach benchmark results for both themes, with their full configs, so that I (and everyone else) can replicate these results.

For example, to benchmark p10k with your config:

time (repeat 100 zsh -dfis <<< 'source ~/.p10k-pure.zsh; source ~/powerlevel10k/powerlevel10k.zsh-theme') 

Run this command several times to avoid contamination due to cold IO caches and the like. Please don't use zplugin in these benchmarks.

If the latency is high, I can debug it provided that you provide the information I mentioned above.

[sinetoami]

It was very interesting to notice that zinc prompt visually load really faster and smoother than powerlevel10k on these benchmarks what I did. However, the gitstatus on powerlevel10k it's more stable and faster than zinc. I'm not sure if it had some significant difference, but I actually think that zinc prompt load faster than powerlevel10k.

OS: Arch Linux 5.3.1-arch1
Zsh Version: 5.7.1
CPU: Intel Core i5 3210M

zinc - first time: 23.13s user 13.29s system 98% cpu 36.809 total
zinc - second time: 23.19s user 13.75s system 98% cpu 37.521 total
zinc - third time: 23.16s user 13.43s system 98% cpu 36.972 total

p10k - first time: 27.07s user 12.35s system 117% cpu 33.437 total
p10k - second time: 26.87s user 12.48s system 115% cpu 33.977 total
p10k - third time: 27.15s user 12.59s system 114% cpu 34.753 total

[romkatv]

@sinetoami Your benchmark results indicate that p10k loads slightly faster than zinc.

Now, since the results of this benchmark don't match your experience when using zsh with your real zsh config files, it means there is something in them that makes it different from the benchmark. Something that causes p10k load slower than it does in the benchmark. You can try to figure out what it is by removing parts of your config and checking when loading time improves. The first thing I would suggest is to load p10k without zplugin. Simply clone powerlevel10k repository and put source ~/.p10k-pure.zsh and source ~/powerlevel10k/powerlevel10k.zsh-theme in your ~/.zshrc.

[romkatv]

I've watched your screencast a few more times and I think I understand what's going on there. I see that your zsh isn't very fast to load but I cannot say whether there is difference in latency between zinc and p10k.

Is it possible that you perceive difference in latency because zinc is printing an empty line when it starts up? Maybe you can tell it not to print it? Or, alternatively, you can print the empty line when using p10k. I don't know at which point zinc prints the empty line so you might want to try several things in order to replicate the same delays.

1. Print an empty line from ~/.zshrc.

Add echo somewhere in ~/.zshrc. Perhaps before p10k zplugin ice.

2. Print an empty line right before sourcing p10k.

Add echo to p10k zplugin ice.

3. Print an empty line right before p10k initialization.

Add print-line() { echo; add-zsh-hook -D precmd print-line }; add-zsh-hook precmd print-line to the p10k zplugin ice.

---

I've attempted to check whether zplugin might be making things slower than they should be. The short answer is no. When powerlevel10k is the only loaded plugin, sourcing it directly is 11ms faster than loading it with zplugin. It's expected to be faster because it must take some time to load zplugin itself. The extra 11ms added by zplugin isn't much, and it might pay off if zplugin reduces the total loading time when using many plugins (I haven't tried to verify this).

I've recorded a screencast showing what I've measured and how:

The screencast starts with the following docker command:

docker run -e LANG=C.UTF-8 -e TERM -it --rm debian:buster bash -uec '
  apt update && apt install -y curl git zsh sudo nano
  useradd -ms /bin/zsh me
  sudo -u me sh -c "$(curl -fsSL \
    https://raw.githubusercontent.com/zdharma/zplugin/master/doc/install.sh)"
  >>~me/.zshrc echo "zplugin ice atload\"source config/p10k-pure.zsh\" lucid
zplugin light romkatv/powerlevel10k
# source ~/.zplugin/plugins/romkatv---powerlevel10k/config/p10k-pure.zsh
# source ~/.zplugin/plugins/romkatv---powerlevel10k/powerlevel10k.zsh-theme"
  exec su - me'

It installs zsh and zplugin on Debian and logs in as a user with the following ~/.zshrc:

### Added by Zplugin's installer
source '/home/me/.zplugin/bin/zplugin.zsh'
autoload -Uz _zplugin
(( ${+_comps} )) && _comps[zplugin]=_zplugin
### End of Zplugin installer's chunk

zplugin ice atload"source config/p10k-pure.zsh" lucid
zplugin light romkatv/powerlevel10k

# source ~/.zplugin/plugins/romkatv---powerlevel10k/config/p10k-pure.zsh
# source ~/.zplugin/plugins/romkatv---powerlevel10k/powerlevel10k.zsh-theme

Note that powerlevel10k and its config are loaded with zplugin.

After logging in, the current directory is changed to ~/.zplugin/plugins/romkatv---powerlevel10k because it's more interesting to observe what happens next if you are in a Git repository. You can see that true is instant while exec zsh is almost instant.

The following command is used to measure zsh startup latency:

time ( repeat 100 zsh -is <<< '' )

It repeatedly starts interacive zsh sessions and exits them once the first prompt renders. The command takes 5.115s (51ms per run).

Then ~/.zshrc is edited so that powerlevel10k and its config are sourced directly, without zplugin. Here's the new ~/.zshrc.

source ~/.zplugin/plugins/romkatv---powerlevel10k/config/p10k-pure.zsh
source ~/.zplugin/plugins/romkatv---powerlevel10k/powerlevel10k.zsh-theme

After starting a new zsh session everything proceeds as before. This time the benchmark completes in 4.016s (40ms per run).

Thanks to docker, it should be relatively straightforward to reproduce this screencast for everyone who wishes to do so.

[maximbaz]

@romkatv I've just tested and it is now significantly better, nice work! I haven't performed any measurements or comparisons against spaceship, but the terminal startup now feels sufficiently fast that I don't feel any inconvenience. I will give it a longer try and let you know if I spot anything weird.

[romkatv]

@romkatv I've just tested and it is now significantly better, nice work!

Thanks for verifying! With two confirmations and no complaints I consider this issue resolved.

(@sinetoami We can still continue the discussion here if you like.)

I haven't performed any measurements or comparisons against spaceship

I have. Your Spaceship fork loads 15-20ms faster. To convert it to relative zsh startup latency improvement you need to know your total zsh startup latency. It's easy to measure.

time (repeat 100 zsh -is </dev/null)

Look at total at the end of the output and divide it by 100. On my system total is 18.7 seconds, which means it takes 187ms to start zsh with all my user configs. I think this is considered high but as I mentioned earlier I don't care.

With Spaceship (async) I get 170ms, meaning that zsh starts 10% faster. Your numbers may be different, and the speedup may or may not matter to you.

but the terminal startup now feels sufficiently fast that I don't feel any inconvenience. I will give it a longer try and let you know if I spot anything weird.

Awesome. Do let me know if anything is broken, missing, inconvenient or just feels off. High-quality bug reports and feature requests are gold to me. They are the only means I have for discovering what users want.

[romkatv]

This can happen if you setopt prompt_sp prompt_cr after sourcing powerlevel10k. Try sourcing powerlevel10k at the very end of zshrc. Does it help?

It doesn't (maybe these are default options?), however what does help is putting this in the very top of my .zshrc: unsetopt prompt_sp prompt_cr

What if you add these to ~/.cache/p10k-instant-prompt-*.zsh file?

Used to have it there but then moved to https://github.com/romkatv/powerlevel10k/blob/master/powerlevel10k.zsh-theme because it reduces the amount of code where users can fuck up the options. I think antigen is the culprit. It reverts options after sourcing https://github.com/romkatv/powerlevel10k/blob/master/powerlevel10k.zsh-theme. I've added unsetopt back to p10k-instant-prompt. The more the merrier. What I really need is to unset these options after zshrc finishes sourcing but I don't think there is a hook for that.

Please update powerlevel10k, remove unsetopt prompt_sp prompt_cr that you've added earlier to zshrc and try again. Let me know if it works.

[romkatv]

Could you describe in more detail what happens? What do you do and what do you observe?

Happens a few times when I open tmux panes in any direction and with tiling window terminals. I think that's have relation with prompt resizing...

Could you explain what is happening in that gif? At the beginning there is one big terminal window. Then you press something and the window splits in two. What is the relationship between the original window and the two new windows? Do I understand it correctly that the right window of the two is the original window that has been resized, and the left window is a new shell?

Can you reproduce this problem if you update powerlevel10k?

Are you using a plugin manager?

[maximbaz]

Please update powerlevel10k, remove unsetopt prompt_sp prompt_cr that you've added earlier to zshrc and try again. Let me know if it works.

I was about to report that I still reproduce the issue, but turns out I had to manually rm -rf ~/.cache/p10-* and restart terminal to let p10k regenerate the files. After this everything was perfect.

In general, I think we need some better story around regenerating those files. For example, if I modify POWERLEVEL9K_LEFT_PROMPT_ELEMENTS and then launch a new terminal, I will see that cached instant prompt still uses the old POWERLEVEL9K_LEFT_PROMPT_ELEMENTS definition.

I would expect this:

• I modify POWERLEVEL9K_LEFT_PROMPT_ELEMENTS
• Next time I launch a new terminal I see inconsistency, but in the very end p10k regenerates cache
• Next time I launch a new terminal everything is consistent with my current settings

[romkatv]

Please update powerlevel10k, remove unsetopt prompt_sp prompt_cr that you've added earlier to zshrc and try again. Let me know if it works.

I was about to report that I still reproduce the issue, but turns out I had to manually rm -rf ~/.cache/p10-* and restart terminal to let p10k regenerate the files. After this everything was perfect.

Sorry about that. I forgot to change one more line in my commit. Fixed now.

In general, I think we need some better story around regenerating those files. For example, if I modify POWERLEVEL9K_LEFT_PROMPT_ELEMENTS and then launch a new terminal, I will see that cached instant prompt still uses the old POWERLEVEL9K_LEFT_PROMPT_ELEMENTS definition.

I would expect this:

• I modify POWERLEVEL9K_LEFT_PROMPT_ELEMENTS
• Next time I launch a new terminal I see inconsistency, but in the very end p10k regenerates cache
• Next time I launch a new terminal everything is consistent with my current settings

That's how it works. Do you see something else?

[maximbaz]

Just updated again, that's how it works now with the latest master indeed! 🙂

At a glance everything is perfect now! I'll report if I see anything weird 😉

[romkatv]

Just updated again, that's how it works now with the latest master indeed!

It should've worked this way even before the last commit. There might be bugs though. The interaction between different caches and zsh instances that run simultaneously but with different settings is quite complex. I did think about all this while writing code and I also tested it a bit, so in theory it's supposed to work reasonably well even in tricky cases.

At a glance everything is perfect now! I'll report if I see anything weird

Thanks!

[sinetoami]

Could you explain what is happening in that gif? At the beginning there is one big terminal window. Then you press something and the window splits in two. What is the relationship between the original window and the two new windows? Do I understand it correctly that the right window of the two is the original window that has been resized, and the left window is a new shell?

I will try. So, I'm i3-gaps user. When I press a shortcut the i3 opens a new terminal. What I wanted to show you with the gif was the % char at the top of two terminals on the right side and a one on the left side. Can you see that? My intention open those terminals like that was trying to confirm mine thought about prompt resizing. But I'm not sure about that, by the way.

Can you reproduce this problem if you update powerlevel10k?

Still.

Are you using a plugin manager?

Yes.

PS.: If you don't understand me, feel free to say. My English is terrible I know that, but I can try to figure it out a better way to explain. 👍

[romkatv]

@sinetoami Please update powerlevel10k and restart zsh. It'll probably show you an error. Apply the recommended solution. Does it work now?

[romkatv]

FYI: The configuration wizard now enables instant prompt automatically. In addition to adding the stanza at the top of ~/.zshrc (which you already have), it also adds this bit at the bottom:

(( ! ${+functions[p10k-instant-prompt-finalize]} )) || p10k-instant-prompt-finalize

It's not strictly necessary but it allows me to fix some of the issues that people can run into. For example, the issue @sinetoami is having.

---

One user has reported that they cannot close zsh with Ctrl-D when instant prompt is enabled. It's normal that Ctrl-D does nothing if you press it before everything is fully initialized (instant prompt doesn't change this), but this user says that Ctrl-D stops working altogether. I'm unable to reproduce this but I'll keep trying. Please try it too and let me know if it works for you or not.

[maximbaz]

but this user says that Ctrl-D stops working altogether. I'm unable to reproduce this but I'll keep trying. Please try it too and let me know if it works for you or not.

I'm using Ctrl-D all the time and it works for me flawlessly.

[sinetoami]

@romkatv I won't like to say that, but the update made it even worse than before 😕.
Before there was only a % at the top of the prompt in some opened terminals. Now I have that in every opened terminal and, plus, when I split tmux panes or open new terminals they are totally messed up the prompt (example: duplicate the first line many times, break the first line, don't resize the prompt properly). Even running p10k without plugin manager, any plugin and extra custom configurations had the same result. Maybe I'm doing something wrong because @maximbaz and you don't have any similar issue now... but I can wait for some updates...

[maximbaz]

Could you try to provide a minimal possible .zshrc that reproduces the issue? Ideally start in a terminal without tmux with empty .zshrc, install p10k, run config wizard and see if you reproduce the issue then. If not, try to add your own configs incrementally until you find what settings messes up your prompt.

[sinetoami]

@maximbaz yes, yes. I actually did that. Starting with the .zshrc completely naked, installed p10k manually, used p10k configure to generate the custom configs and so on.

[romkatv]

@sinetoami What's the smallest number of actions that you need to take in order to get any kind of brokenness in your prompt (having a % counts as brokenness)? Is it enough to simply open a terminal? If not, then what do you need to do to have brokenness? You don't need to catalog all kinds of brokenness and display all the crazy action in a gif. Just find the shortest possible chain of actions that always leads to the same broken results and describe it precisely.

Once you find that shortest possible chain of action that shows brokenness, figure out the shortest possible zshrc config that will still demonstrate the issue when you perform those actions. Can you reproduce the issue with this config?

if [[ -r "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh" ]]; then
  source "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
fi
POWERLEVEL9K_DISABLE_CONFIGURATION_WIZARD=true
source ~/powerlevel10k/powerlevel10k.zsh-theme
(( ! ${+functions[p10k-instant-prompt-finalize]} )) || p10k-instant-prompt-finalize

If not, what do you need to add to the config to get brokenness?

Is this something you could try?

but I can wait for some updates...

No updates are coming until I can reproduce this. You are the only one who can help with it.

[sinetoami]

@romkatv I will try to figure it out this by myself for a while before relate here what you need. I will be back with a better answer.

[romkatv]

@sinetoami I encourage you to post your findings as you go. I might be able to guess the culprit before you find the smallest possible test case.

[romkatv]

@sinetoami I took your public dotfiles and tried to reproduce the issue. Here's my command:

docker run -e LANG=en_US.utf8 -e TERM -it --rm archlinux/base bash -uexc '
  cd
  pacman -Syu --noconfirm
  pacman -Sy --noconfirm zsh git curl gawk nano
  sh -c "$(curl -fsSL https://raw.githubusercontent.com/zdharma/zplugin/master/doc/install.sh)"
  git clone https://github.com/sinetoami/dotfiles.git
  git clone https://github.com/asdf-vm/asdf.git ~/.asdf
  shopt -s dotglob
  cp -r ~/dotfiles/zsh/* ~/
  ln -s /root /home/snio
  cat >~/instant-zsh.zsh <<END
instant-zsh-pre() {
  if [[ -r "\${XDG_CACHE_HOME:-\$HOME/.cache}/p10k-instant-prompt-\${(%):-%n}.zsh" ]]; then
    source "\${XDG_CACHE_HOME:-\$HOME/.cache}/p10k-instant-prompt-\${(%):-%n}.zsh"
  fi
}
instant-zsh-post() {
  (( ! \${+functions[p10k-instant-prompt-finalize]} )) || p10k-instant-prompt-finalize
}
END
  cat >~/.zshenv <<END
function keychain() false
function ruby() false
END
  exec zsh'

It works fine, so it's not zsh configs that break stuff.

I then went ahead and tried splitting terminal windows in different terminals. I managed to reproduce something that looks quite similar to your issue by opening a tab in Tilix and then clicking "Add terminal right”. It shrinks the current terminal to 50% of its width and creates another terminal to the right of it with the same width. The new terminal has broken prompt.

After debugging this I found out what appears to be a bug somewhere. If I add the following lines to the very top of zshrc:

echo $COLUMNS
/bin/true
echo $COLUMNS

I'll get this output when clicking "Add terminal right”.

80
40

40 is the correct answer, 80 isn't. I don't know if it's a bug in zsh or Tilix and what the proper workaround is. For now I added a call to /bin/true as it seems to help. Please update powerlevel10k and see if it helps.

[sinetoami]

Please update powerlevel10k and see if it helps.

Still the same behavior.

It works fine, so it's not zsh configs that break stuff.
... I don't know if it's a bug in zsh or Tilix...

Interesting that you noticed that. I repeated these steps with gnome-terminal, xfce4-terminal and urxvt-unicode. Both gnome-terminal and xfce4-terminal have the same bugs. But urxvt-unicode didn't. And here the interesting thing: my urxvt-unicode doesn't have NerdFont (or similar) support but even the prompt resizing works perfectly and never breaks or have any instability about powerlevel10k with instante prompt feature active. When I disable the instant prompt feature the powerlevel10k works equal perfectly with all terminal emulators.

But here the actions that show brokenness for me. I'm a xfce4-terminal and tmux user. Inside tmux I have some shortcuts that split up the host terminal in panes with horizontal or vertical directions. Just using xfce4-terminal without tmux I have the same behavior, the difference is without tmux the WM open terminals and dividing them with equally width or height. For example, I have a terminal and I decided to open a new one... after that, if my new terminal is opened on my right hand the left terminal breaks the prompt. And the new one prints a % on top of the prompt. And when I close this last terminal opened, the old one breaks one more time in a random way.

I don't know if what I said is sufficient for your analysis. Let me know if don't.

Update:
Using urxvt-unicode more intensively today and I noticed that sometimes was printed % on top of the prompt too. But the prompt resizing doesn't actually break in any those times.

[romkatv]

@sinetoami Thanks for the info. I think I have the complete picture now.

tl;dr: There are two low level bugs that break zsh prompt. One triggers when you resize your terminal window. The other -- when your prompt is printed very quickly after a terminal is created. None of these are specific to powerlevel10k and can be reproduced with plain zsh with very simple prompt.

Let's first deal with the breakage of the left (original) terminal when you split a terminal window. What happens here is that the original terminal gets shrunk. That is, its width gets reduced. There are two ways terminals handle resizing. The first is to simply erase a portion of the terminal window when shrinking, and add empty space when expanding. The second is to reflow all text. Reflowing preserves all content in the terminal window but it will wrap around differently because of the different window width. Here's a demonstration of both.

1. Type this:

PROMPT='${(pl.$COLUMNS..-.)}'$'\n> ' zsh -f --prompt-subst

2. Type reset and press enter a few times. Observe the content of your terminal.
3. Shrink the terminal window (reduce its width). Observe the content of your terminal.
4. Expand the terminal window (increase its width). Observe the content of your terminal.

This is what I get with urxvt, which uses the first method of handling terminal resizing.

Initial (after step 2):

Shrunk (after step 3):

Expanded (after step 4):

Note that the last prompt is always correct. This is because urxvt sends SIGWINCH to the shell after the terminal window is resized. Zsh handles SIGWINCH by re-expanding and redrawing prompt. If you try the same experiment with Bash, you'll notice that the last prompt looks the same as the previous prompts after resizing.

And here's how it looks in GNOME Terminal, which reflows text on resizing.

Initial (after step 2):

Shrunk (after step 3):

Expanded (after step 4):

Notice that old prompts maintain their content when the terminal is resized. If you used Bash, the current prompt would also keep its content. In zsh, however, the current prompt gets repainted after the window is resized and the text is reflown. Zsh tries to repaint it on the same location where it was before the window got resized, but it gets it wrong and repaints the new prompt one line below the intended, so a part of the previous prompt remains on the screen. This is why you see more dashes after the terminal window has been shrunk. The extra dashes are the ghosts of the current prompt that haven't been erased after the window got shrunk.

I've attempted to fix this issue in zsh but then dropped it. See #175. So the current state of affairs is that in most terminals zsh prompt breaks when you resize the terminal window. This problem is not specific to powerlevel10k. It's also not specific to instant prompt although it manifests differently. Basically, when using instant prompt, it becomes more apparent. To see what I mean, try the following in GNOME Terminal.

1. Type this:

PROMPT='${(pl.$COLUMNS..-.)}'$'\n> ' zsh -f --prompt-subst

2. Type reset and press enter just once. You should have only one prompt on the screen.
3. Shrink the terminal window (reduce its width). Observe the content of your terminal.
4. Scroll up. Observe the content of your terminal.

Shrunk (after step 3):

Scrolled up (after step 4):

So, after shrinking it looked like everything is fine, but in fact there was a mess above the view port.
If you resize your terminal back and forth several times, you can generate as much garbage output as you like.

Now let's deal with the breakage of the right (newly created) terminal when you split a terminal window. What happens here is that the TTY (or PTY) for the new terminal originally reports its size as 24 x 80. A few milliseconds later the TTY gets "resized" and the shell receives SIGWINCH as a notification. If the shell prints prints its prompt in the first few milliseconds, it might be broken because the real terminal size isn't 24 x 80 despite what it reports. Here's how you can observe this:

1. Add this at the very top of your ~/.zshrc.

typeset COLUMNS LINES
PROMPT='left> '
RPROMPT='<right'
return

2. Split your terminal window a bunch of times.

Here's what I've got after starting tilix and clicking "Add terminal right":

The left window is the original. Notice that it reports correct dimensions (10 x 90 is what I set as the default size in Tilix preferences). The right window reports incorrect size of 24 x 80, has the dreaded inverted % sign, and a mess of a prompt. In fact, it looks exactly like a broken instant prompt from Powerlevel10k.

So this problem always existed. How come you never noticed this until you enabled instant prompt? The reason is simple. Your prompt wasn't fast enough to be rendered before the first SIGWINCH.

Upstream bug reports:

• TTY always created with size 24 x 80 and then resized gnunn1/tilix#1777
• https://gitlab.gnome.org/GNOME/vte/issues/188

I'm yet unsure what do to about this in Powerlevel10k.

[romkatv]

I've written a more robust workaround for the bug that causes stty size to incorrectly report 24 80 when starting a new terminal. See details in gnunn1/tilix#1777 (comment). @sinetoami Please update powerlevel10k and check whether the new terminal no longer prints % when you split windows.

I don't plan to do anything about prompts that get broken when the terminal window is resized. Thus, when you split a terminal window, all prompts except the current prompt may be broken in the original terminal.

There is another issue with instant prompt that is much more severe than these. If your zshrc can sometimes ask for console input (e.g., a keyring password, or Update [Y/n]?) while loading, instant prompt will break stuff. How exactly it'll break stuff depends on the tool that wants to interact with the user. I've reverted the change to the configuration wizard so that it no longer enables instant prompt. I've also added the following note to the docs:

IMPORTANT UPDATE: Instant prompt is incompatible with zsh startup configs that may require
console input. This includes asking for a keyring password and [Y/N] confirmations. It is
currently NOT RECOMMENDED that you enable instant prompt.

If your zshrc never asks for console input, there is nothing for you to worry about. If some part of it can ask for console input, it needs to be done before instant prompt. I'm looking for a better solution but don't have good ideas at the moment..

[maximbaz]

Hmm nice catch... Have you considered introducing some sort of a hook like "prompt_finished_loading" so that users can put commands that request console input in them?

[romkatv]

Hmm nice catch...

I don't get credit for finding this issue. It was reported by users whose zsh broke.

Have you considered introducing some sort of a hook like "prompt_finished_loading" so that users can put commands that request console input in them?

Such commands need to run before instant prompt. They cannot be run between the instant and the real prompts because they'll eat buffered keybard input (things like ls ~/projects that got typed when instant prompt was showing). This isn't important though. What is important is that restructuring zshrc is too difficult for most users to and I cannot automate it. Moving interactive commands into special_hook(), or moving them above instant prompt -- are both too hard.

[sinetoami]

First of all, nice answer 👍.

I've followed the steps you did and you completely right. With gnome-terminal and xfce4-terminal the behavior are the same as you said here:

The left window is the original. Notice that it reports correct dimensions (10 x 90 is what I set as the default size in Tilix preferences). The right window reports incorrect size of 24 x 80, has the dreaded inverted % sign, and a mess of a prompt.

The urxvt-unicode does not have this behavior.

So this problem always existed. How come you never noticed this until you enabled instant prompt? The reason is simple. Your prompt wasn't fast enough to be rendered before the first SIGWINCH.

Very interesting. But... Why you and @maximbaz don't notice that before I mention? Power of processing or something slow down SIGWINCH rendering?

Please update powerlevel10k and check whether the new terminal no longer prints % when you split windows.

Still prints. Every time on gnome-terminal and xfce4-terminal. urxvt-unicode less times than the others. Theoretically urxvt-unicode is rendering more fastest... And probably this makes sense because I always open a new client/slave of urxvtd, which is a urxvt-unicode deamon started soon as my WM. So this gives me less ms to startup a new terminal. Am I wrong?

[romkatv]

Very interesting. But... Why you and @maximbaz don't notice that before I mention?

When I open a tab in GNOME (whether the first or in addition to existing tabs), terminal dimensions are always reported correctly. When I launch Tilix, terminal dimensions are always reported correctly. They are only incorrect when I create additional tabs in Tilix.

By the way, there is at least 100 people using instant prompt right now, likely many more. The error you are seeing is peculiar to your configuration, which is rare.

Still prints. Every time on gnome-terminal

Let's stick with gnome-terminal for simplicity. Please open it and type the following:

() {
  emulate -L zsh
  typeset -m 'ZSH_VERSION|VTE_VERSION'
  command grep -E 'VTE_VERSION|stty' "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
}

Post the output here.

Describe what you do to trigger the bug with gnome-terminal.

• Is it enough to close gnome-terminal (all tabs, everything) and then launch it? Will it trigger the bug?
• If not, is it enough to click "New Tab" to trigger the bug?
• If not, what is necessary to trigger the bug?

[romkatv]

While at it, please also do the following. Add this code at the top of your ~/.zshrc (before everything):

setopt trapsasync
trap 'echo WINCH ${LINES}x${COLUMNS}' WINCH
echo BEGIN ${LINES}x${COLUMNS}
sleep 1
echo END ${LINES}x${COLUMNS}
return

Then do the same thing that you normally do with GNOME Terminal that would trigger the bug. Post here the output.

[romkatv]

FYI: Another user with a similar setup to @sinetoami has kindly helped with debugging this issue. You can find some details in gnunn1/tilix#1777.

I've implemented (what I believe to be) a robust workaround for VTE bugs, written docs, added warnings and enabled instant prompt in the configuration wizard (there is a new screen that asks about it). You can trigger one warning by adding setopt prompt_cr at the bottom of ~/.zshrc, and another by adding echo test.

If I’ve messed up things too badly, or overestimated the ability of users to understand this complex feature, I might have to revert some stuff later. Waiting for bug reports to start rolling in.

[sinetoami]

() {
  emulate -L zsh
  typeset -m 'ZSH_VERSION|VTE_VERSION'
  command grep -E 'VTE_VERSION|stty' "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
}

Post the output here.

ZSH_VERSION=5.7.1
VTE_VERSION=5802

Describe what you do to trigger the bug with gnome-terminal.

Enabling the instant prompt feature and just open a new terminal, tmux pane or terminal tab. With and without using plugins manager the same thing happens.

• Is it enough to close gnome-terminal (all tabs, everything) and then launch it? Will it trigger the bug?

No. Yes.

• If not, is it enough to click "New Tab" to trigger the bug?

Yes.

I made a fresh install of my zsh settings (from scratch) using my SSD. And the same happens too.

While at it, please also do the following. Add this code at the top of your ~/.zshrc (before everything):

setopt trapsasync
trap 'echo WINCH ${LINES}x${COLUMNS}' WINCH
echo BEGIN ${LINES}x${COLUMNS}
sleep 1
echo END ${LINES}x${COLUMNS}
return

Then do the same thing that you normally do with GNOME Terminal that would trigger the bug. Post here the output.

Always shows:

BEGIN 24x80
END 24x80

In the very left prompt(the most old one) prints this:

WINCH 37x72
WINCH 37x48
WINCH 37x35
WINCH 37x28

In this case, I had 4 terminals opened, if I open one more terminal it will output a new WHICH thing with a new size prompt.

PS.: sorry for the delay. I'm very busy.

[romkatv]

PS.: sorry for the delay. I'm very busy.

No worries. In case you've missed my last comment:

FYI: Another user with a similar setup to @sinetoami has kindly helped with debugging this issue.

[Miguel-Bento-Github]

Hi there,

I'm not sure I find a fix in this thread.
Currently it takes a few seconds for my p10k to start.
I have a very basic config, you can find the zshrc and p10k config below.

I'm in macOS BigSur 11.4
I use both Kitty or the internal VS Code terminal, both take a long time to start.
I've deleted ASL files as per other people issues which changed nothing in my case, removing NVM made it faster although I'll need to add it back in the future, maybe as an alias to avoid slowing down the loading process.

.zshrc

# Enable Powerlevel10k instant prompt. Should stay close to the top of ~/.zshrc.
# Initialization code that may require console input (password prompts, [y/n]
# confirmations, etc.) must go above this block; everything else may go below.
if [[ -r "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh" ]]; then
  source "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
fi

# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
export ZSH="/Users/<myUsername>/.oh-my-zsh"

# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="agnoster"

# Set list of themes to pick from when loading at random
# Setting this variable when ZSH_THEME=random will cause zsh to load
# a theme from this variable instead of looking in $ZSH/themes/
# If set to an empty array, this variable will have no effect.
# ZSH_THEME_RANDOM_CANDIDATES=( "robbyrussell" "agnoster" )

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion.
# Case-sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment the following line to disable bi-weekly auto-update checks.
# DISABLE_AUTO_UPDATE="true"

# Uncomment the following line to automatically update without prompting.
# DISABLE_UPDATE_PROMPT="true"

# Uncomment the following line to change how often to auto-update (in days).
# export UPDATE_ZSH_DAYS=13

# Uncomment the following line if pasting URLs and other text is messed up.
# DISABLE_MAGIC_FUNCTIONS="true"

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# Caution: this setting can cause issues with multiline prompts (zsh 5.7.1 and newer seem to work)
# See https://github.com/ohmyzsh/ohmyzsh/issues/5765
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# You can set one of the optional three formats:
# "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# or set a custom format using the strftime function format specifications,
# see 'man strftime' for details.
# HIST_STAMPS="mm/dd/yyyy"

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)

source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
alias c="cd /Volumes/Code/code/"
source ~/powerlevel10k/powerlevel10k.zsh-theme

# To customize prompt, run `p10k configure` or edit ~/.p10k.zsh.
[[ ! -f ~/.p10k.zsh ]] || source ~/.p10k.zsh

Thanks in advance for the help

[romkatv]

@Miguel-Bento-Github Please open a separate issue.