Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*BSD/macOS do not have 'echo -e' #606

Open
ghost opened this issue Sep 8, 2024 · 16 comments
Open

*BSD/macOS do not have 'echo -e' #606

ghost opened this issue Sep 8, 2024 · 16 comments

Comments

@ghost
Copy link

ghost commented Sep 8, 2024

BSD-derived systems ( {Free,Net,Open}BSD & macOS ) do not have echo -e. This causes oh-my-bash to fail in unexpected ways. It's a simple matter to fix it...but you should stop using it and use printf instead (and modifying your CONTRIBUTING guidelines). Here is a list of files that currently use it:

  • tools/autossh.sh
  • plugins/vagrant/vagrant.plugin.sh
  • completions/svn.completion.sh
  • lib/base.sh
  • lib/omb-prompt-base.sh
  • aliases/general.aliases.sh
  • aliases/general.aliases.md
  • themes/hawaii50/hawaii50.theme.sh
  • themes/pro/pro.theme.sh
  • themes/iterate/iterate.theme.sh
  • themes/dulcie/dulcie.theme.sh
  • themes/rana/rana.theme.sh
  • themes/powerline/powerline.base.sh
  • themes/nwinkler/nwinkler.theme.sh
  • themes/nwinkler_random_colors/nwinkler_random_colors.theme.sh
  • themes/agnoster/agnoster.theme.sh

As soon as I get done with what I'm currently working on, I'll attempt a fix and submit a pull request.

Alternatively, you can install GNU coreutils and

alias echo=/usr/local/bin/gecho

but that seems to be a pretty heavyweight solution.

@akinomyoga
Copy link
Contributor

akinomyoga commented Sep 8, 2024

BSD-derived systems ( {Free,Net,Open}BSD & macOS ) do not have echo -e.

Oh My Bash is not written in the portable POSIX shell. Oh My Bash is written in the Bash language because it is a set of settings for Bash. The command echo in Bash is not an executable file but a built-in command of Bash which does support echo -e.

That said, with specific combinations of the shell settings, (set -o posix and shopt -s xpg_echo, which are both disabled by default), the behavior of Bash's built-in echo -e changes. In this sense, it is generally good to avoid using echo at all. Another case would be the case where the Bash builtin echo is disabled by enable -n echo, but it is out-of-support if any of the Bash builtins are changed by the enable builtin (it is impractical not to use Bash builtins at all).

As soon as I get done with what I'm currently working on, I'll attempt a fix and submit a pull request.

Could you wait for a while before starting to work on it? I have conflicting changes to the calls of echos in my local branch. I'll push it.

@akinomyoga
Copy link
Contributor

I merged my local branch in the master and pushed it (770d6ef). Some of echo -e have disappeared (for a slightly different reason), but there are still many occurrences of echo -e. You can start working on it.

@akinomyoga
Copy link
Contributor

Some remarks:

  • tools/autossh.sh

This is a file that is intended to be used as an independent executable script, so it is hard to consider the case it is used with the unusual combination of the shell options set +o posix and shopt -s xpg_echo. However, is that forcibly enabled in some BSD systems even for independent executable scripts?

  • completions/svn.completion.sh

This file is synced with the completion file provided by the upstream Subversion project [1], so we would like to avoid custom changes on our side as much as possible. You should first ask the upstream Subversion project to fix echo -e, then we can pull the change.

[1] https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/bash_completion

@ghost
Copy link
Author

ghost commented Sep 8, 2024

akinomyoga wrote:

Some remarks:

  • tools/autossh.sh

This is a file that is intended to be used as an independent executable script, so it is hard to consider

<snip />

  • completions/svn.completion.sh

This file is synced with the completion file provided by the upstream Subversion project

<snip />

OK... I just used egrep -l on all the files in the oh-my-bash installation to build the list that was in my original post:

cd .../oh-my-bash
find . -type f -exec egrep -l 'echo \-e' {} \; | sed -e 's/\.\///g'

akinomyoga wrote:

I merged my local branch in the master and pushed it (770d6ef). Some of echo -e have disappeared (for a slightly different reason), but there are still many occurrences of echo -e. You can start working on it.

OK...thanks...

akinomyoga wrote:

BSD-derived systems ( {Free,Net,Open}BSD & macOS ) do not have echo -e.

Oh My Bash is not written in the portable POSIX shell. Oh My Bash is written in the Bash language because it is a set of settings for Bash. The command echo in Bash is not an executable file but a built-in command of Bash which does support echo -e.

<snip />

I had an alias that was masking the builtin behavior. Most of my problems are now gone. That being said, I understand that bash is not the POSIX shell. I understand that you can run bash in "POSIX mode" but you're subject to FSFs interpretation of what POSIX really means. I understand that's a common thing but FSF (and, particularly, rms) seems to have their own private versions of the POSIX standards that they adhere to.

On FreeBSD, you have to explicitly install bash. The default shell when you create a new user is sh (the POSIX shell). The behavior of sh and bash (POSIX mode) is sometimes quite different. I tend to not trust bash (POSIX mode) at all.

It would be very hard to port oh-my-bash to the POSIX shell because of its heavy use of arrays (among other things).

<snip />

That said, with specific combinations of the shell settings, (set -o posix and shopt -s xpg_echo, which are both disabled by default),

Yes, they're disabled in my invocation of bash as well.

<snip />

As soon as I get done with what I'm currently working on, I'll attempt a fix and submit a pull request.

OK...

All that being said, printf is still a much better solution...it's more flexible/capable and it avoids this dichotomy between builtin echo and /bin/echo. I have a couple of more days on what I'm working on...then I can start working on converting echo -e to printf. Thanks for your quick response. You can mark this report as in progress or whatever you use to track such matters.

Thanks for your help.

@ghost
Copy link
Author

ghost commented Sep 8, 2024

I have a bash function library that would help immensely in getting rid of echo usage in themes. It's also portable to zsh. Instead of doing something like:

echo -e '\[\033[1m\]this is bold text\[\033[0m\]'

you could write

printf "%s" "$(__attr__ bold)this is bold text$(__reset__)"

It handles all the common text attributes (bold, italic, underline, etc), 4-bit (foreground) color and 8-bit foreground/background color. This is common stuff on xterm and xvt terminals. Where would I install it in oh-my-bash to be able to use it in themes?

@akinomyoga
Copy link
Contributor

It would be very hard to port oh-my-bash to the POSIX shell because of its heavy use of arrays (among other things).

Yes, but we don't intend to port it. We don't expect users to try that either; users can still try, but we are not going to help such users.

echo -e '\[\033[1m\]this is bold text\[\033[0m\]'

you could write

printf "%s" "$(__attr__ bold)this is bold text$(__reset__)"

In Oh My Bash (OMB), it is supposed to be replaced with something like this

_omb_util_print "\[$_omb_term_bold\]this is bold text\[$_omb_term_normal\]"

It handles all the common text attributes (bold, italic, underline, etc), 4-bit (foreground) color and 8-bit foreground/background color. This is common stuff on xterm and xvt terminals. Where would I install it in oh-my-bash to be able to use it in themes?

Since the command substitutions have a fork cost (and we continue to support Bash 3.2 where Bash-5.3's function and value substitutions are not going to be available), we want to continue to cache the results of $(any command generating control sequences). Then, if you want to use the results of your tools, you can overwrite those variables after loading Oh My Bash. You can obtain the list of variables to overwrite by running the following command in a session where OMB is loaded:

$ compgen -v _omb_term_

@ghost
Copy link
Author

ghost commented Sep 9, 2024

akinomyoga wrote:

In Oh My Bash (OMB), it is supposed to be replaced with something like this

_omb_util_print "\[$_omb_term_bold\]this is bold text\[$_omb_term_normal\]"

Ah...OK... I'll have a look.

Pardon my ignorance but I only started using oh-my-bash a few days ago -- to rewrite some scripts I've supported for > 20 years that were getting pretty long in the tooth.

@ghost
Copy link
Author

ghost commented Sep 9, 2024

I'm not a GitHub wizard... I'm having some problems. I'm able to clone the repo locally:

$  git clone [email protected]:/ohmybash/oh-my-bash.git ./oh
Cloning into './oh'...
remote: Enumerating objects: 4952, done.
remote: Counting objects: 100% (2295/2295), done.
remote: Compressing objects: 100% (651/651), done.
remote: Total 4952 (delta 1864), reused 1758 (delta 1643), pack-reused 2657 (from 1)
Receiving objects: 100% (4952/4952), 4.75 MiB | 9.57 MiB/s, done.
Resolving deltas: 100% (2944/2944), done

I can create a local branch:

$  git branch remove-echo

$  git checkout remove-echo
Switched to branch 'remove-echo'

I can't push the local branch to the remote:

$  git push origin remove-echo

ERROR: Permission to ohmybash/oh-my-bash.git denied to sw030453.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Do you have to do something to allow this to happen?

TIA...

@akinomyoga
Copy link
Contributor

akinomyoga commented Sep 9, 2024

Which web page or documentation did you reference in learning how to make a pull request? I think you should find another web page that correctly describes the procedure.

In short, you need to create your own remote copy of the repository by pressing the "Fork" button on GitHub. Then, in your local repository, you need to set up the references to the remote repositories correctly. You can use the following commands:

$ cd <your local repository>
$ git remote rename origin upstream
$ git remote add origin [email protected]:sw030453/oh-my-bash.git
$ git fetch origin

Please search the internet by yourself if you want to learn what each command does.

@ghost
Copy link
Author

ghost commented Sep 9, 2024

I know what all that does…. Like I said, I’m not a GitHub wizard. Thanks for your quick response.

@ghost
Copy link
Author

ghost commented Sep 9, 2024

OK... I have a working repo. I did a couple of innocuous changes and committed them to make sure my repo works.

I have some questions:

  • Is there an automated test suite? If so, where is it and how do I use it?

  • Are _omb_util_{print,put} safe to use everywhere? I'm assuming that they may not be safe for use in lib/*... Is this correct?

Thanks in advance...

@akinomyoga
Copy link
Contributor

OK... I have a working repo. I did a couple of innocuous changes and committed them to make sure my repo works.

I have some questions:

  • Is there an automated test suite? If so, where is it and how do I use it?

Not really. We have a test for the installation script in .github/workflows/test.sh, but it's intended to be run in a temporary environment in the GitHub CI (continuous integration). You can technically run it by bash .github/workflows/test.sh outside the temporary environment, but in such a case, you need to manually clean up the temporary directory created by the test.

  • Are _omb_util_{print,put} safe to use everywhere? I'm assuming that they may not be safe for use in lib/*... Is this correct?

You can use it everywhere (except in tools/*.sh). These functions are defined in lib/omb-util.sh, and lib/omb-util.sh is loaded at the very beginning at the following place in oh-my-bash.sh:

oh-my-bash/oh-my-bash.sh

Lines 109 to 117 in 770d6ef

# Load all of the config files in ~/.oh-my-bash/lib that end in .sh
# TIP: Add files you don't want in git to .gitignore
_omb_module_require_lib omb-util
_omb_module_require_lib utils
_omb_util_glob_expand _omb_init_files '{"$OSH","$OSH_CUSTOM"}/lib/*.{bash,sh}'
_omb_init_files=("${_omb_init_files[@]##*/}")
_omb_init_files=("${_omb_init_files[@]%.bash}")
_omb_init_files=("${_omb_init_files[@]%.sh}")
_omb_module_require_lib "${_omb_init_files[@]}"

As is observed in the above part, line 111 loads lib/omb-util.sh and line 112 loads lib/utils.sh. The other lib/*.sh and lib/*.bash are loaded on line 117. So you can use it everywhere except the beginning part of oh-my-bash.sh (lines 1..110) and scripts in tools/*.sh that are intended to be executed as separate shell programs.

@ghost
Copy link
Author

ghost commented Sep 9, 2024

I'm all done making changes. Examine the file files.txt for a summary of what exactly I did. I'm using my repo in two different installations of oh-my-bash:

  • macOS 14.6.1 (23G93) (Intel Core i5)
  • FreeBSD 14.1 p3 (AMD64)

I'll generate a pull request in a couple of days. There were some files where I changed

#!/usr/bin/env bash

to

#!/usr/bin/env -S bash

You can look at the man(1) page for /usr/bin/env to read about what exactly -S does... For purposes here, adding -S allows /usr/bin/env to pass command-line parameters to bash...as in:

#/usr/bin/env -S bash --posix --restricted

to make bash operate in restricted, POSIX mode while running that script only; or

#/usr/bin/env -S bash -O x

to make bash do a set -x before running that script only. It's a useful tool when needed and incurs no performance penalties.

files.txt

@akinomyoga
Copy link
Contributor

I'm all done making changes. Examine the file files.txt for a summary of what exactly I did. I'm using my repo in two different installations of oh-my-bash:

I don't have any conclusive opinion about them until I see the actual changes (instead of just a list of changed files), but the following files have the respective upstream repositories. Since we just copy the file from the upstream occasionally, we wouldn't want to change it on our side. The changes should be first introduced in the upstream and then should be pulled to our repository.

.x tools/bash-preexec.sh
.x- completions/svn.completion.sh

I don't remember whether the other files have corresponding upstream one by one, so there might be a few other files that we don't want to change. It also depends on the size of the changes, so I'll judge them after checking the submitted PR.

Also, please be prepared that I normally request additional changes in PRs after reviewing the PRs. It's normal, and it's independent of the PR quality.


#!/usr/bin/env bash

to

#!/usr/bin/env -S bash

You can look at the man(1) page for /usr/bin/env to read about what exactly -S does...

That seems useful but it doesn't seem to be POSIX. As far as I check XCU 3 env in the online publication of IEEE 1003.1-2024, the option =S is not recognized as a portable feature, which means that env doesn't necessarily have the option =S in a wide range of environments.

@ghost
Copy link
Author

ghost commented Sep 9, 2024

akinomyoga wrote:

That seems useful but it doesn't seem to be POSIX. As far as I check XCU 3 env in the online publication of IEEE 1003.1-2024, the option =S is not recognized as a portable feature, which means that env doesn't necessarily have the option =S in a wide range of environments.

OK... I've backed all those out. There were only about a half-dozen or so.

akinomyoga wrote:

Also, please be prepared that I normally request additional changes in PRs after reviewing the PRs. It's normal, and it's independent of the PR quality.

I spent a very long time writing software professionally (prob more years than you are old). I would be alarmed if you didn't do this... ;-)

@akinomyoga
Copy link
Contributor

akinomyoga commented Sep 9, 2024

I spent a very long time writing software professionally (prob more years than you are old). I would be alarmed if you didn't do this... ;-)

Thank you for your understanding. Some people new to OSS seem to feel offended when they first receive many requests for changes on their work. I don't have prior knowledge about the OSS experience of the PR submitter, with whom I communicate through the internet, so just in case, I wanted to tell that one doesn't need to worry when they receive many requests on the PR.

@akinomyoga akinomyoga mentioned this issue Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant