grep, sed and spanish accents

[ALGBBSH]

Problem description

grep and sed don't work with spanish accent. For example, "echo camión|grep camión", the terminal cannot find anything.

Steps to reproduce

grep (v.3.4)
echo "camión"|grep "camión"
the terminal cannot find anything.

In sed (4.8) doesn't work either.
echo camión|sed s/'ó'/'\'"'o"/g

Could it be a problem of variable LANG?

Expected behavior

echo "camión"|grep "camión"
Reply: camión

echo camión|sed s/'ó'/'\'"'o"/g
Reply: camion

Additional information

Updatable packages:
git/stable 2.26.1 aarch64 [upgradable from: 2.26.0]
Android version:
9
Kernel build information:
Linux localhost 4.4.148-perf+ #1 SMP PREEMPT Wed May 22 10:44:11 2019 aarch64 Android
Device manufacturer:
Sony
Device model:
G8141

[S-E-TT]

It works in Android's native version of grep

[joseanoxp]

It works in Android's native version of grep

They still haven't solved that nonsense, disappointing.

[S-E-TT]

Btw, is there a good reason the native Android apps are not in $PATH by default?
If they were it would be a lot easier to just delete Termux's grep and then the Android one would be used. Either way, just having it on the end by default seems smart to me, there might be apps available there that's not in Termux too

[ghost]

Btw, is there a good reason the native Android apps are not in $PATH by default?

1. If user don't have corresponding package installed for some reason, utility will be picked from /system but user will not know what exactly is being used.
2. System utilities may have different behavior. In this case, results of (1) will get even worse.
3. System utilities may have different command line args. In this case, results of (1) will get even worse.
4. Access to system utilities can be restricted by SELinux. I, for example, can't list /system/bin at all.
5. Termux purpose is to bring a normal port of GNU/Linux tools, not use ones from system. Issues, of course, are expected.
6. Termux scripts rely on tools from $PREFIX, their usage syntax and behavior.

Want to have /system/bin in $PATH, configure it in ~/.bashrc or what is being loaded by your shell.

If they were it would be a lot easier to just delete Termux's grep

It is essential package and included as dependency in other essential/non-essential packages. Deleting it will simply cause serious dependency tree breakage which will lead to issues with package manager.

[S-E-TT]

At least my Samsung Galaxy has accessible binaries in /vendor/bin, but that might not be the same for every phone. But besides that, you make some very good points, xeffyr. Let's hope the bug gets fixed in a future release then.

[Ferdi265]

Interestingly, busybox grep from termux repos does support UTF-8, while GNU grep does not.

Scripts relying on termux packages that want to workaround this can therefore depend on busybox and call busybox grep. (same with sed)

Example:

[tomty89]

@xeffyr @Grimler91 @fornwall Is/was there a reason that grep and sed aren't linked against libiconv instead of / in addition to libandroid-support?

[Grimler91]

Is/was there a reason that grep and sed aren't linked against libiconv

Probably just historic reasons, we relied on the limited libiconv (and zlib) functionality from android's libraries for many years

Simply linking against libiconv does not have an effect on this issue though, the resulting grep does not depend on libiconv (it is not needed), maybe some other patch is needed as well

[tomty89]

Right, I thought it was relevant because when I just built grep on device and it worked as oppose to the repo build. But then I disable iconv and it still works. (With or without libandroid-support linkage)

Interestingly, I notice that the on-device build has linkage to libdl while the repo build does not. Not sure if that's relevant.

On my device:

$ ls -l /system/lib64/libdl.so
lrw-r--r-- 1 root root 47 Aug  8  2018 /system/lib64/libdl.so -> /apex/com.android.runtime/lib64/bionic/libdl.so
$ ls -l /system/lib64/libc.so
lrw-r--r-- 1 root root 46 Aug  8  2018 /system/lib64/libc.so -> /apex/com.android.runtime/lib64/bionic/libc.so

[zixijian]

So is there any temporary solution?
My machine's /system/bin/grep can match the characters I need, but does not support the -v -w parameter.

[Grimler91]

So is there any temporary solution?
My machine's /system/bin/grep can match the characters I need, but does not support the -v -w parameter.

busybox grep should support those flags at least

[zixijian]

Can use 3.1 version.

wget https://mirrors.ustc.edu.cn/gnu/grep/grep-3.1.tar.xz
tar xf grep-3.1.tar.xz
cd grep-3.1
./configure 
make 
mv $PREFIX/bin/grep $PREFIX/bin/grep.bak
cp src/grep $PREFIX/bin/grep

[Ferdi265]

@zixijian: it's not specific to the 3.1 version.

Building 3.6 from source works fine as well and builds a working grep. Building 3.7 (the version currently shipped by termux) doesn't compile without patches for me. The gnulib.patch from this repo makes it compile, but the resulting grep does not work.

I also don't know if termux packages use any specific flags for ./configure, I currently just used a ./configure without any arguments.

[Grimler91]

I also don't know if termux packages use any specific flags for ./configure

For grep, just --prefix=$PREFIX basically (without it you won't be able to make install). All other options are auto detected by the configure script, and when we cross-compile it seems that some option is messed up by the auto detection