FAQ/Troubleshooting - Devanagari not displaying encoding issues in shells zsh/powershell/cmd etc.,

[sumeet-chand]

SOLUTION UPDATE: Use a font that supports Devanagari in your shell

Background: vedic-lang https://vedic-lang.github.io/ cannot run programs written in Sanskrit characters in
common terminals shell. Need to add in FAQ/Troubleshooting steps

Issue: Encoding Errors with Devanagari Characters - Unable to Convert Unicode Character U+094B to UTF-8

Description: When attempting to display the Unicode string "योग.ved" in zsh/powershell/cmd terminals, it appears as "य"ोग".

Troubleshooting:

The octal representation of UTF-8 encoding for each character:

\340\244\257 corresponds to "य" (U+092F)
\340\245\213 corresponds to "ो" (U+094B)
\340\244\227 corresponds to "ग" (U+0917)

UTF-8 encoding for character "ो" (U+094B) is incorrect.

Question: How to fix in zsh (for MacOS/Linux) and PowerShell (for Windows) terminals?

[sumeet-chand]

@PtPrashantTripathi is it working for you?

If so can you run this powershell commands so I can find out the difference and test and record as FAQ/Troubelshooting

# Check Terminal Encoding
Write-Host "Terminal Encoding:" -ForegroundColor Yellow
[Console]::OutputEncoding

# Check System Locale
Write-Host "`nSystem Locale:" -ForegroundColor Yellow
Get-WinSystemLocale

# Check PowerShell Version
Write-Host "`nPowerShell Version:" -ForegroundColor Yellow
$PSVersionTable.PSVersion

# List Installed Fonts
Write-Host "`nInstalled Fonts:" -ForegroundColor Yellow
Get-ChildItem 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts' | ForEach-Object { $_.GetValue("") }

# Check PowerShell Execution Policy
Write-Host "`nPowerShell Execution Policy:" -ForegroundColor Yellow
Get-ExecutionPolicy

# List Available Keyboard Layouts
Write-Host "`nAvailable Keyboard Layouts:" -ForegroundColor Yellow
Get-WinUserLanguageList

[PtPrashantTripathi]

Yes, I believe many terminals, including Windows CMD and macOS Terminal, do not support rendering Devanagari script.

However, since I use Ubuntu, my GNOME Terminal does support it. Many Linux-based terminals offer a wide range of font support. For example, the KDE terminal, Konsole, is excellent for Devanagari language rendering.

If these options don't work, you can try changing the font for your terminal. Monospace fonts are generally preferable.

You can find some suitable fonts here: Sanskrit Fonts on GitHub.

Try different ones to see which works best for you.

[sumeet-chand]

Can confirm issue is resolved by changing fonts to one that supports Devanagari characters on relevant shell