pandoc-citeproc fails on Linux: "Could not find locale data for C-UTF"

[dhimmel]

As reported in manubot/manubot#167, calling pandoc-citeproc --bib2json on some Linux systems results in the following error:

pandoc-citeproc: Could not find locale data for C-UTF

I believe the error is printed by this line of the source code:

pandoc-citeproc/src/Text/CSL/Data.hs

Line 44 in 24fad3f

show (CSLLocaleNotFound s) = "Could not find locale data for " ++ s

We have experienced this error on two occasions, both using the version of pandoc-citeproc that is bundled with pandoc 2.7.3:

1. calling pandoc-citeproc --bib2json via a subprocess in python on ubuntu-latest via GitHub Actions. pandoc was installed using dpkg --install here. build log
2. calling pandoc-citeproc --bib2json via a subprocess in python on linux via AppVeyor. Pandoc was installed via conda here. build log

Note that the error does not occur on our Travis CI linux builds nor on local linux builds. So perhaps it has something to do with how locales are set for certain CI-optimized linux distros.

I tried lot's of workarounds in manubot/manubot#136 and couldn't figure out any workarounds for this error. @jgm any suggestions?

[jgm]

What does locale say on your system?

[dhimmel]

On the GitHub Actions build, locale returns:

LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

[dhimmel]

Thanks @jgm for the fix in ed70000. In the near-term, is there a way to set our locale information to avoid triggering the error?

[jgm]

Sure, you could export LANG=en_US.UTF-8.