[BUG] Unable to handle UnicodeEncodeError in RichHandler

[AdrianVollmer]

• I've checked docs and closed issues for possible solutions.
• I can't find my issue in the FAQ.

Describe the bug

When logging invalid utf8 with RichHandler, the logging breaks completely. Even when applying the provided fix. MWE:

import logging
from rich.logging import RichHandler

logger = logging.getLogger()
logger.setLevel("INFO")

#  console_handler = logging.StreamHandler()  # This works
console_handler = RichHandler()  # This throws an exception
console_handler.setLevel("INFO")

logger.handlers = [console_handler]

invalid_output = "\udcf1"

logger.info(invalid_output)

This is the output:

$ PYTHONIOENCODING=utf-8 python testrich.py
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/rich/logging.py", line 170, in emit
    self.console.print(log_renderable)
  File "/usr/lib/python3/dist-packages/rich/console.py", line 1673, in print
    with self:
  File "/usr/lib/python3/dist-packages/rich/console.py", line 865, in __exit__
    self._exit_buffer()
  File "/usr/lib/python3/dist-packages/rich/console.py", line 823, in _exit_buffer
    self._check_buffer()
  File "/usr/lib/python3/dist-packages/rich/console.py", line 2060, in _check_buffer
    self.file.write(text)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf1' in position 60: surrogates not allowed
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Call stack:
  File "/home/adrian/git/New-Solvency/testrich.py", line 15, in <module>
    logger.info(invalid_output)
  File "/usr/lib/python3.12/logging/__init__.py", line 1539, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.12/logging/__init__.py", line 1684, in _log
    self.handle(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.emit(record)
  File "/usr/lib/python3/dist-packages/rich/logging.py", line 172, in emit
    self.handleError(record)
Message: '\udcf1'

It looks like a duplicate of #212, but I'm opening a separate bug because as you can see, the error message is suggesting to set PYTHONIOENCODING=utf-8, but it doesn't work. I don't find this to be an acceptable solution anyway, because I cannot request my users to set this variable. There should be some graceful failure mode.

Platform

Click to expand

I'm using Debian sid. My terminal is Konsole. My shell is zsh.

$ .venv/bin/python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                  │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=88 ColorSystem.TRUECOLOR>                                     │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                  │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 26                                                            │
│    is_alt_screen = False                                                         │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=88, height=26),              │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=88,                                             │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        max_height=26,                                            │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        markup=None,                                              │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=88, height=26)                        │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 88                                                            │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available.                           │
│                                                       │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│                                                       │
│ truecolor = False                                     │
│        vt = False                                     │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ {                                  │
│     'TERM': 'screen-256color',     │
│     'COLORTERM': 'truecolor',      │
│     'CLICOLOR': None,              │
│     'NO_COLOR': None,              │
│     'TERM_PROGRAM': 'tmux',        │
│     'COLUMNS': None,               │
│     'LINES': None,                 │
│     'JUPYTER_COLUMNS': None,       │
│     'JUPYTER_LINES': None,         │
│     'JPY_PARENT_PID': None,        │
│     'VSCODE_VERBOSE_LOGGING': None │
│ }                                  │
╰────────────────────────────────────╯
platform="Linux"
$ uv pip freeze | grep rich
rich==13.9.4

[github-actions]

We found the following entries in the FAQ which you may find helpful:

• How do I render console markup in RichHandler?
• How do I log a renderable?

Feel free to close this issue if you found an answer in the FAQ. Otherwise, please give us a little time to review.

This is an automated reply, generated by FAQtory