fix: encoding error after brand new installation

[JinkeJ]

Fix encoding error after brand new installation

Description

• Fix encoding error after brand new installation
• Fix [BUG] run_windows.bat script fails with a UnicodeDecodeError in Python when trying to launch Kotaemon in browser #665

Type of change

• New features (non-breaking change).
• Bug fix (non-breaking change).
• Breaking change (fix or feature that would cause existing functionality not to work as expected).

Checklist

• I have performed a self-review of my code.
• I have added thorough tests if it is a core feature.
• There is a reference to the original bug report and related work.
• I have commented on my code, particularly in hard-to-understand areas.
• The feature is well documented.

[DrAcula666-dev]

I am facing the same problem. Fixed it with:

def init(self):
self.dev_mode = getattr(settings, "KH_MODE", "") == "dev"
self.app_name = getattr(settings, "KH_APP_NAME", "Kotaemon")
self.app_version = getattr(settings, "KH_APP_VERSION", "")
self.f_user_management = getattr(settings, "KH_FEATURE_USER_MANAGEMENT", False)
self._theme = KotaemonTheme()

    dir_assets = Path(__file__).parent / "assets"
    with (dir_assets / "css" / "main.css").open(encoding="utf-8") as fi:
        self._css = fi.read()
    with (dir_assets / "js" / "main.js").open(encoding="utf-8") as fi:
        self._js = fi.read()
        self._js = self._js.replace("KH_APP_VERSION", self.app_version)
    with (dir_assets / "js" / "pdf_viewer.js").open(encoding="utf-8") as fi:
        self._pdf_view_js = fi.read()
        # workaround for Windows path
        pdf_js_dist_dir = str(PDFJS_PREBUILT_DIR).replace("\\", "\\\\")
        self._pdf_view_js = self._pdf_view_js.replace(
            "PDFJS_PREBUILT_DIR",
            pdf_js_dist_dir,
        ).replace("GR_FILE_ROOT_PATH", BASE_PATH)
    with (dir_assets / "js" / "svg-pan-zoom.min.js").open(encoding="utf-8") as fi:
        self._svg_js = fi.read()

    self._favicon = str(dir_assets / "img" / "favicon.svg")

    self.default_settings = SettingGroup(
        application=BaseSettingGroup(settings=settings.SETTINGS_APP),
        reasoning=SettingReasoningGroup(settings=settings.SETTINGS_REASONING),
    )

    self._callbacks: dict[str, list] = {}
    self._events: dict[str, list] = {}

    self.register_extensions()
    self.register_reasonings()
    self.initialize_indices()

    self.default_settings.reasoning.finalize()
    self.default_settings.index.finalize()
    self.settings_state = gr.State(self.default_settings.flatten())

    self.user_id = gr.State("default" if not self.f_user_management else None)

[ksze]

Actually why does the environment even assume an encoding other than utf-8? It makes close to zero sense these days. Why don't we fix the environment setup in the run_* scripts so that the Python runtime just assumes utf-8 for everything? Otherwise this will come back to bite us in lots of places where file i/o is needed.

https://dev.to/methane/python-use-utf-8-mode-on-windows-212i

e.g. adding the -Xutf8 option to the CALL python line in the run_windows.bat script works for me.