[Runtime Issue]: Unable to set temporary directory to a path that contains Japanese characters #483
Comments
Any reason you'd want to do that with a locale like japanese? Does checking this box fix the issue? |
I don’t really know for sure if there are any reasons why you would want to check that box or uncheck that box. I think that checking that box will break some legacy applications that require code page 932, but I haven’t tested that out on Windows. I have run Japanese applications in Wine before, and I’ve had to switch off of my default UTF-8 locale because the application wasn’t working properly. If you want, I can try some of those same applications on Windows to see if that checkbox breaks anything. That being said, here’s what I do know for sure:
Yes. As I wrote previously,
|
|
Thanks for your reply. Don't hesitate to open a PR with a fix if you can while you have a setup ready for reproduction. |
|
Adding platform-conditional _wfopen() handling should fix this issue. |
Consider this program:
#include <cstdio>
int main(void) {
const char *filename = u8"ディセント3.txt";
auto fp = std::fopen(filename, "r");
if (fp) {
std::fclose(fp);
return 0;
} else {
return 1;
};
}
If a file named ディセント3.txt exists, then will that program
successfully open it? The answer is: it depends.
filename is going to point to these bytes:
Raw bytes: e3 83 87 e3 82 a3 e3 82 bb e3 83 b3 e3 83 88 33 2e 74 78 74 00
Characters: ディセント3.txt␀
Internally, Windows uses UTF-16. When you call fopen(), Windows will
convert the filename parameter into UTF-16 [1]. If the program is run
with a UTF-8 Windows code page, then the above bytes will be correctly
interpreted as UTF-8 when being converted into UTF-16 [2]. The final
UTF-16 string will be this*:
Raw bytes: ff fe c7 30 a3 30 bb 30 f3 30 c8 30 33 00 2e 00 74 00 78 00 74 00
Characters: ディセント3.txt
On the other hand, if the program is run with code page 932, then the
original bytes will be incorrectly interpreted as code page 932 when
being converted into UTF-16. The final UTF-16 string will be this*:
Raw bytes: ff fe 5d 7e fd ff 67 7e 63 ff 67 7e 7b ff 5d 7e 73 ff 5d 7e fd ff 33 00 2e 00 74 00 78 00 74 00
Characters: 繝�繧｣繧ｻ繝ｳ繝�3.txt
In other words, if that program gets compiled on Windows with a UTF-8
execution character set, then it needs to be run with a UTF-8 Windows
code page. Otherwise, mojibake might happen.
*Unlike the first string, this one does not have a null terminator. This
is because the Windows kernel doesn’t use null terminated strings for
paths [3][4].
---
Before this commit, Descent 3 would pass UTF-8 to fopen(), even if
Descent 3 is run with a non-UTF-8 Windows code page [5]. This commit
makes sure that Descent 3 gets run with a UTF-8 Windows code page.
The Windows code page isn’t just used by fopen(). It also gets used by
many other functions in the Windows API [6]. I don’t know if Descent 3
uses any of those other functions, but if it does, then this commit will
also help make sure that those functions receive strings with the
correct character encoding. Descent 3 uses UTF-8 for strings by default
[7]. Making sure that Descent 3 uses UTF-8 everywhere will make
encoding-related mistakes less likely in the future.
Fixes DescentDevelopers#483.
[1]: <https://stackoverflow.com/a/7950569/7593853>
[2]: <https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen?view=msvc-170#remarks>
[3]: <https://stackoverflow.com/a/52372115/7593853>
[4]: <https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html>
[5]: <DescentDevelopers#475 (comment)>
[6]: <https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page#-a-vs--w-apis>
[7]: adf58ec (Explicitly declare execution character set, 2024-07-07)
Build Version
v1.5 de75a80
Operating System Environment
CPU Environment
Game Modes Affected
Game Environment
I’m using game data from the US Windows Dual-Jewel version of Descent 3 v1.4.
Description
If
Descent3.exeis stored in a directory that has Japanese characters in its name, then it will immediately crash if you try launching it:If you press “Cancel”, then the game will try to continue, but it will immediately give you another similar error. That second error does not give you an option to continue.
You can workaround this problem by enabling Windows’s “Beta: Use Unicode UTF-8 for worldwide language support” setting.
Regression Status
Last commit that doesn’t have this problem: 9263055
First commit that does have this problem: 82084e9
Steps to Reproduce
Set your system locale to a non-Unicode Japanese locale.
Go to Windows Settings > Time & language > Language & region > Administrative language settings > Change system locale.
Make sure that “Beta: Use Unicode UTF-8 for worldwide language support” is unchecked.
In the “Current system locale” dropdown box, select “Japanese (Japan)”.
Click “OK”.
Reboot.
Follow the README’s build instructions.
Create a
D3-open-sourcefolder with the required files and folders (See the README’s usage section for details).Try running
Descent3.exe. You should be able to get to the title screen without any errors.Rename the
D3-open-sourcefolder toディセント3.Try running
Descent3.exe. Before the Dolby ad starts, you’ll get an error about not being able to set the working directory toディセント3\custom\cache.The text was updated successfully, but these errors were encountered: