tracked_files needs first run in order to be created in prefix; Special K still shows warning, but works fine if vcrun and d3dcompiler_47 are installed through winetricks

[CartoonFan]

System Information

• SteamTinkerLaunch version: v14.0.20230903-1
• Distribution: Arch Linux
• Installation Method: AUR

Issue Description

First off, I apologize: this issue was mostly due to me deleting steam_api64.dll (which NieR: Automata--the game I was testing--requires to not crash), thinking it was a part of the Special K installation. The next big problem was the tracked_files file not appearing in the game prefix, which happens if you don't do a first run beforehand to create it in there. Not sure if that last part's a bug or not--I feel like STL should create it if it's not there--but overall, it's probably not a huge issue by itself.

A slightly more annoying thing is that vcrun2022 and d3dcompiler_47 have to be installed through Winetricks (the former for Special K, and the latter for both Special K and ReShade), which makes the launch process much longer. The warning about Special K not finding the specific dll it wants still shows up, and the setup process for it doesn't trigger, but installing both vcrun2022 and d3dcompiler_47 allow it to run properly. Again, I'm not sure if that's a bug (it feels like it) or not. I think it took two launches for the Winetricks verbs to actually install, as well, making the process take even longer. As far as I can tell, this needs to be done for any game that uses Special K.

Logs

524220_gamelaunch.log
steam-524220.log
524220_steamtinkerlaunch.log

[sonic2kk]

I saw your initial issue earlier with your "Ted Talk", I apologise that I didn't get a chance to fully dig into it (took a nap after work to beat a heatwave). But I am assuming that all of that from earlier is now resolved, and that the core of the issue is what you've described here, which should boil down to:

1. tracked_files won't exist in the game prefix (i.e. /home/Gaben/Games/steamapps/compatdata/524220) until the game is launched at least once, and this causes Special K to fail (because it relies on vcrun2022).
2. Special K requires vcrun2022 and d3dcompiler_47 to be installed in order to work. Once installed for a game it will work, but it will show a warning message that can be (safely?) ignored.

If that is the case, right now the following is needed for games to use Special K (or at least NieR:Automata), where Step 2 and Step 3 could be done in either order (though it will crash without Step 2, which resolves the crash):

1. Run the game at least once
2. Install vcrun2022 and d3dcompiler_47 (for example, using Winetricks)
3. Install Special K by enabling the Special K checkbox
4. Profit! NieR:Automata (and hopefully any other Special K supported game) will work, though Special K will show a warning.

And once again assuming the above is correct, SteamTinkerLaunch would need to do the following:

1. At least document that the game should be ran once before installing Special K
2. Install vcrun2022 and d3dcompiler_47 during the Special K installation step
3. More? Perhaps not!

---

So if that's all correct, let me move onto my understanding of what STL should be doing right now:

1. It should be installing d3dcompiler_47 into the game files, however it appears that the DLL override is not being set (${D3D47//.dll}=n,b is used for ReShade so it can load d3dcompiler_47, but there is no check at all for ReShade.
2. It should run the Steam vcrun2019 installation package instead of using Winetricks, where it can take a very long time and sometimes fail. Perhaps this isn't working for some reason?
   • With a fresh NieR:Automata prefix, could you confirm if installing d3dcompiler_47 as the only Winetrick fixes the problem? Winetricks would set the needed DLL override during installation, so that step isn't necessary.
   • Doing this would confirm whether or not the vcrun2019 install step is actually working from STL.
   • If the game still crashes, you could try clicking on the "Steam first run setup" button on the Main Menu, and from there enabling vcredist2022 (or 2019, or whatever other you want to try). Then you can click "Install without saving". This will run the Steam Evaluator Script for the game temporarily modified to install vcrun2022, as though it were one of the dependencies for the game to run. When you run a game for the first time, sometimes it has to do some first-time setup by installing various packages. STL has a list of these packages, with vcredist2022 being one of them, so essentially here we are performing the vcredist2022 as though the game told Steam it required it.

I would also be interested in knowing if these steps can cause conflicts with other games. They shouldn't, but does this mean that with these tweaks, SpecialK generally speaking works? I'm sure some games will be a PITA but I'd like to make sure that basically this isn't a case of fixing SpecialK for NieR:Automata and breaking it for all others.

I can do some light testing once I get around to trying to fix these issues (i.e. properly installing vcrun for SpecialK and also properly exporting d3dcompiler_47). I tested Special K a long time ago and it crashed NieR:Automata, your comments on a previous issue suggested workarounds that appear to work from your screenshots. I also tested NieR:Replicant and it worked, but it was a bit crashy (this could've been other things, not specifically SpecialK). So at the least I can test both of these games afterwards to make sure compatibility is preserved :-)

---

I appreciate the enormous amount of time you have spent investigating and troubleshooting this issue, and for doing your best to make sure it contains the most accurate information possible in order to best report bugs and suggest improvements. Your comment on the previous issue and your comment here have given me a lot of hope that Special K isn't utterly broken beyond repair at the moment, and also that games will generally work with it albeit with some workarounds (including NieR:Automata).

And finally,

The warning about Special K not finding the specific dll it wants still shows up, and the setup process for it doesn't trigger, but installing both vcrun2022 and d3dcompiler_47 allow it to run properly. Again, I'm not sure if that's a bug (it feels like it) or not. I think it took two launches for the Winetricks verbs to actually install, as well, making the process take even longer. As far as I can tell, this needs to be done for any game that uses Special K.

I agree that this is likely a bug. Could you give some more information on this warning that Special K shows, i.e. does it go away, and also what you mean by the Special K setup process not triggering?

Thanks!

cc @zany130 you may be interested in the discussion here :-)

[sonic2kk]

I should also note that, if we can perform these vcrun and d3dcompiler installation steps without Winetricks, hopefully that will resolve the tracked_files issue (since d3dcompiler will be picked up by the Wine DLL override, and vcrun should be picked up by Steam and put into the prefix during the evaluator first run).

Just wanted to say again that the investigation you did and mentioning even details like tracked_files at all (something I think a lot of users may glance over) is very useful for investigating problems and coming up with ways to fix them!

[zany130]

TBH I am very out of the loop with Special K. I was at one point keeping up with it and trying to get it to work with ReShade but gave up. So not sure I can add much here.

However, I don't remember much needing to be done to get it to work other than

Adding the .dll the same way you would for ReShade (dxgi.dll for d3d11+, d3d11.dll,d3d9,d3d8,OpenGL32.dll, there were a few more you could install it as I think but don't remember)

and then adding the .dll override

not sure if vcrun is new, though I think that should usually be installed by the game's first-time setup?

[sonic2kk]

Not all games use vcrun and some use older versions, it is possible Special K may require a specific kind.

No worries that you can't add much but if there is a development here it may be something that would interest you as well, just keeping you in the loop also :-)

[sonic2kk]

I tested out NieR:Automata with SpecialK and I have learned a few things:

1. STL complains about a random DLL, api-ms-win-crt-string-l1-1-0.dll - If this is not in the prefix, STL will not install SpecialK. This is not in any new prefix I create, so SpecialK fails to install. Manually copying the DLL from a prefix (or the XCOM 2 Windows release game files, which interestingly has it) will fix the error. However, when running a game, it will be removed from the prefix. This check can probably be removed.
   • Frostworx was the one who noted that this was required almost 2 years ago (I guess this is when Special K stuff was originally written?) on the SpecialK forums. It is not unlikely that this workaround is no longer required!
2. SpecialK will crash on boot unless the vcrun2022 winetrick OR vcredist2022 is installed from the Steam one time setup option -- Likely, we can automate this step.
3. No DLL exporting is required, somehow d3dcompiler_47 and dxgi are picked up to inject Special K.
4. Minor point: I had a very old version of SpecialK installed that SteamTinkerLaunch was using, like from mid 2022 (probably when the SpecialK rewrite happened) and this did not work at all with NieR:Automata. Removing these files and allowing STL to download a fresh latest and default release fixed my problems.

So the takeaways from this on my end are the following:

1. Remove the the api-ms-whatever check, it is no longer needed
2. Document that the game you want to use SpecialK with should be launched at least once beforehand to create an initial prefix (unless testing reveals that the vc2022 install takes care of this)
3. Install vc2022 instead of 2019
4. Profit?

Likely the vcrun2022 winetrick worked because it should do the same thing as running vcredist2022. I am not sure why it appeared that d3dcompiler_47 was required as a Winetrick, as I am not sure it is exported anywhere. In my testing for NieR:Automata, it wasn't needed.

I am going to test all of this with a fresh prefix and no SpecialK files, then try to do all this from scratch. Fingers crossed!

[sonic2kk]

Confirmed that the following steps worked:

1. Fresh install of a SpecialK-compatible game, i.e. NieR:Automata
2. Install vcredist2022 from "Steam first run setup" menu (this creates a copy prefix, strangely? I guess it re-runs the games setup entirely with a new prefix)
3. Install Special K
4. Profit!

In the code I am having difficulty getting STL to automatically install the vcredist2022. It is very fast, a matter of seconds, from the menu button, but it would be nice to automate this as part of the SpecialK installation. Using vcrun2022 winetrick also works no problem, but I would like to do this automatically. STL is supposed to install vcredist2019 but maybe this is failing.

I think as far back as vcredist2015 is supposed to be compatible, but perhaps the core issue is that this step is failing with STL. I would have to test manually running these.

[CartoonFan]

Confirmed that the following steps worked:

1. Fresh install of a SpecialK-compatible game, i.e. NieR:Automata

2. Install vcredist2022 from "Steam first run setup" menu (this creates a copy prefix, strangely? I guess it re-runs the games setup entirely with a new prefix)

3. Install Special K

4. Profit!

In the code I am having difficulty getting STL to automatically install the vcredist2022. It is very fast, a matter of seconds, from the menu button, but it would be nice to automate this as part of the SpecialK installation. Using vcrun2022 winetrick also works no problem, but I would like to do this automatically. STL is supposed to install vcredist2019 but maybe this is failing.

I think as far back as vcredist2015 is supposed to be compatible, but perhaps the core issue is that this step is failing with STL. I would have to test manually running these.

Nice. Unfortunately, I'm a little behind on this issue; I've been trying to test #894, but the ReShade dll looks like it's only installed if the Special K checks pass, so...I couldn't really do much there 😅

[sonic2kk]

I was able to confirm that vcredist2019 works, but that vcredist2015 does not. I was also able to confirm that even though SteamTinkerLaunch is supposed to try and install vcredist2019 (currently on master, locally I have vcredist2022) that it does not. From looking at the logs it does not ever attempt to do this, which seems to be a bug.

So the remaining issue to solve is how to get SteamTinkerLaunch to properly install vcredist2022 automatically 🤔

[sonic2kk]

but the ReShade dll looks like it's only installed if the Special K checks pass

Is this related to the ms-api-whatever DLL? If so, I can push that change as a priority. I had to remove this to actually get SpecialK to install for me with minimal manual intervention (outside of vcredist2022, no manual intervention was needed).

[CartoonFan]

but the ReShade dll looks like it's only installed if the Special K checks pass

Is this related to the ms-api-whatever DLL? If so, I can push that change as a priority. I had to remove this to actually get SpecialK to install for me with minimal manual intervention (outside of vcredist2022, no manual intervention was needed).

I'm guessing so. It's the only thing in the way that I can think of 🤔

[CartoonFan]

OK! Finally caught up with the replies here 😁

But I am assuming that all of that from earlier is now resolved, and that the core of the issue is what you've described here, which should boil down to:

1. tracked_files won't exist in the game prefix (i.e. /home/Gaben/Games/steamapps/compatdata/524220) until the game is launched at least once, and this causes Special K to fail (because it relies on vcrun2022).

The only thing I'd want to add here is that--with vcrun2022, I think--a single run did not create tracked_files in the prefix. I either had to use Steam first run setup + Re-create Wineprefix--which worked great for this purpose--or some combination of two game launches or installing something else through Winetricks (usually d3dcompiler_47). I was kind of frustrated that it wouldn't show up after a single launch.

I would also be interested in knowing if these steps can cause conflicts with other games. They shouldn't, but does this mean that with these tweaks, SpecialK generally speaking works? I'm sure some games will be a PITA but I'd like to make sure that basically this isn't a case of fixing SpecialK for NieR:Automata and breaking it for all others.

I can do some light testing once I get around to trying to fix these issues (i.e. properly installing vcrun for SpecialK and also properly exporting d3dcompiler_47). I tested Special K a long time ago and it crashed NieR:Automata, your comments on a previous issue suggested workarounds that appear to work from your screenshots. I also tested NieR:Replicant and it worked, but it was a bit crashy (this could've been other things, not specifically SpecialK). So at the least I can test both of these games afterwards to make sure compatibility is preserved :-)

I was planning to check Monster Hunter: World. Previously, the third-party mod loader crashed with vcrun2015 (which may have just been the steam API dll again), which led me to try vcrun2022, and that caused Special K to throw the ms-api warning...leading me to return to NieR: Automata to see what went wrong (and create this bug). All of that might have been unnecessary, but it seems like we're all gaining from it 😁

I appreciate the enormous amount of time you have spent investigating and troubleshooting this issue, and for doing your best to make sure it contains the most accurate information possible in order to best report bugs and suggest improvements. Your comment on the previous issue and your comment here have given me a lot of hope that Special K isn't utterly broken beyond repair at the moment, and also that games will generally work with it albeit with some workarounds (including NieR:Automata).

Thanks 😊 . I still feel kind of dumb for presenting all that as legitimate bugs when it was just me deleting a dll that turned out to be important. Like, someone took the time to read and process all that, and then most of it just got invalidated because I did something dumb. I don't want to waste anyone's time, you know?

Just wanted to say again that the investigation you did and mentioning even details like tracked_files at all (something I think a lot of users may glance over) is very useful for investigating problems and coming up with ways to fix them!

Thanks for the supportive words 🫡. I figured I'd just throw as much as I could out into the wild to give you (or whoever else) the information needed to solve the problem. In the other issue, you mentioned wanting to know how tracked_files worked (if Steam populates an empty file, etc.), so I thought I'd throw that into the rest of the files as well.

The warning about Special K not finding the specific dll it wants still shows up, and the setup process for it doesn't trigger, but installing both vcrun2022 and d3dcompiler_47 allow it to run properly. Again, I'm not sure if that's a bug (it feels like it) or not. I think it took two launches for the Winetricks verbs to actually install, as well, making the process take even longer. As far as I can tell, this needs to be done for any game that uses Special K.

I agree that this is likely a bug. Could you give some more information on this warning that Special K shows, i.e. does it go away, and also what you mean by the Special K setup process not triggering?

This was in reference to the api-ms-win-crt-string-l1-1-0.dll error--the issue we're currently working on 😁

Hope this helps. Once again, thank you for your kind words 🙏 💜

Hopefully, I didn't ramble on too much 🤣

[sonic2kk]

Not too much rambling, I appreciate keeping up with this! Even as maintainer it's easy for me to get a bit lost with all of this.

The only thing I'd want to add here is that--with vcrun2022, I think--a single run did not create tracked_files in the prefix. I either had to use Steam first run setup + Re-create Wineprefix--which worked great for this purpose--or some combination of two game launches or installing something else through Winetricks (usually d3dcompiler_47). I was kind of frustrated that it wouldn't show up after a single launch.

I'm not sure that I experienced that, but it is late and I may have just forgotten. To my understanding, all I had to do was (with the ms-api-whatever dll check removed from the code):

1. Install game (i.e. NieR:Automata) fresh
2. Run it once with GE-Proton8-14 (Proton Experimental appears to work as well)
3. Run NieR:Automata with SteamTinkerLaunch and install vcrun2022 manually (it fails for some reason during SpecialK install -- This will need fixing as part of solving this issue)
4. Enable SpecialK (latest and default both worked)
5. Profit

The issue with tracked files could be something prefix-specific. I am not sure what might be causing it.

This was in reference to the api-ms-win-crt-string-l1-1-0.dll error--the issue we're currently working on

Ah okay, I caught up in the end with this and #896 should resolve it. I will probably just merge it after posting this comment -- seems pretty important. This will hopefully allow you to test out the (very basic) ReShade and SpecialK stuff for #894.

I was planning to check Monster Hunter: World. Previously, the third-party mod loader crashed with vcrun2015 (which may have just been the steam API dll again), which led me to try vcrun2022, and that caused Special K to throw the ms-api warning...leading me to return to NieR: Automata to see what went wrong (and create this bug). All of that might have been unnecessary, but it seems like we're all gaining from it

I think this issue has helped highlight that the SpecialK installation code was in dire need of some tweaking. Even if it took us a while to figure out what exactly the problems were and what causes them, I think indeed for me I've gotten to learn a lot more about how STL handles SpecialK. It's much less of a mystery box for me now! Plus it's good fun tracking down issues like this :-)

still feel kind of dumb for presenting all that as legitimate bugs when it was just me deleting a dll that turned out to be important

It's no problem, you spent the time figuring out the problem. I've been here too, I can very easily get myself tangled up in rabbit holes where I get all the details wrong in the end (happened this summer with Vortex, spent about a week entirely on the wrong track). I appreciate any user who spends time to try and understand the problem because even if initially there was some misunderstandings, clearing that up helps everyone. And hey, it's always better than just opening a blank issue 😉

---

To help straighten things out in my mind, this issue has uncovered a few issues:

1. There is a now-invalid Special K DLL check that is preventing Special K from being installed.
2. vc2022 is not installed automatically, and has to be manually installed to prevent a crash with loading the SpecialK DLL.
3. Multiple game launches may be required to get tracked_files to appear in the game prefix, and if this file is missing, Proton doesn't like that very much and won't launch the game.

So to resolve this issue, we need to do the following:

1. Remove that SpecialK DLL check thing that is no longer relevant (Special K: Remove api-ms-win-crt-string-11-1-0.dll check #896)
2. Automate the installation of vc2022 for SpecialK compatibility (even vc2019 doesn't seem to be working)
3. Investigate why/how tracked_files can be missing and how all this ties into SpecialK

Please feel free to correct me if I am wrong on any of these points, or if more needs to be done. It is possible more might crop up, i.e. there may be one or two generally useful tweaks on top of this that can help other games, but that's fine :-)

[sonic2kk]

I did some messing around trying to figure out why vc2022 was failing, and I discovered that SpecialK works out-of-the-box when vcredist2022 (or vcredist2019) is not installed. I commented out the code in STL that automatically runs this as part of the SpecialK installation, and when it isn't executed, SpecialK "just works" at least in testing with NieR:Automata.

I am going to push these changes to a branch that removes this installation, and when it's up, please feel free to test. You should be able to test with a clean prefix (re-create wineprefix should work, just a fresh prefix) and with no SpecialK files in the game folder (this is less important, but I prefer testing from as much of a clean slate as possible). So once this branch is up, when you run that version of STL, all you should have to is enable the SpecialK checkbox. STL won't install dxsetup, vcrun2019/vcrun2022, and it won't do any DLL checking. This means the STL logic only gets the relevant SpecialK DLL and INIs and puts them in the game files.

In short, the solution to the vc2022 issue (and hopefully the tracked_files issue) is to remove the vc2022 installation altogether, as SpecialK no longer seems to require it. Win!

I'll post another comment here when this branch is up, and you can test it the same way you tested the ReShade branch :-)

[sonic2kk]

The PR is up at #898, the relevant branch is specialk-remove-steamworks-install. This should make SpecialK work out-of-the-box, fully resolving this issue!

If you have time please do test, hoping for some good news! Thanks for all your patience with this as I fumble my way around :-)

[CartoonFan]

I did some messing around trying to figure out why vc2022 was failing, and I discovered that SpecialK works out-of-the-box when vcredist2022 (or vcredist2019) is not installed. I commented out the code in STL that automatically runs this as part of the SpecialK installation, and when it isn't executed, SpecialK "just works" at least in testing with NieR:Automata.

I am going to push these changes to a branch that removes this installation, and when it's up, please feel free to test. You should be able to test with a clean prefix (re-create wineprefix should work, just a fresh prefix) and with no SpecialK files in the game folder (this is less important, but I prefer testing from as much of a clean slate as possible). So once this branch is up, when you run that version of STL, all you should have to is enable the SpecialK checkbox. STL won't install dxsetup, vcrun2019/vcrun2022, and it won't do any DLL checking. This means the STL logic only gets the relevant SpecialK DLL and INIs and puts them in the game files.

In short, the solution to the vc2022 issue (and hopefully the tracked_files issue) is to remove the vc2022 installation altogether, as SpecialK no longer seems to require it. Win!

I'll post another comment here when this branch is up, and you can test it the same way you tested the ReShade branch :-)
The PR is up at #898, the relevant branch is specialk-remove-steamworks-install. This should make SpecialK work out-of-the-box, fully resolving this issue!

If you have time please do test, hoping for some good news! Thanks for all your patience with this as I fumble my way around :-)

All done 😄

The game crashed with ReShade and SpecialK enabled together (also took a really long time, due to whatever's going on with #894), so I just disabled ReShade. Like you suggested, I started with clean prefixes, and removed the Special K stuff from the folder beforehand. Game loads quickly, with no external dependencies needed and no errors (that I noticed).

I'm still half-awake, but I'd also consider this a win. Well done! 👏

For fumbling around, your solutions seem to be pretty on point 😁

Thanks for all your hard work! 👍 🙏

[sonic2kk]

That's interesting that using ReShade and SpecialK together took a while, not sure what's up with that but that's for another time methinks.

Glad this solution worked! I'll have to test with another game before merging, and make a couple of other technical changes (there's hardcoded notifier text that should be in the language files to be translatable, and some code cleanup I should do...). There are other technical changes I'd like to implement as well, I feel like there is opportunity to clean up some of the Special K code as well as some of the SpecialK+ReShade code (even if it doesn't work fully well together (for now?)) so it's all a bit cleaner. To be honest there's a lot of refactoring I'd like to do with STL, but since I've touched these areas recently, I think it's a good place to start. Just in case you see some movement around the SpecialK stuff, and feel free to make an issue if something breaks 😅

Thanks a bunch for your active involvement here. Once the changes are merged I'll close this issue, though feel free to comment/re-open if this issue comes up again (or open a new issue if you feel there's a different problem that comes up).

[CartoonFan]

That's interesting that using ReShade and SpecialK together took a while, not sure what's up with that but that's for another time methinks.

Glad this solution worked! I'll have to test with another game before merging, and make a couple of other technical changes (there's hardcoded notifier text that should be in the language files to be translatable, and some code cleanup I should do...). There are other technical changes I'd like to implement as well, I feel like there is opportunity to clean up some of the Special K code as well as some of the SpecialK+ReShade code (even if it doesn't work fully well together (for now?)) so it's all a bit cleaner. To be honest there's a lot of refactoring I'd like to do with STL, but since I've touched these areas recently, I think it's a good place to start. Just in case you see some movement around the SpecialK stuff, and feel free to make an issue if something breaks 😅

Thanks a bunch for your active involvement here. Once the changes are merged I'll close this issue, though feel free to comment/re-open if this issue comes up again (or open a new issue if you feel there's a different problem that comes up).

You're welcome! Thanks for your attention here, as well; your efforts really helped turn this issue around for the better 😆

I feel like this solution should work well, but I'll comment here or in a new issue if I discover any problems.

Thanks again! 🥳 🎉

[sonic2kk]

No problem! I tested two games (NieR:Automata and NieR:Replicant). I didn't test either too in-depth, I did play a little bit of NieR:Automata (had a save file from one of my many playthroughs) but not so much NieR:Replicant (I'll give it a proper play one day, I swear!). They both loaded SpecialK and didn't have any immediate problems which is good enough for me, any problems after that are probably general Linux compatibility problems I would assume.

Now that #898 has been merged, this issue should be fixed in v14.0.20230907-2 and above. I have updated the Fixes section of the changelog to include the fixes made as a result of this issue.

Closing this issue now :-) Hopefully Special K integration works a lot better now!

[CartoonFan]

Thank you very much. I'll give it a test on my end, as well 👍

[sonic2kk]

Also gave the SpecialK wiki page a bit of a makeover as part of this change. It had not been significantly updated since 2021.