Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PCK files include human-readable script code #24716

Closed
miskatonicstudio opened this issue Jan 2, 2019 · 60 comments
Closed

PCK files include human-readable script code #24716

miskatonicstudio opened this issue Jan 2, 2019 · 60 comments

Comments

@miskatonicstudio
Copy link

Godot version:

3.0.6

OS/device including version:

Linux Pop!_OS 17.10 (Artful Aardvark)

Issue description:

One of the people playing our game noticed that the source code is available in the .pck file downloaded by Steam. The game (Intrepid) is an escape room and the source code contains all the codes (answers to puzzles) used in the game. The game was released for free, so this is not a problem for now (only this one person found out the solutions this way) but we'd like to use Godot for future games too, and for that we need to be sure that crucial details will not be available by investigating the .pck file.

Steps to reproduce:

  • build the game
  • investigate .pck file

Example:
screenshot from 2019-01-02 15-01-13

Minimal reproduction project:

@girng
Copy link

girng commented Jan 2, 2019

good discussion in #19790 as well?

@Zireael07
Copy link
Contributor

.pck can be opened? I tried doing it when I found exported a test project that I later mistakenly deleted the source of... and couldn't...

@Calinou
Copy link
Member

Calinou commented Jan 2, 2019

@Zireael07 It's a custom binary format, opening it in a text or hex editor can reveal some data directly as text (as shown above).

There's also third-party tools which can "parse" custom binary formats and extract resources out of them, without any prior knowledge of the format. These tools rely on common patterns found in binary files such as PNGs to extract data.

Lastly, it would be fairly easy to write a PCK extraction tool (either as part of Godot itself, or as a standalone tool).

@bojidar-bg
Copy link
Contributor

Maybe we can encrypt the .pck similar to how .gde files are encrypted? Or alternatively, encrypt each file separately with keys derived from a key given at export.

@girng
Copy link

girng commented Jan 2, 2019

would #24582 be a solution to this issue? (the "compiled" option under the script tab)

@bruvzg
Copy link
Member

bruvzg commented Jan 2, 2019

.pck can be opened? I tried doing it when I found exported a test project that I later mistakenly deleted the source of... and couldn't...

In case it might be useful here's my old PCK extraction tool (AFIAK PCK format was not changed ever) or you can find everything necessary for extraction in core/io/file_access_pack.cpp. It's quite simple format, basically list of file names, sizes and offsets and raw content.

@girng
Copy link

girng commented Jan 2, 2019

@bruvzg okay very cool. thank you so much for sharing that.

i found that godot does still create .gdc files, however, the raw text of the gdscript is still visible inside the .pck file itself. i initially thought if we could see the raw gds in the .pck, we could then extract the .pck and then look at all the gdscript files. that's not the case. they get extracted as .gdc.

now that i think of it, this seems like sorcery

@bruvzg
Copy link
Member

bruvzg commented Jan 2, 2019

Compiled .gdc actually stores all original identifier names XORed with 0xB6, so it should be possible to decompile .gdc into readable code.

@girng
Copy link

girng commented Jan 2, 2019

@bruvzg would that still be possible even if exporting with encryption enabled for scripts? (using a AES key)?

@bruvzg
Copy link
Member

bruvzg commented Jan 2, 2019

@bruvzg would that still be possible even if exporting with encryption enabled for scripts? (using a AES key)?

  1. Search this github repo for "encryption"
  2. Take a look at modules/gdscript/gdscript.cpp#L730-L803 (second non thirdparty search result)
  3. Open exported executable in the disassembler of your choice.
  4. Search for "GDScript::load_byte_code" and "gde" string references (strings literals used in function from step 2).
  5. Search in between and you'll find encryption key in a 10 seconds (even if you do not understand assembler at all, there are only 4 addresses to try).
  6. PROFIT!!!

Encryption won't help much, it's just 5 minute delay.

@girng
Copy link

girng commented Jan 3, 2019

dang @bruvzg. thanks. very interesting.
since the extracted .pck shows the compiled gdscript files (.gdc), it does work, but just not when editing the .pck file itself with a text editor (raw gds code is displayed).

i wonder if it's possible so the .pck file doesn't show raw gds, but the compiled gds instead?

@bruvzg
Copy link
Member

bruvzg commented Jan 3, 2019

Decompiling gdscript turns out easier than I expect, here's proof-of-concept gdc decompiler, PCK extractor and binary <-> text resource converter: https://github.com/bruvzg/gdsdecomp

Clone into Godot's modules subfolder and rebuild engine.

@vnen
Copy link
Member

vnen commented Jan 3, 2019

Any encryption in the exported game will not avoid a persistent person who wants to get the contents. That is because not matter what you do, the encryption key must be somewhere in the binaries.

Also note that the PCK is essentially a file system, with the ideal of being easy and fast to seek to any specific file without having to extract the PCK itself. We couldn't encrypt the whole PCK just because it would require extracting the contents somewhere (which then could be looked by any person anyway). Encrypting each individual file is possible, but still the key needs to be somewhere, and the things are decrypted in memory, so anyone with CheatEngine or the likes of it could check it out.

Compiled GDScripts are not really "compiled", they are just the tokenized form written in a binary format.

@vnen
Copy link
Member

vnen commented Jan 3, 2019

If you really need to avoid people from peeking, the only solution is to host the data on your own server.

@girng
Copy link

girng commented Jan 3, 2019

Compiled GDScripts are not really "compiled", they are just the tokenized form written in a binary format.

i see. very interesting, thanks!
edit: @bruvzg wow, starred. definitely following that

@miskatonicstudio
Copy link
Author

The initial idea (mentioned in the title of this issue) is not to encrypt the files (that would not stop anyone, just delay them, as many people here already mentioned), but to obfuscate the files. While the names of variables seem to be hidden in the .pck file, the string content is very much visible. Is there a way to hide it somehow from the people who simply open the .pck file in Vim ?

@girng
Copy link

girng commented Jan 3, 2019

While the names of variables seem to be hidden in the .pck file, the string content is very much visible.

i tested it. opening a .pck in a text editor shows "uncompiled" gds (even variable names, etc)

@miskatonicstudio
Copy link
Author

That seems like a serious issue for someone who wants to make bigger, professional games using Godot.

@girng
Copy link

girng commented Jan 3, 2019

@miskatonicstudio yeah, even now the hacker will prob stumble upon gdsdecomp 🗡
but, it's impossible to protect 100%. i also agree with vnen.
i also think it shouldn't be that easy to see the source in .pck. in my opinion at least

@JohnSilverLong
Copy link

JohnSilverLong commented Jan 3, 2019

I have been working with C# and i also found it strange when the project.dll is add to pck the classes itself also have to be add to pck for resources that have reference to those classes.

Been working on a custom packer (for mono) that does 3 steps compile than obfuscate the project.dll and after that collect all stuff what should be packed into the pck.
For the script files i create temp empty dummy files and than pack it all with the godot PCK script seems to work so far but i did not found a way to make the resources itself less readable.

@ArdaE
Copy link
Contributor

ArdaE commented Jan 3, 2019

@vnen Do you not put locks on your house doors, because anyone who's willing enough will find a way in? Or not lock your bike because anyone who's determined enough will steal it anyway? Just because you cannot prevent the most determined people does not mean we shouldn't provide any protection.

@girng
Copy link

girng commented Jan 3, 2019

i think vnen was just saying that to make a point that it can never really be hack proof. there will always be some way for a naughty person to get what they want. whether it's using ollydbg, gdscript decompiler, etc.

@ArdaE
Copy link
Contributor

ArdaE commented Jan 3, 2019

@girng I have the utmost respect for the core developers for their contributions to Godot. But @vnen closed another, more inclusive issue (#19790) after stating "... a false sense of security. Obfuscation is not really security." So he's not "just saying that to make a point"; he used that line of argument to dismiss a similar, earlier issue raised by others. I am thankful to him for all his contributions, but I wholeheartedly disagree with him on his stance on this topic.

P.S. I used to hack games when I was a kid (for single-player games, long before the multi-player games existed). So take it from someone who knows how things can be made harder to hack. We're not protecting national secrets here, and a little protection goes a long way.

@girng
Copy link

girng commented Jan 3, 2019

@ArdaE yes, i understand. he's not wrong though, since it's a open source project, it's gonna be impossible to protect because all algorithms will be found in the repo.

however, IMO, i do believe raw gdscript should not be available in the .pck (would solve the OP's issue too), but i don't know if that is possible because my knowledge is limited.

@ArdaE
Copy link
Contributor

ArdaE commented Jan 3, 2019

There seems to be great misunderstanding of what obfuscation means here. It doesn't matter if someone knows the algorithm used to obfuscate script files. Obfuscation is a one-way process; when done right, you can't get back to the original scripts, ever, nor does Godot need to do so to run the scripts. For those who seem to confuse it with encryption, please look up .NET obfuscators.

Further, as I've stated, "impossible" is not a reason to not provide any protection (at the end of the day, nothing short of quantum encryption is impossible to break). The idea is simply to make things difficult. If you don't need protection, that's fine, but I ask you to please not dilute and dismiss the discussion for those of us who care for it.

@volzhs
Copy link
Contributor

volzhs commented Jan 3, 2019

someone is working for compiling gdscript to gdnative.
http://blog.moblcade.com/?p=1
yeah. it looks awesome.
this will solve the issue for projecting source code once it's done.

I hope he makes PR someday.

related issue here #11068

@akien-mga
Copy link
Member

akien-mga commented Jan 3, 2019

To clarify, the use case shown in the OP needs to be solved. End users should not be able to access human readable source code (scripts, scenes) by opening the .pck file in a text/hex editor, nor by extracting the contents of the .pck somehow.

This is easy to do, scripts and scenes need to be converted to binary formats on export (and we're supposed to have an option to do exactly that already - but maybe it's not working as expected?).

Further obfuscation to prevent crackers from getting access to resources and binary code is not planned. It's well established by now that even by spending millions on DRM and anti-cracking measures, AAA studios can only win a few days at most. It's not Godot's job to fight that (and I'd argue that it's pointless to even try).

There's a PR to restore the script encryption feature. As mentioned above, it's not particularly efficient, but if it makes some users feel more confident that their code is "harder" to crack, it doesn't hurt.

To further on the "door lock" analogy, sure, it's good to lock your door to prevent the neighbour's kid to enter and snatch your Switch. That's what should be done by preventing scripts and scenes to be human-readable in .pck files. But you can put 15 locks on a heavy duty door, it's no use against determined robbers if all it takes to get in is to break a window.

There seems to be great misunderstanding of what obfuscation means here. It doesn't matter if someone knows the algorithm used to obfuscate script files. Obfuscation is a one-way process; when done right, you can't get back to the original scripts, ever, nor does Godot need to do so to run the scripts. For those who seem to confuse it with encryption, please look up .NET obfuscators.

While this is true, obfuscation is not planned for Godot scenes nor GDScript. This issue is just badly titled, what it asks for is just to make scenes and code non human-readable.

@ArdaE
Copy link
Contributor

ArdaE commented Jan 4, 2019

At any rate, we can just use AES and sleep sound knowing it is hard to break if you can't find the key.

@bojidar-bg How is encryption any type of protection for code that needs to be loaded into memory? I'll simply let your code load and decrypt itself nicely using whatever advanced algorithm and key you have devised, and then peek into its memory space using a debugger. (Mind you, I guess it would address the title of the issue, and provide some false sense of security).

@akien-mga
Copy link
Member

akien-mga commented Jan 4, 2019

How is encryption any type of protection for code that needs to be loaded into memory? I'll simply let your code load and decrypt itself nicely using whatever advanced algorithm and key you have devised, and then peek into its memory space using a debugger.

Well, that's your "locked door". Experienced robbers can always break your lock or find another way in, but that's good enough to prevent most unwanted visitors.

@ArdaE
Copy link
Contributor

ArdaE commented Jan 4, 2019

Well, that's your "locked door". Experienced robbers can always break your lock or find another way in, but that's good enough to prevent most unwanted visitors.

Touché.

@akien-mga
Copy link
Member

akien-mga commented Jan 4, 2019

To be clear, I think most users will be perfectly happy with this kind of "protection" offered by non-human readable binary files and optionally, some encryption. Sure, it can be broken, and quite easily if you know what you're doing, but it's still good enough for 99.9% of their legit users who have nothing to do in this code (and likely no interest to look into it beyond curiosity).

A real obfuscation feature as discussed above could be interesting for the users who really want this kind of protection, but I'm not sure the additional complexity is worth it for the engine out of the box. Might be worth exploring this possibility as a C++ module, and let interested users compile it in themselves. If it turns out not to make things too complex and a very popular module, we could consider merging it in the core.

@girng
Copy link

girng commented Jan 4, 2019

Might be worth exploring this possibility as a C++ module, and let interested users compile it in themselves. If it turns out not to make things too complex and a very popular module, we could consider merging it in the core.

Your time to shine, ArdaE 💯. I'll be the first one to test it out too, please lmk.

@akien-mga akien-mga modified the milestones: 3.1, 3.2 Feb 21, 2019
@Darkinggq
Copy link

https://www.youtube.com/watch?v=FdNXTFD59Tw

it's very sad bad hacking file open pck (Images, Music, 3D Models)

@n3xxt
Copy link

n3xxt commented Apr 28, 2020

As someone who was planning to use Godot for production and who has supported it via Patreon during the past 4 months, I have found this issue a real no-go deal breaker with the following irreversible takeaways:

(AA)
It is disturbing to see comments from the core GoDot contributors @vnen such as the following

If you really need to avoid people from peeking, the only solution is to host the data on your own server.

Even more so frustrating is the amount and names of upvoters to this.

(BB)
The obfuscation / encryption idea / suggestion / concept is here because GoDot is actually a cross-platform self contained interpreter of GdScript, and why GDScript is not even being compiled into GDNative in export / publish is a mistery;

not so much of a mystery anymore considering that core contributors are of the following opinion:

vnen commented on Jan 3, 2019
...the PCK is essentially a file system, with the ideal of being easy and fast to seek to any specific file without having to extract the PCK itself
...
Compiled GDScripts are not really "compiled", they are just the tokenized form written in a binary format.

(CC)
closing open issues without actually resolving them:

(#19790) opened Jun 26, 2018 [closed, unresolved]
"... a false sense of security. Obfuscation is not really security."

I'm closing this since DRM won't be added and conversion to binary is already tracked by #12452
#19790

(#12452) opened Oct 28, 2017
vnen commented on Jun 30, 2018
#12452

(DD)
The only 3rd-party seemingly active in producing the expected result [compiling GDScript into C/GDNative] has abandoned their project

http://blog.moblcade.com/?p=1

(EE)
Redirecting the issue towards "others have this, too"

bruvzg commented on Mar 12, 2019

Do other game engines have any built-it "assets protection"? Nop, Unity and Unreal forums are full of the similar "Protect game resources" topics, with the exactly same answer: "it's pointless". (And asset stores are full of third-party snake-oil tools).

#19790

And that comment has 5 upvotes.

Both Unreal and Unity do not contain source of the either Blueprints / C# code; Unity has a popular 5-star obfuscator in the asset store plus an ati-hack solution, whereas Unreal actually compiles Blueprint and the end result is ahead of time compiled binary.

(FF)

akien-mga commented on Jan 4, 2019
A real obfuscation feature as discussed above could be interesting for the users who really want this kind of protection, but I'm not sure the additional complexity is worth it for the engine out of the box. Might be worth exploring this possibility as a C++ module, and let interested users compile it in themselves. If it turns out not to make things too complex and a very popular module, we could consider merging it in the core.

Sounds promising but has not been delivered ever since.

(GG)

reduz commented on Jun 28, 2018
3.1 will have an experimental option to convert all text resources to binary on export
#19790
yet,
akien-mga commented on Dec 16, 2018
We ran out of time to implement new features for 3.1, so it will have to wait for 3.2. If it's implemented during 3.2 development, it might still be cherry-picked to a 3.1.x release though.

But this issue is pretty vague, and since the feature apparently exists already (albeit opt-in), the most useful would be to have users testing it and report bugs if they experience any.
#12452

"apparently exists already"

apparently, it is buggy:

[editor]
convert_text_resources_to_binary_on_export=true
reduz commented on Oct 30, 2018
It will work with (maybe) bugs
#12452

(HH)
...and all this is happening ever since the initial feature request dated back to Oct 4, 2016 (today is Apr 28, 2020)
#6709
closed and "merged" to
#12452

reduz commented on Nov 26, 2017
No, no time, will be pushed for 3.1

akien-mga commented on Dec 16, 2018
We ran out of time to implement new features for 3.1, so it will have to wait for 3.2.
#12452

To conclude, it seems that Godot project and their maintainers are rather interested in achieving academic goals than preparing itself for a heavy commercial AAA production.

With all due respect, I am afraid this direction and approach are not sustainable for the real world application use case.

Please tell me, why a person with commercial goals would want to continue supporting this project with further voluntary contributions - directly monetary via Patreon and indirectly with knowledge / product / title releases, while witnessing all this negative attitude towards the real world needs of a commercial publisher/developer/entity?

This is a rhetorical question, mind you.

ref

#6709
#12452
#19790
#37669
#33228
#24582
#24716

@Zireael07
Copy link
Contributor

@n3xxt: The fact that Unity has obfuscators has nothing to do with Unity devs itself, as you mentioned, it's a 3rd party asset.

The fact is, any obfuscation can be beaten, and for a project like Godot that is created in 90% by people doing things in their free time, any time sunk into obfuscation is therefore wasted time.

@mhilbrunner
Copy link
Member

The fact is, any obfuscation can be beaten, and for a project like Godot that is created in 90% by people doing things in their free time, any time sunk into obfuscation is therefore wasted time.

I mean, thats a bit like saying "we'll never get completely lifelike graphics, so any effort spent improving graphics realism is wasted".

This reduces the question for some security to an all-or-nothing state, which doesn't reflect reality.

In security, you have the concept of threat modeling, or more simply "Which kind of attack are you trying to prevent?"

Nobody is saying "lets make Godot games unhackable for nation-state agencies".
Not even "lets make Godot games unhackable for advanced users".

The requests I've read are simply "please prevent even the most unskilled player being able to open my game in a text editor and seeing the whole code reproduced plaintext, as well as all assets and secrets".

This is a quite possible and IMO worthwhile goal.
A goal that Akien above acknowledged.
It's simply a question of time and resources, of when it will be done, not if adding this feature is wasted time.

@n3xxt
Please have some patience, Godot isn't a commercial effort. Of course you're free to use any other engine, of course you're free to not donate if you don't like the direction of the project.

But donating or using the engine won't magically make things that much quicker.
If this is really the only dealbreaker for your commercial game efforts with Godot, sponsor a dev to work on this feature for you or hire an inhouse dev to add this feature.
Or go with another engine. :)

@Zireael07
Copy link
Contributor

The requests I've read are simply "please prevent even the most unskilled player being able to open my game in a text editor and seeing the whole code reproduced plaintext, as well as all assets and secrets".

Oh, I wasn't aware it was such a simple request. Because most people asking for obfuscation were interested in commercial games, I thought they wanted commercial level obfuscation. Such a basic obfuscation could probably be achieved easily with Base64 or a simple cipher. (Unfortunately, my c++ skills are nearly zero :( )

@bruvzg
Copy link
Member

bruvzg commented Apr 28, 2020

For the reference, encrypting everything is easy (and probably really slow with the large PCK), here's simple implementation: #38308

@Xrayez
Copy link
Contributor

Xrayez commented Apr 28, 2020

@bruvzg I feel like this could solve most use cases. If performance is an issue, perhaps it would be reasonable to have an option to separate "sensitive" resources like scripts, shaders etc. into their own encrypted PCK file, while also having another plain PCK file for assets. Namely. the decision on what resources get encrypted would be on a game developer rather than Godot core devs.

If no files are marked for encryption, then nothing changes, just export everything the way it is now.

@girng
Copy link

girng commented Apr 28, 2020

Hi @n3xxt!

With all due respect, I am afraid this direction and approach are not sustainable for the real world application use case.

I'm releasing a commercial game. I would just like to add: Take a look at Path of Exile. Their content.ggpk has been "cracked" if you will, and every single file has been extracted. There is even a public website that lets you view its data. The only "truly compiled" code is in their Client.exe. Which could still be disassembled with ollydbg and other methods, IIRC.

What I'm trying to say is, view their content.ggpk as Godot's data.pck. Obviously there are public unpackers for .pck, but I think it shouldn't be a negative for real world application use cases. I hope you have just a little bit more patience, maybe @bruvzg's solution is what you want!

@vnen
Copy link
Member

vnen commented May 4, 2020

Addressing issues that pointed to me, just to clarify.

(AA)
It is disturbing to see comments from the core GoDot contributors @vnen such as the following

If you really need to avoid people from peeking, the only solution is to host the data on your own server.

Well, it is true though. You can make it harder to figure out, but only way to completely avoid it is to not have it in the binary at all.

(BB)
The obfuscation / encryption idea / suggestion / concept is here because GoDot is actually a cross-platform self contained interpreter of GdScript, and why GDScript is not even being compiled into GDNative in export / publish is a mistery;

GDScript predates GDNative by many years. We do have plans to compile GDScript to native code eventually.

not so much of a mystery anymore considering that core contributors are of the following opinion:

vnen commented on Jan 3, 2019
...the PCK is essentially a file system, with the ideal of being easy and fast to seek to any specific file without having to extract the PCK itself
...
Compiled GDScripts are not really "compiled", they are just the tokenized form written in a binary format.

Those are not opinions, those are facts. Like everything else, whether or not we should reconsider is open to discussion.

We will have a proper compiled form in GDScript for Godot 4.0 (though that won't really hide plain text strings in the source file, they can still be viewed in a hexdump like in pretty much any other language). You'll still be able to encrypt the resulting file (as it's already possible in stable releases for quite a while).

(CC)
closing open issues without actually resolving them:

(#19790) opened Jun 26, 2018 [closed, unresolved]
"... a false sense of security. Obfuscation is not really security."

I'm closing this since DRM won't be added and conversion to binary is already tracked by #12452
#19790

It's pretty normal to close issues are like an existing one (duplicates), since there's no need for two reports of the same problem. You can see that this very issue here was never closed, and that one could be considered a duplicate of this, so there's still a tracked issue.

Please tell me, why a person with commercial goals would want to continue supporting this project with further voluntary contributions - directly monetary via Patreon and indirectly with knowledge / product / title releases, while witnessing all this negative attitude towards the real world needs of a commercial publisher/developer/entity?

People should not expect to dictate the project goals just because they are donating. Not even the companies do that. What companies do is to hire their own developers to add features to the engine that they are missing, instead of waiting for voluntary contributors to do the work.

@miskatonicstudio
Copy link
Author

I've seen a lot of updates recently, so I'd like to chime in. The solution of the original problem from this issue was suggested by @mhilbrunner : it is not about making the data completely unreadable by experts, it is just about being unable to read the crucial bits of PCK files with a normal text editor ;) Any obfuscation would prevent it, and I'd consider it a solution.

If users start reverse engineering the game, then little can be done to prevent it other than (as mentioned) storing the binary on the server. But the original problem was just that a normal user can open a PCK file in Vim and read the solutions to the puzzles.

@ChristianB84
Copy link

Variable and function names don't have to be human-readable for executing scripts, so there is no need to make them accessible to everyone. This is not comparable to not having a lock on your door, it's more comparable to sending everyone a map of your house that includes where to find all the valuable stuff and telling them when you're not home. You don't want that information out there at all, not even in pseudo-"encrypted" form with the key included in the binary.

@vnen
Copy link
Member

vnen commented Nov 18, 2020

I believe this is solved by #38308. You can encrypt any file in the PCK. See https://docs.godotengine.org/en/latest/development/compiling/compiling_with_script_encryption_key.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests