Add editor for the EXTN chunk

[colinator27]

With some analysis of the latest GameMaker compiler and some testing of my own, I managed to figure out exactly how the EXTN chunk is formatted. It's actually pretty simple. I may do some more of this for another chunk later on.

Here are the notes I took down, sorry if it's a little messy (indentation should be pretty obvious?):

EXTN: ListChunk
---
<normal FourCC and chunk length in every chunk>

Int32 - Count of (used) extensions
<normal list of 32-bit int pointers to item data, like other ListChunks, using count of used extensions>

Each item:
 32-bit string pointer to empty string: literally "" (just a null terminator), always (hardcoded seemingly)
 32-bit string pointer to extension name (Name)
 32-bit string pointer to class name (ClassName)
 
 List of "extension includes" (files), takes format of List<T>:
  Int32 - Count
  <list of 32-bit pointers to each item, as in List<T> format>
  
  Each item:
   32-bit string pointer to filename (Filename)
   32-bit string pointer to name of function called when game ends, as stored in the extension file, not as referenced in GameMaker, unless they are the same (Final)
   32-bit string pointer to name of function called when extension initializes, as stored in the extension file, not as referenced in GameMaker, unless they are the same (Init)
   Int32 - extension type/kind, value taken directly from project file (Kind)
    0 -> unknown; likely unused
    1 -> DLL
    2 -> GML "placeholder"
    3 -> unknown, but potentially used before GM:S
    4 -> either an included data file, or a "generic placeholder" file
    5 -> JS "placeholder"
    
   List of functions in the include file (if Kind is 2 [which is "GML placeholder"], then this is always empty):
    Int32 - Count
    <list of 32-bit pointers to each item, as in List<T> format>
    
    Each item:
     32-bit string pointer to function name, as referenced in GameMaker (Name)
     Int32 - function ID (Id) - just a unique ID of some kind (very likely based on code analysis, incrementing)
     Int32 - function type/kind (Kind)
      
     Int32 - function return type (ReturnType). Value taken directly from project file:
      1 -> string
      2 -> double
     
     32-bit string pointer to external function name (in the external file); not as it is referenced in GameMaker code, unless they are identical (ExtName)
     
     Int32 - Count of arguments
     Array (NOT a List-type thing) of Int32, count of items is the "Count of arguments" just stated - represents each argument's type. Value taken directly from project file:
      1 -> string
      2 -> double

So, anyway, I guess this issue could be used to track the implementation of the EXTN chunk in the tool/lib, unless this isn't meant to be where this is done.

[krzys-h]

Thanks! That will definitely be useful. According to https://pcy.ulyssis.be/undertale/unpacking-corrected, Altar.NET also has a vague implementation of that structure, but I never looked for it.

If you are looking for something to analyze, do you happen to know what EMBI is? It seems to have been introduced in GMS2 and it was empty when I checked iirc. My guess from the name is EmbeddedItem, although that doesn't really explain what it could be.

[colinator27]

So I did some more reading of decompiled and obfuscated compiler code to attempt to figure out what EMBI is...

It appears to stand for "embedded images", and has something to do with some strange texture page (the texture page is created at the beginning of compilation, and seems to be empty?). No matter what I did in a test project, I literally could not get EMBI to have any real content.

Nevertheless, here's what I found based off of the code:

EMBI - stands for "embedded images"?
Couldn't figure out how to fill the chunk with any data by editing a project and recompiling...
---
<normal FourCC and chunk length found in every chunk>

Int32 - literally just the number 1, always (unknown)

Array of some kind:
 Int32 - count of items in array
 
 Each item:
  32-bit string pointer, some kind of texture page identifier?
  32-bit pointer to something relating to a texture page entry?

It doesn't seem to be commonly used if at all... I saw some strings that said things like "front", "back", and "res" but I still have no idea. My initial guess was that it could contain splash images, but that exports as a standalone PNG file. Unless it's different on other platforms, or there are other kinds of images I'm neglecting.

(Edit: I just wanted to make it clear that the array is not of type List<T>; it doesn't use pointers)

[NarryG]

Just to add to this, despite the fact that Undertale Switch used some other previously unused data, this chunk appears to be unused. There's the name and then 16 bytes, none of which appear to have any significant value

[krzys-h]

@NarryG but we already had somebody try to load another game that apparently does ;)

I guess we will just ignore EMBI for now then.
Please see if 22ef7c1 looks about right, I can't generate a data.win with extensions using trial of GMS2

[colinator27]

Tested 22ef7c1 and it seems to work on an example project I made with an extension with a DLL and GML placeholder, along with some various functions to test enumerators and lists (the data was all correct in the debugger). Re-exporting it seemed to somehow make the chunk smaller by 16 bytes(?) but it seemed to still have the correct data.

[krzys-h]

If you are testing on the latest versions of GMS1.4, the chunk got smaller because the paddings have changed and I didn't implement this yet because I don't know which version exactly it changed on.

[colinator27]

Oh, okay, that makes a lot more sense. But that was testing with GMS2 actually (probably the same reason though). Maybe I should test with 1.4, though I only have the latest one and I bet it's the same.

(Edit: I disabled the ROOM chunk serialization so that I could re-export it.)

[krzys-h]

Yeah, GMS2 has that padding change too. The latest version of GMS1.4 has that too. I know that 1.4.1773 definitely doesn't.
You can get older versions though, I know there are no links on the changelog page but you can just change version number in the URL and the file is still there ;) (and you can use Wayback Machine to get numbers for releases so old that they are not listed at all)

[krzys-h]

And in case they do ever go down, they are now here too: https://archive.org/details/GMStudio1.4.x

1.4.1773 and 1.4.9999 are only 2 versions apart, so I tested both of them and it turns out 9999 is the only one with the new padding, going to implement it now

[krzys-h]

Paddings implemented in 8d54bc2
Leaving this issue open until I add an editor for extensions

[krzys-h]

I wanted to reopen this but then I noticed I never closed it :P
I just had it fail on my own extension https://github.com/krzys-h/GMWebExtension (hehe, guess what I'm trying to make)
It failed on a GML 'placeholder' file which had some random-looking data appended at the end
The complete dump of this include file list item is:

DC 1A 00 00 - file name: GMWebBrowser.gml
D4 15 00 00 - end function: empty string
F1 1A 00 00 - init function: __webextension_init
02 00 00 00 - type: GML
00 00 00 00 - external function list: empty
random crap starts here:
D3 69 A7 09
19 1D EB BD
5C 7A 7A 63
6A FD C9 70

(no, this is not the instructions, the instructions got compiled to a standard script instead)

[krzys-h]

Just tried replacing this data with zeroes, the extension still works fine, the init function got called as it should.
FYI the GML code in question is:

#define __webextension_init
__webextension_set_device(window_device())

[colinator27]

Interesting... Possibly something changed between 1.4 and 2. Could that be some strange padding? Not really sure, I may look into this later if I find the time.

[colinator27]

So I actually managed to figure out what that data at the end was. Basically at the end of the EXTN chunk, for each extension, 16 bytes of data are appended. They seem to have something to do with some "product ID" on the extensions, being some kind of unique identifier?

Anyway I added support for this in the last commit in #30.

[LinXP]

About EMBI
Error (0.1.19): http://i.prntscr.com/gYzrTDzBSS6fsdguETDPew.png
data: http://i.prntscr.com/hHaaaFyRSlyUdydixnTCFg.png

[krzys-h]

@LinXP made it a separate issue #74

[colinator27]

By the way, just fixed a pretty important bug in #76 for this.

[Kneesnap]

The only remaining issue is the lack of a UI? Can I get a confirmation?

[Kneesnap]

My mistake, I did not mean to close this.

Example gave with extensions: Fromto VM

[Grossley]

An extension editor in the GUI now exists, is this issue resolved?

[Grossley]

Closing due to the implementation of an extension editor into UndertaleModTool