The need for a better relationship with GDScript when using other languages

[ghost]

GDScript has the role of being a fast to write, easy to use language. It does well in that area and it's sufficient for most game code. But sometimes you need a faster language (to do procedural generation, for example). The issue is that none of the languages available right now can interact with GDScript very well.

C#
Calls between C# and GDScript are slow; integrating C# into GDScript code means having to translate many of the advanced data structures of C# (like structs, multidimensional arrays and so on) to simpler data that GDScript understands which is time consuming; C# is also vastly undocumented in the sense of interacting with Godot types, which makes this job even harder.

C++/GDNative
Hard to setup and maintain; runs in the same problem as C# which is having to translate data structures (like structs in C++) to GDScript; even less documented than C# (to setup and write).

Possible solutions

• Making GDNative as easy to setup and write as GDScript (or just easier at least).
• Vastly documenting which X language types are which in GDScript (which is not really feasible: the main documentation is already out of date with the current state of the engine, imagine the several unofficial languages).
• Automatic conversion of common types (Automatic conversion from C# array to GDScript array #26104)
• Adopting official prefixes for Godot's types in other languages, making clear what's from Godot and what's from the language. So Godot's Vector2 in C# would be GDVector2 for example.
• Adding some more common data structures to GDScript, like structs and multidimensional arrays, so they can be easily translated.
• Improving GDScript call speed, allowing better communication between itself and other languages.

[neikeq]

Calls between C# and GDScript are slow

That's because GDScript calls are slow. You will find the same problem for pretty much any other language trying to interact with GDScript, as you noticed is the case with languages integrated with GDNative.

integrating C# into GDScript code means having to translate many of the advanced data structures of C# (like structs, multidimensional arrays and so on) to simpler data that GDScript understands which is time consuming

There is nothing that can be done about this except perhaps some little improvements here and there. The same way you cannot bind every engine function to be usable from GDScript. You will find functions in the engine that are suffixed with _bind (or similar) that act as wrappers to be callable from GDScript.

• Improving GDScript drastically (especially in for/while loops and array handling which are terribly slow right now) rendering the need for a performant language non existant.

GDScript performance is most likely already tracked by another issue.

• Making GDScript a hybrid language, where using optional static indicators increases performance.

This is already implemented, and further improvements are most likely also already tracked by another issue.

• Making GDNative as easy to write as GDScript, integrating it into Godot's script editor.

I don't know what you mean by "as easy to write". I will assume you refer to the workflow in general for GDNative-powered language bindings. This is not really a GDNative issue. Some improvements may be done to GDNative/NativeScript and the Godot editor, but at the end of the day it depends on the people working on those language bindings.

• Replacing GDScript with a static and performant, yet easy to prototype custom language.

This is unlikely to happen. Instead, expect mode improvements on optional-typed GDScript and the possibility of implementing JITing or compiling AOT to improve performance.

• Creating another language alongside GDScript, one that focuses on performance.

This is not going to happen. Use C# or GDNative language bindings.

• Vastly documenting which X language types are which in GDScript

Can you give an example where this, if not documented, is not trivial to figure out?

[avril-gh]

Most things on the list are, (...)

Because this topic is as old as godot, and it was allready discussed for years. There has been many languages implemented and droped and to put it short, after long time and many implementations GDScript has came out by evolution to be best approach from many angles.
so after most questions has been answered by life, speed is always the concern, and nowaday's disscussion are "to JIT or not to JIT", ect.

[QbieShay]

Maybe it is possible to have a plugin that ships with whatever compiler your gdnative script needs

[avril-gh]

i just been moved by your other issue #25753 which sounded like "GDScript is bad, everyone hate it, lets abandon it". I personaly like it very much, its very good and i know many people think that way too.
Sorry if i missunderstand. My mistake.

[girng]

neikeq's response is really good

I meant being able to create a C++/Nim/whatever the same way you create a GDScript file, and editing it in the same way too; using Godot's built-in script editor.

yeah, i think you just want "official" godot editor executables for their respective language bindings? for example, nim, rust, crystal, etc? you have to find developers who excel in that language and godot to maintain those AFAIK. there's been some talk in crystal's gitter about crystal bindings.. but IMO the core
devs have much bigger fish to fry

but really, this "gdscript is slow" really hurts.. my heart sinks every time. gdscript is fast enough for 98% of games out there! and plus, the optional static typing and future JIT optimizations will make it perform even faster in the future. there is about zero reasons not to use it. really, it's such a blessing @mateusak, take advantage of it! time is gold

[girng]

@mateka i see. sorry if i misread 👍. i know how ya feel. the vid tutorial on gdnative does seem a bit complex now skimming over it. however, it's low level stuff so prob no way around that ;O

[vnen]

I don't know why you need to pass so much data to GDScript. Why do you need a multidimensional array for? If you are going to loop that in GDScript, you are already losing (some of) the performance you gain from using a faster language.

If you need to arbitrarily access data, you need to use Godot data structures (Arrays and Dictionaries), to take advantage of the easy access. I honestly don't think you need custom data structures in GDScript. You should use the other language to process that (since you are already using another language anyway). Passing data between languages should be reduced to a minimum.

For instance, if you are using C# to generate a map, you should not pass a bi-dimensional array to GDScript and construct a TileMap there, you should instead create the TileMap directly in C# and pass only the resulting Node, which can be easily accessed in GDScript.

Also, you technically can create a C# object and keep a reference to it in a GDScript variable, from which you can access properties and methods. So you should keep the data in the faster language and only expose an API for GDScript.

[vnen]

Making GDNative as easy to write as GDScript.

This is vague and technically impossible. GDNative is not a language, so you really can't "write" it. It's just a C-compatible interface.

We all agree docs should be better, but I don't think there's any way to make GDNative as easy as GDScript (and also don't think it's necessary). If you want to write C++ you need to know how to deal with a compiler as well, there's no way around it.

[QbieShay]

@mateusak I think for multidimensional arrays you can use a 1D array and have a get_2D(row, col, size_row, size_col) and get_3D(x, y, z, size_x, size_y, size_z) functions implemented onto array that does some index magic for you. Possibly you also have append2D and append3D that appends in a coherent order for the way that is then read from the other functions

[karroffel]

There's no place right now that functions like an asset store for GDNative. Just making a place to allow people to share their GDNative projects organizedly, with the ability to list dependencies and such, would go a long way.

I think I know what you mean here, but sharing on a "function" level will lead to a very high fragmentation of functionality and suddenly your project depends on 34 GDNative libraries which all do just one thing. You don't want left-pad or event-stream in Godot's ecosystem.

There actually is a place to share GDNative libraries - the Asset library. There are at least 4 GDNative libraries in there for providing VR drivers to Godot, there are also at least 2 libraries to bring Python support into Godot.

A GDNative setup has to include any binary dependencies it has, it's just that generally those dependencies are vendored into the package.

I think the general usage of GDNative should not be "here's this library which does one thing", because the management overhead of that will be quite big once you have two or three of those.

I'd say the best way to use GDNative is to have one library per project that you write (and any possible third party GDNative libraries (like VR)) and use Godot types as much as possible.

For example, as @QbieShay pointed out, you can have your generation of procedural data done in a GDNative class, but instead of using int filed[HEIGHT][WIDTH] inside you can use PoolIntVector field and have accessor functions that map your 2D grid onto that 1D array. Because it's a PoolIntVector you can painlessly pass it to Godot with basically 0 additional overhead, while inside of GDNative you can manipulate it using raw pointers.

[vnen]

Passing nodes around just means I have to duplicate everything I want to pass to GDScript. I'll pass a TileMap node to my TileMap node so it can rewrite it...

My point is that you shouldn't need to rewrite. Just create the TileMap in C#, pass it to GDScript to add to the tree (or even add directly from C#). As others have said, you should focus on structures available in Godot itself when you want to pass data around.

[QbieShay]

Yes, it is not elegant indeed. That said, I think it's super great that Godot dares to support multiple different languages, I guess making things nice to use is hard when so much flexibility is required.. but anyway, I believe something might be done to improve the workflow. Not sure about improving the calls performance, but maybe the workflow for cpp could be indeed improved

[karroffel]

@mateusak It's not like people don't care to publish, it's having something to publish in the first place. In the case of the Rust bindings it's mostly a problem of them not being ergonomic enough that anyone would use them on a day-to-day basis. For nim I guess it's just size of target audience and need for something like that.

Currently the only things in the asset library made with GDNative are things that would usually be part of the engine, like VR drivers or scripting language implementations. If you just want a function or two then pulling in GDNative is just overkill IMO, and at that point you can write this function on your own and use GDNative for more things as you already have it set up.

Using continuous memory to simulate multi-dimensional arrays is actually the ONLY way to do them, it's just that languages hide that from you. That's because flat memory is all there is in your system, everything else is just abstraction. Now the problem is if you have one language that has that abstraction (C#, C++, Rust, ..) and a system that does not have it (Godot's Variant class + GDScript). In that case there's no way for you to do it, as there's just no way to express it.
So here we are not fighting for "we need a statically typed language that's fast" but for "there is a need for more expressive types".

In which case I agree with you, I'd love to see more expressiveness in GDScript, but I also don't expect that to happen since it doesn't really align with its goals.

Good language design is not about implementing cool features, but to create a coherent system.
C is great because of that, C++ isn't, Rust is, Haskell is, many other languages are, and I would say GDScript is too. (That doesn't mean I don't want typed arrays 😋 )

[neikeq]

... Calling C# methods from GDScript takes painfully long (_for reference calling an empty method takes 0.05s, ...

Are you sure about this? 1 ms for a simple call would already be pretty bad, but 50 ms? Can you provide an example benchmark were this can be confirmed?

[QbieShay]

@mateusak have you tried cpp instead? Apparently c# is a 'special case' because the objects can't be easily translated due to csharp's own architecture.

[mnn]

If you want to write C++ you need to know how to deal with a compiler as well, there's no way around it.

Current state (at least beta3) goes way beyond "just know how to use gcc". There is currently no "preset", no fully working template to get GDNative with C++ running in a few minutes. I spent literally half a day trying to make demo working...

We all agree docs should be better,

Yes, docs with C++ demo are broken and often confusing - on many places it is not clear about what directory the docs are talking (I think bindings repo had more clear guide, but I didn't find that out until I no longer needed it).

but I don't think there's any way to make GDNative as easy as GDScript (and also don't think it's necessary).

There should be easier way to get it set-up. Why are not cpp bindings (binaries for all platforms) for each official version prepared? (Seeing them even for beta releases would be nice.) Why couldn't we have fully working template project to download? Why at least all needed commands for first time are not in some script to simply run, not having to copy&paste many commands in different directories manually? I don't know scons at all, but isn't that thing a build tool? Coming from JavaScript world, where are scaffoldings/generators for many FE frameworks, even smaller ones, this is a giant contrast. It is literally two commands to set-up a new project: install package from npm and run generator which asks you few questions and generates everything, so you can just run dev server and start developing in literally under a minute from start.

In the end I found some random demo, which had broken installation guide (I had to manually download all dependencies, I really don't like git submodules). Yeah, but that whole process of setting-up GDNavetive with C++ took me like 4 or 5 hours, just to get a demo running...

I understand beta can have some bugs (like #25812), but why marking it as beta, so users can start testing it, when docs are not finished? I am not saying polished, but at least in a state when they reflect current version? I don't know, this process doesn't seem to me thought-out very well.

I am sorry if this sounds too negative, but as you can tell from the amount of wasted time, it was quite frustrating for me.

[mnn]

@QbieShay I wouldn't dare to use C++ to write procedural generation. I can barely detect errors in C#. Now say, Rust, I would give that a try. But unfortunately it doesn't seem very complete.

What a coincidence, the reason I am trying C++ is to finish benchmark for voxel related tasks. So far I got only data generation part for GDScript and C#. GDScript's performance was bad (as expected, slower more than by 11 times compared to C# in Godot), but C# + Godot was a nice surprise, since it was slightly faster than Unity + C# (bench does not count startup times, only measures time of the method which generates voxel data). Possibly Mono version difference? I also noticed perf regression of GDScript between stable and beta by ~1.75%, but my sample size is quite small (several runs with increasing "difficulty", but only one run per each difficulty; highest took GDScript over 2 minutes, C# was slightly over 10 seconds).

Now I am fighting a bit with C# <-> GDScript interoperability (#25825), probably will have to limit usage of GDScript as much as possible (in bench and game as well, if we decide to use Godot), because passing data is slow and works correctly only in some cases. Originally I wanted to write just world gen and mesher in C# or C++ (only if significantly faster, I am a C++ noob) and rest in GDScript, but that won't probably be the case 😞.

[piratesephiroth]

I actually don't really see a reason to insist in using and improving GDScript (other than C# support being in alpha state)
All it does is save you from writing a bit of code. Also said code is full of ugly snake case names and indentation for blocks.

C# is already extremely simple and the performance is way better, so to me it's really a no-brainer.
I think Godot should just switch over to C# scripting.

[Zireael07]

1. Why is this not tagged mono?
2. I am thinking of moving more performance intensive parts of my biggest project to c#, and I have no idea how to make gdscript and c# play ball with each other (array problems notwithstanding). Can we maybe get a note on that in the docs?

[neikeq]

1. Why is this not tagged mono?

Why is this an issue with the mono module? If there is any improvement that can be done in the mono module in this regard, please open a new issue for that specific thing.

[Zireael07]

A better relationship between GDScript and C#, as listed here, may involve improvements to mono module?

[neikeq]

@mateusak You can pass a RandomNumberGenerator instance between C# and GDScript. I don't understand what is your issue with it.
As of ce67808 you can also use the default rng from C#. The equivalent of the rng global functions.

[neikeq]

@Zireael07 I already talked about this in my first comment. There is not much that can be done here, and if there is any specific thing that can be done in the mono module it should go into its own issue.

[neikeq]

this means it's impractical in almost any of the scenarios where you'd need the Godot generator

Why...? Like I said you can re-use the same RandomNumberGenerator instance.

[neikeq]

I don't think I follow. What is the problem with RandomNumberGenerator or the default rng? What do you mean it's supposed to be a global generator? Sorry if the problem you're describing is obvious but I just don't understand it.

[Zireael07]

Just as an example, if I have a Math.cs which contains a Arc() function, how do I call said function from Gdscript? Most importantly, scripts where I need it already extend Godot classes, e.g. VehicleBody or Node2D.

[neikeq]

You can't access it. You can only access Godot objects from GDScript (Math would need to inherit Godot.Object).

[Zireael07]

And if it inherited Object, would it be possible to access?

[neikeq]

Yes. You would then access it as you would do with other GDScripts.

[vnen]

I believe this discussion has outlived it's usefulness. There's nothing really actionable from the discussion, and we usually don't like issues with multiple points since it's tough to track what has been done and when to close it.

For stuff that is still missing, please open a proposal at https://github.com/godotengine/godot-proposals