Class (instead of file) based script assignment in editor

[Xydium]

Following some discussion with Karroffel and getting neikeq's GodotSharp module functional, I had an idea to make script/class organization easier, by having Godot automatically parse the project files for all classes/scripts and having a popup that lists them (with search bar and filters) to select a class rather than pointing a given node to a file directly. For example, on a Node2D selecting a subclass would make this window appear:

For GDScript/VisualScript, the class path could either be the file path in dot notation, or to further reduce the dependency on the file system's organization, GD/VS could have namespaces added or simply always appear top-level with no namespaces.

XCode operates similarly where the selection menu for a custom subclass on a view/component will list all of the classes in the project that inherit from the required type.

[bojidar-bg]

I partly agree -- especially for C#/DLScript where the classes are known beforehand, it would be nifty to have this; but for languages as GDScript and VScript where every file is a class, and there is no global "polling" of all files, it makes no sense at all (even less with the scenes idea, maybe you wanted to have just a "dotified" path there?)

[Xydium]

Scenes.MainScene doesn't refer to the main scene in the project. It would be equivalent to a folder structure set as res://Scenes/MainScene/TestScript.gd. The Dot notation is to be graphically consistent with namespaces even if it is technically a filepath. It is mostly for C# though where one file can contain multiple classes.

[neikeq]

Two possible issues:

• While this makes sense for languages like C# where there is totally no file/class relation, it would affect usability for GDScript (and other possible similar languages. Java for example has a file/public class relation). I want to select GDScript scripts with a file dialog not by searching a qualified name. And since we don't know the script type the user will choose in advance, we cannot display different loading forms.
• How are you going to open the script in the editor? A Godot script MUST be a file (not a qualified name), that's how the API works and I don't think it's worth changing it. This one doesn't apply if you only intend to use the qualified time at load time and instantly get the actual file with a new virtual method in the ScriptLanguage API.

This is one of the solutions I had in mind for solving the namespace detection issue with C#: neikeq/GodotSharp#9, but I soon discarded the idea for the reasons above (we are just going to use a simple parser to find the namespace).

[Xydium]

Fair concerns, ideally there would be no path association with script loading, so to assign a script you would simply point it to a script with the correct inheritance for the node it is being attached to, and if the file were moved it would update the reference.

Maybe a better alternative is to be able to sort scripts by what they extend, such as having an option to search res:// for scripts that extend KinematicBody2D, with the option to include those that indirectly subclass, such as with extends "MyScript.gd".

[karroffel]

@neikeq if you're using a parser to find the namespace, does it mean F# can't be used anymore? I liked that the code just had to live in the assembly and be CLI compliant, with this you kick out all the CLI compliant languages that run on Mono, don't you?

[neikeq]

@karroffel The current behaviour is that it iterates through the typedef table until it finds a class that derives from GodotEngine.Object and matches the name of the script file. The problem with this is that you can't have NamespaceOne.Player and NamespaceTwo.Player, making namespaces pointless. This will be the default behaviour for any language that has no parser, including F# (in other words, it will be the same as now). Alternatively you can write a parser for it (see 4613cb7).

[touilleMan]

On my godot python binding, I've deal with this problem following the zen of python: "Explicit is better than implicit"

So to be available from Godot, a Python class should have been explicitly decorated as exposed. Obviously only one exposed class is allowed per file so it feels just like GDScript.

from godot import exposed

@exposed
class Player(Node):
    def _ready(self):
        pass

While it works great with dynamic language where a lot can be done at load time, I'm not sure such a thing could be achieved on C#...

@neikeq I don't know much about C#, is there a __file__ attribute (like the C one which the preprocessor replaces by the file path at compile time) ?
If so you could implement this in the decorator to end up with some classes flagged as exportable and with there associate file path. From there it would be trivial to run your scan between the compiled class and the source file.

[neikeq]

@touilleMan I tried that already. I was close to achieving this by using the CallerFilePath attribute. This was an attempt: https://gist.github.com/neikeq/db685eda63e558d207abf12dececa51b
The problem was that the path provided by CallerFilePath is absolute, and considering it's at compile time... I thought about saving the path to the project somewhere and then using it to fix the script paths when running the game, but I don't think anyone would like the idea of exporting their dev directory path with the game.

[touilleMan]

@neikeq at least you made me puke my tea with your commit message 😆

I guess the simplest way that remains is to force user too keep namespace and file path the same for the exposed classes. Potentially providing a configuration field the user could overwrite to specify by hand a custom path for the class.

[NathanWarden]

What I think would be a better generalized solution for all resources, would be for each resource whether script or mesh to simply have a unique id that would be stored in an import file. So, each resource would have the unique ID instead of a file path. The unique ID could then be associated with each asset. So if a script of any type, or a material, texture, audio file, etc would no longer be dependent on a specific path.

This would require Godot to scan the project when you load it and create a Map of each asset ID and it's path, and then when a script slot has UniqueID in it, then it will internally know that ID X belongs to path X. If an asset is moved, then it's import file is also moved and the Map is updated with the new path.

This would allow all resources to be moved around and literally nothing would ever break.

edit: I think I'll add this as a feature request.

[NathanWarden]

Now, as far as having to type in the fully qualified script name, I think this is a bad idea that sounds very good on the surface. For instance, what happens when you decide a script that you've used in several places in your project has been poorly named?

You have to either:
A) Go to each and every place where you've used it and correct the name.
or
B) Just leave it as it is.

With my solution in the previous post you:
A) Rename the file
and
B) Rename the class in the file

and then everything is auto-magically updated for you since the Unique ID didn't change.

[Xydium]

@NathanWarden That's actually closer to what I had in mind: a system that eliminates the dependence on specific files and allows some degree of automatic refactoring. My original post was more about filtering down the options for script files when selecting them, but in general the class/file system Godot uses is tedious to use in cases like accessing another scripts enums due to the lack of static variables, etc.

[NathanWarden]

@Xydium Ah, I think I understand, like having a search feature that would help narrow it down instead of digging through directories or something like that? I think that would be a great idea!!!

I opened a feature requiest for using unique IDs... #15673

[karroffel]

I would still very much like a system like this.

It would be important to not break old code, so what about a system like this?

---

Registry

A script registry would scan the project for all detected script files and queries the appropriate ScriptingLanguage for any "exported" classes. An "exported" class would be identifyable by name.

There could be a "low level" API for querying and even procedurally inserting exposed classes.

By accessing the ScriptRegistry singleton it would be possible to get Scripts by name for example.

Namespaces

Since "exported" classes would need to have a unique name namespaces might be optionally needed, using a default "empty" namespace would work too.

GDScript integration

With this design it wouldn't be required to be limited by the file-based architecture that's currently implemented in Godot.

To not break old code, the ScriptRegistry integration should be optional and old code would still behave like it does.

What could be done with this registry in place, is that multiple classes (or in the case of GDScript - inner classes) could be put into one file.

example of usage

fruits.gd

# no "extends" here, will default to Reference then

export class Fruit:
    var name
    func _init():
        name = "some fruit, has not been given a name yet"

export class Apple extends Fruit:
    func _init():
        name = "Apple"

export class Pear extends Fruit:
    func _init():
        name = "Pear"


export class FruitBox:
    var fruit
    
    func set_fruit(f):
        fruit = f
    
    func get_fruit():
        return fruit

# for namespaces this could be used
# 
# export("fruits") class ...
# or this for nested namespaces maybe?
# export("fruits", "tasty")

some_other_file.gd

extends KinematicBody2D # maybe a Player controller?

func on_hit_by_object(obj):
    if not obj.is_in_group("has_fruit_attachement"):
        return
    
    if obj.get_fruit() is Pear: # inheritance check via name instead of preload()
        show_dialog_bubble("Why are you throwing pears? Their shape doesn't even hurt me on impact")
    else:
        show_dialog_bubble("Ohhh you hit me with a ", obj.get_fruit().name)

func on_fire_button_pressed():
    var fruit_box = FruitBox.new() # creates class by name, not preload()
    fruit_box.set_fruit(Apple.new())
    shoot_box(fruit_box)

GDScript would check if an identifier exists in the registry, like this ScriptRegistry.get_class("SomeClass", ["fruits", "tasty"]) and if it exists it can be used. This is similar to how GDScriptNativeClass works, it checks if an identifier names a class known by ClassDB, Autoloads work the same way.

---

TL;DR:

• add a ScriptRegistry singleton that scans for exported classes
• No big changes needed for this to work
• existing code will still function
• purely optional
• namespaces to allow duplicated names of classes
• referring to a class would look like accessing a Godot internal class

WDYT?

[Xydium]

@karroffel That would be much more convenient, especially if it works with autocompletion. Static variables would be a nice inclusion too, especially for enums.

[willnationsdev]

Just discovered that I've been working on a similar project. Hopefully somebody else hasn't already been working on this too. XD Gotta figure out how we're gonna coordinate on this mess, cause I'm trying to create one branch that handles this Issue and another.

[reduz]

I think gdscript needs no changes for this. Entire system does not need go become more complex for differences with other systems, or an apparent ease of use that is not.

…

On Mar 9, 2018 19:35, "Will Nations" ***@***.***> wrote: Just discovered that I've been working on a similar project. Hopefully somebody else hasn't already been working on this too. XD Gotta figure out how we're gonna coordinate on this mess, cause I'm trying to create one branch that handles this Issue and another <#6067>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7402 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF-Z20W_kL-5IcDwgujmNV3BuESUIPn9ks5tcwOzgaJpZM4LYlq8> .

[willnationsdev]

@reduz So then, if we wanted a feature of this sort, it would have to be a feature that is exclusive to and part of the /modules/gdscript/ module? Or you just don't want GDScript to support this at all because of the added complexity?

[reduz]

Also selecting class from a global list is imho a terrible idea. You still know better where your file is than where your class is on a large project.

For C#, I still think the best that can be done is marking the main class in the file with some tag, I fail to see why doing this is not possible.

[reduz]

@willnationsdev I see this pointless in GDScript too. I don't think it's needed.

[reduz]

In other words, going from script oriented to class oriented is a definitive no from me.

[mysticfall]

@reduz

Also selecting class from a global list is imho a terrible idea. You still know better where your file is than where your class is on a large project.
...
In other words, going from script oriented to class oriented is a definitive no from me.
I beg to differ on this one, at least when it involves C# support.

I can understand how GDScript might not need any such changes, as it is a special purpose language which means anything written in that language would presuppose a Godot specific workflow.

But we can't assume the same with C#, or potentially other similar general purpose languages that are not used as scripts. Such a language like C# is never meant to be used by referencing the sources.

It was simply an unfortunate side effect of introducing the C# binding after GDScript - which is more suitable for such a script oriented approach - rather than being a conspicuous choice to provide an optimal workflow for C# users as well.

C# already has a well established convention and mechanism to resolve and manage dependencies which is all assembly/class based rather than script based, and it's the way what most of its tools and libraries presupposes and what the developers are accustomed to.

And unlike GDScript, there exist a vast amount of third party libraries that can be referenced from a C# project in their binary format. Godot currently supports referencing those dlls from the main *.csproj file, but when it comes to addons we still need to find a way to make it possible for a C# project to reference C# addons with its own binary dependencies both within Godot and external IDEs.

If we are to adhere to a workflow that is alien to a typical C# development process, it would make the development set up more difficult and common practices like building your addon from a popular IDE and distributing it as a NuGet package would provide only a limited usage in Godot, because any classes contained in such an assembly cannot be referecend within the editor.

And I disagree that providing a global class list to be a terrible idea. It's the way most of the IDEs supports a similar functionality (i.e. selecting a target class for a unit test), and with auto completion or namespace support, it's pretty easy to find a wanted class that way.

Actually, in most cases, it's much easier than referencing them by their source locations. Developing a C# project can involve dealing with multiple libraries and classes can be located in a deep namespace hierarchy. So, doing a quick search by their names is often much more convenient than having to navigate between different file system locations.

And it's also how Unity supports its C# binding as well, so I think it'd be better if we also support such a workflow that is more common to typical C# development projects including those involving Unity, rather than shoehorning a binary based language into a workflow designed for a scripting language.

[eon-s]

With optional typing, something like syntatic sugar but for classes can be useful to make better code.

Writing

func a_function(load("res://some_directory/sub_directory/character_weapon.gd"): weapon)
  var load("res://some_directory/sub_directory/character.gd"):character = load("res://some_directory/sub_directory/character.gd").new()
...

may look ugly, probably 🤔

[willnationsdev]

Idk if the static typing was meant to include scripted types (not privy to any of that debate). If it does though, I'm guessing that in that case we'd have to use something without identifiers of any kind, like...

const MyType = preload("res://dir/dir/another_dir/MyType.gd")

func my_func(p_obj: MyType):
    # do something with p_obj where (p_obj is MyType) == true  

[eon-s]

@willnationsdev I know, also the whole thing an be made by a plugin that registers every new GD file in a singleton (writing consts in the script) for global access, but something integrated may be better.

[willnationsdev]

if you want a script as a global class in GDScript, this is really easy to do and I thought about it many times, but it would be GDScript-only. I'm sure it would aid in usability for more complex projects.

@reduz Wait, really? What I suggested would be okay? I don't see how that would be very different from registering names / namespaces in a HashMap for each language though. The entire concept of a TypeDB that could coalesce names and namespaces for all types of scripts and/or scenes is basically doing the same thing as using singletons, only the singletons route is LESS performant, not more (as far as I'm aware):

1. The editor creates a singleton node that has the same name as your project name.
2. If you add a plugin, the editor creates a singleton node with a name for the plugin (defined in the plugin.cfg). All of the generated singleton names are UpperCamelCase.
3. Any time a script file is saved, a new constant with an UpperCamelCase name is automatically added to the script that represents the singleton node. (would use the singleton that is most relevant, i.e. a script file in a plugin directory would go into that plugin's singleton script, etc.).
4. Enable GDScript and VisualScript to inherit from a Script object instance directly rather than having to supply a string literal filepath to them.

If you followed those steps, then you'd be able to do something like this:

//project my_project
my_project.godot
    addons
        my_plugin
            plugin_type.gd
project_type.gd
test.gd

# test.gd
extends MyPlugin.PluginType

func _ready():
    var obj = MyProject.ProjectType.new()

This would actually require minimal changes to GDScript, and I'm guessing you'd be able to do something similar to make VisualScript be able to access the singleton nodes by name as well.

So, if something like that would be permissible, how is that better than a centralized type / script database in core? I mean, with this version, any time you made changes to a script, the "namespacing" script would have to be completely reloaded, causing you to interpret the file and recreate all of the different constants, etc. Whereas if it's all in a core db of some kind, it's just a simple HashMap insertion in constant time, which is WAY faster than anything else.

[reduz]

@Xydium I understand your problem perfectly.

So, if I I have to choose between adding a lot of code complexity so your tiny bit of code, that you will not even use often, is snaller.

Or..

Leave as is, and you write a bit more code in this particular case.

Put yourself in my shoes. What do you think I should do?

[reduz]

@mysticfalls I think this is a mono specific problem that would need to be discussed with @neikeq. I personally dont understand why the restriction of same file name as class is in place. I imagine it should be possible to do away with it.

[Xydium]

@willnationsdev TypeDB? Do you mean something like ClassDB? The singleton database of classes that was renamed and made accessible from scripts three days after I opened this issue in 2016 as per commit ce26eb7?

[willnationsdev]

@Xydium It was a reference to this other issue I recently created. #17387

[reduz]

@willnationsdev I had something much simpler in mind, but I have the feeling this thread is kind of a mess, because each of you have a different idea in mind of what you need..

[willnationsdev]

XD right. Well, @Xydium's issue seems pretty similar to mine in the sense of adding namespaces / typenames to GDScript for usability purposes. Oh, but I see he wants to bring into the current namespace a subset of types with a single line statement...gotcha.

[mysticfall]

@reduz Ok, then I'll ask the admin (anyone knows whom I can ask this?) to reopen #15661 so that we can discuss it from the Mono's side first. Thanks!

[willnationsdev]

@reduz What was the "something much simpler" you had in mind? I wouldn't mind getting started on an implementation since I've already wasted a few months of my own time (my fault), lol. XD

[Xydium]

@reduz I agree that it shouldn't be implemented if I'm the only one who would want such a feature. @karroffel's explanation of standard use cases is more complete, and again, the idea connects more with a separate, statically-typed GDScript than a mere enhancement of the current GDScript. The only way to know if it's a desired feature by the community would be to poll the community.

@willnationsdev Working within the file-based GDScript, your idea from #17387 would also solve the issue in my example use case.

[willnationsdev]

@Xydium Awesome. Well, the idea from there is functionally identical to the concept of adding tons of singletons for top-level namespaces. And if he's alright with doing that, then there's hope. Just need to hear what his idea is! excitement building~

Edit:

I suppose the simpler idea may involve just directly adding the items to the static global_map in the GDScript class? That truly would be something that only works for GDScript though, whereas my idea would work with any scripting language, technically speaking. get_node("/root/MyPlugin").PluginType.new() or something like that for non-GDScript languages. Or create syntax sugar of some sort for other languages too...?

[Xydium]

@willnationsdev I'd still prefer to see optional statically-typed variables (for better code completion) and optional class-based scripts (so that scripts can be static types for variables and thus receive better code completion), which such a ScriptDB would assist in creating. However I understand @reduz's concern regarding simplicity, and mixing two paradigms in one language could be detrimental. At the same time, there are situations where dynamic types are better and vice versa, so the ability to employ static types only when they provide benefits would be nice.

[ghost]

Versus:
import enemies.*

Actually, I don't think this is any better tbh, in fact, it could even be a detriment. Because that doesn't give us the power to manually preload what we want, based on a certain map / scene that a player is loading. Having more control over what we preload and reference in dictionaries/array is a good thing! I feel like your example is over-exaggerating a problem that's easily solvable by GDScript already. (Hence your example :P)

[QbieShay]

@girng I don't see why you would want control over loading code. I understand with scenes (and that wouldn't change) but code?

[ghost]

@QbieShay having control in what you preload in Godot is extremely powerful for smooth level transitions, low memory usage (only load what you need, instead of * ), and more but just off the top of my head

[QbieShay]

@reduz I don't think that adding the classDB would make the code that much more complicated.

I've been using Godot for some months now and i already made some games with it. What i noticed, when the project started to be more and more complex, is that i felt:

1. More and more scared to refactor
2. Annoyed by having to search for some scripts of which i didn't remember the position

If classDB was there, I imagine it keeping track of where the scripts are and, when you need to move a script, classDB will update. So far, the only way to keep track of that is either having an external tool or run the game to see where it breaks.

What if the game becomes medium sized (no need for GDNative, still using GDScript)? How much time do you need to spend playing the game before noticing that something broke?

About the script searching, I tried to respect the guidelines on where to put resources and scripts, but sometimes scripts just don't fit anywhere.

I don't even want to start the static typing discussion here, but class DB would be a nice step forward for that. Two words about that here

I think that if you look around in open source Godot projects, you will soon find a lot of people that implemented something like classDB in their own projects. IIRC, @Faless did.

[JeroMiya]

I already commented on the other issue, but thought I'd chime in here too:

• Type based scripting isn't more difficult than file based scripting. It could literally be the exact same UI to drag and drop project scripts (engine fills in type reference instead of path reference), PLUS a new script search window, from which you could also drag/drop scripts. Maybe I'm missing something, but I don't understand why working with types instead of files is more difficult. Actually seems like it would be easier, and certainly more flexible w.r.t. script library dependency management.
• For refactoring/renaming types: If at any point a type reference becomes broken (type no longer exists, was renamed, different namespace, etc...), just help the user fix it. List all the broken type references in the project, let them pick a new type (from a searchable/filterable list). Fixes the broken reference wherever it's referenced. Simple. For types in the project, add a "Rename type" command to the context menu for .cs files in the project, that automates the process. Just like Visual Studio, if you rename the file first, ask the user if they want to rename the type as well, then update all the references to that type in the scene.

[hubbyist]

Renaming support is the responsibility of coder and IDE not the game engine IMO.

[mysticfall]

@hubbyist I think a game engine is a kind of IDE in a way.

[hubbyist]

Godot editor is a godot engine game so the IDE like part is not that much coupled with the engine core. And it is a good way of separating concerns I think. Renaming support may be even an official editor plugin but not a core feature. As well multi class in a file is a language specific functionality it seems and must be handled by either editor or the language binding. If PHP bindings would be added, having multiple classes in a file may help as well. But a language binding must not effect the core functionalities and lock engine to the methodologies of a specific language in any way. So binding must conform to engine not the other way around IMO.

[JeroMiya]

Godot editor is a godot engine game so the IDE like part is not that much coupled with the engine core. And it is a good way of separating concerns I think. Renaming support may be even an official editor plugin but not a core feature. As well multi class in a file is a language specific functionality it seems and must be handled by either editor or the language binding. If PHP bindings would be added, having multiple classes in a file may help as well. But a language binding must not effect the core functionalities and lock engine to the methodologies of a specific language in any way. So binding must conform to engine not the other way around IMO.

It's not so much a language binding imposing methodologies of a specific language on the engine so much as the engine imposing the methodologies of a specific language and its binding on other languages and their bindings.

Maybe a little context and history would be useful here.

Historically, most game engines used a proper "scripting" language for game scripting, such as Lua or a custom language built for the engine. These were almost always FILE based scripts - one script per file. And scripts were expected to be small and self contained, essentially "content", and not making use of libraries of functionality beyond that file. The notion was, if something more complicated was needed it would get added to the engine code itself (because in the past engines were custom written on a game-by-game basis).

But in modern times, game engines are far more general purpose, and "scripting" has become less little bits of customized logic in content to pretty much all of the game logic of the game itself. As such, "scripts" are expected to do much more sophisticated things, and thus it makes sense to support the kind of abstractions you would use in traditional "programming" to scale up to more sophisticated code bases - such as using third party libraries, separating code into modules, code-reuse, and so on. But, the file-based methodology that worked well in the past hinders this, IMO.

In my view, the engine should abstract over the specifics of how a language binding handles scripts physically on disk and simply expect the binding to provide it a list of attachable scripts, which the user can then attach to objects in the scene, and let the language binding handle the best way to implement specifics beyond that. For some language bindings, that might be file paths (lua, etc...) or modules/bundles/etc... (python, javascript), or built assemblies (C#, Java, Rust, etc...).

[neikeq]

@reduz Some months ago we were discussing this on IRC and IINW you were going to implement this. Is it still planned?

[akien-mga]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!