Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile gdscript to gdnative when exporting a game and store/run game scripts as gdnative #11068

Closed
ghost opened this issue Sep 8, 2017 · 47 comments

Comments

@ghost
Copy link

ghost commented Sep 8, 2017

I am proposing to compile all gdscript code into gdnative C when exporting the game and store/run scripts as gdnative from here on. This conversion from gdscript to gdnative likely will have to change the Export Templates and might require extra work.

But I think this would not only make scripts run faster in the game but also take way less memory and drive space.

Originally I was thinking of JIT compile gdscript to gdnative at game run time but this method would actually be a compromise between interpreting gdscript and JIT.

This method can probably be used for Visual Script as well converted to gdnative which would make VS relatively as fast as gdscript I guess.

You guys decide if is something worth doing or not. Just putting it out there . Thank you.

@Sslaxx
Copy link

Sslaxx commented Sep 8, 2017

That would require either a C/C++ compiler being supplied with Godot, or having one available being necessary to use it.

@ghost
Copy link
Author

ghost commented Sep 8, 2017

Oh I didn't realize that . I was thinking gdnative is like java opcode but I guess it needs a C compiler . You would not need to compile binary interface to C++ objects tho because gdscript does not make calls to c++ objects so all you would need is simply compile gdscript to C binary gdnative. Perhaps a limited C compiler can be used and incorporated in the export templates along with platform specific linking assembly objects ?

Edit: something like SPCC Small Portable C Compiler or PCC portable C compiler or Tiny C compiler

@karroffel
Copy link
Contributor

I think the "best" solution would be to optionally link to LLVM, GDScript could be compiled to LLVM-IR and then even be JIT'ed while developing/prototyping. LLVM can build libraries for maaaaany different platforms, so that might be the simplest way to go without shipping an entire C compiler.

Thaaaaaat said: LLVM is a pretty huge dependency. It's about 200 Mo on my Linux distro. Like I said, it could be made an optional dependency, but that would require some work from the loading side.

I think in itself it's a nice idea, the real problem is not compiling to C or something but to actually generate the code for all the platforms you want to export to, C + a C compiler is one option, LLVM would be another and manually generating machine code (cough would be useful for a JIT anyway cough) yet another. It's not trivial but should be quite do-able. The main problem is that it most likely would require us to add a huge dependency.

@ghost
Copy link
Author

ghost commented Sep 8, 2017

That's why I was thinking a more portable C compiler like Tiny C which could be used which is way smaller in size and supports multiple platforms. A retargettable compiler like LCC would be the best tho but I think it has licensing issues.

Ah man I'm starting to think is more of a headache than is worth.

@Zylann
Copy link
Contributor

Zylann commented Sep 8, 2017

Unity3D ships (or used to ship?) with a thing nicknamed IL2CPP, which converts C# into C++ which eventually gets compiled to WebAssembly (might also add to their download weight, editor-only though xD), so these kind of things are possible, but I don't know if that was ever done with dynamically-typed languages? Would it mean everything be Variant once converted to C?

@karroffel
Copy link
Contributor

Ideally there would be a lot of type inference going on or we wait for static typing in GDScript.

@neikeq
Copy link
Contributor

neikeq commented Sep 8, 2017

I think the easiest solution is a C transpiler.

@Zylann Yes, everything would be Variant unless you can infer the types. Static typing would improve this.

@karroffel
Copy link
Contributor

Generating C code should be pretty easy, it would be interesting to apply some global type inference on the GDScript code in question. Because it would all be done at export time there could be a lot more effort involved in figuring out types.

Unresolvable method signatures could be made type generic and instantiated accordingly.

That would be quite the undertaking, but I'm pretty certain that it would yield in a huge performance increase overall.

I think the bigger question is how cross-platform exports can be made as painless as possible. The picked toolchain (LLVM, tiny C, portable C) would be pretty important. There are still some problems with better windows compatibility, but I think we can rule this out later on (cc @touilleMan ).

@juan-garcia-m
Copy link

I have been also checking this for a while.
Most portable compilers are either old, non free or GPL, and mostly 32 bits x86.
The way to go is either embedding LLVM with Clang, or using LuaJIT, or directly compiling GDScript to a custom bytecode format and translating directly from there to x86 and ARM binary, in which way we will need to generate .so, .dll and .dylib, not an easy task.
A GDNative addon with embedded LLVM tool plugin could be nice though.

@malucard
Copy link

There are alternatives to LLVM. LLVM is a huge dependency and is actually larger than Godot. It'd be better if we used something else instead. Including Clang as well is just suicide. C code is very easy to compile, so there are a ton of small C compilers available. A faster approach would be compiling GDScript bytecode to machine code directly. Godot is portable, so an IR like LLVM which generates machine code for us would be used, though that'd still be pretty hard to do, so translating to C sounds better.

@vnen
Copy link
Member

vnen commented May 21, 2018

TBH I would just translate GDScript to C (with NativeScript), and let the user responsible for compiling the result. I doubt we can seamlessly compile to every target platform from every host without requiring many dependencies (or being downright impossible, like compiling to iOS from a Windows host). I'm also not convinced it would be useful for most cases.

I also wonder if it's feasible to replace every node that has a GDScript to point to a NativeScript instead. Would that work fine? What about the peculiarities of GDScript, like named singletons, yield(), preload(), and others?

@malucard
Copy link

That sounds kinda pointless. An option would be having a (platform-specific) shell script written by the user, which compiles the code without waiting for the user.

Another approach would be replace the GDScript itself with a native script. So it would compile all GDScript scripts into a .so/.dylib/.dll and a .gdnlib (preferably a single one), and when loading a .gd, it would load a .gdns instead, if available (if not, either interpret it or compile it, depending on how this would be implemented). I don't know how feasible that is, though.

@pchasco
Copy link

pchasco commented Jun 25, 2018

Personally I would lean away from JIT in favor of transpilation only because some platforms ahem, iOS do not support JIT.

And transpilation would be much simpler to implement, aside the obvious caveats which break the typical C call stack like yield. But those issues CAN be worked around.

@OvermindDL1
Copy link

OvermindDL1 commented Jun 25, 2018

And transpilation would be much simpler to implement, aside the obvious caveats which break the typical C call stack like yield. But those issues CAN be worked around.

And even C++20 is getting single-scoped continuations, so could transpile to that (embedded clang or so) for a more direct conversion, or just use llvm/libclang directly and skip the intermediary step.

Also, this might be a duplicate of #5049 now?

@jonbonazza
Copy link
Contributor

Seems like the major concern witj llvm is the size of the dependancy. Why is that a problem again? I seem to be missing that point.

@touilleMan
Copy link
Member

touilleMan commented Jun 28, 2018

@jonbonazza Godot is supposed to be a single non-bloated binary.
For instance reflexion on Godot API (to get functions arguments and return types at runtime for instance) are disabled in release builds to save space. We are talking of a few Mb, so adding ~76mo (that's the size of libclang.so in my Qt5 folder, given they use this for their Qml script language I guess the usecase is kind of similar...) is a no go.

The solution would be to create a new flavour of Godot release (just like we already have for the .net release) but this add to the burden of the release managers (even worst if we take into account combination issue, like people who want .net AND GDscript compiler...)

The solution I see to this trouble is obviously provide this feature as a GDnative extension (so it could be installed when needed through the asset library).
However this means we would typically need to improve the Pluginscript API to allow to overwrite a language (to make .gd files loaded by our module instead of the regular GDScript one).
We could also make the API a bit more intelligent to allow the plugin to choose whether or not it want to compile a given script or to let it to the regular GDScript (for instance for debugging purpose, or if the given script contains hard to implement structures that haven't been implemented yet).

@jonbonazza
Copy link
Contributor

jonbonazza commented Jun 28, 2018 via email

@malucard
Copy link

Soon we end up with an 8 GB executable like Unity but it doesn't matter because disk space is cheap, right? The time spent downloading the editor/a game is not a problem at all!

Bloat is never justified. LLVM is more than double the size of Godot (I'm not sure how big it is, the guy above said 76 MB but I was pretty sure it's more around 100-200 MB, while Godot is way tinier than that). Like this, it's simply bloat. Aren't there any alternatives?

@Zylann
Copy link
Contributor

Zylann commented Jul 10, 2018

If things like that ever get integrated to Godot, it should just be a separate download, so people don't have to install things they don't need (that doesn't prevent from providing an easy-to-use installer similar to what Unity ships with by default, if ever needed).

@OvermindDL1
Copy link

Bloat is never justified. LLVM is more than double the size of Godot (I'm not sure how big it is, the guy above said 76 MB but I was pretty sure it's more around 100-200 MB, while Godot is way tinier than that). Like this, it's simply bloat. Aren't there any alternatives?

Oh that certainly wouldn't be in the distributions, it would only be in the editor, and it would end up making the release distributions smaller by being able to get rid of the interpreter and with LTO (if ever added as a post-process with the engine) would shrink it far further, thus reducing download times.

@malucard
Copy link

The interpreter is tiny. And how come LTO isn't used?

@OvermindDL1
Copy link

Tiny yes, but it also links in to everything, thus forcing everything to be included even if it is not in use. LTO cannot be used between GDNative and the engine right now because they are currently dynamically linked, you'd need another pass upon release compilation to so to compile/link them together, of which if you give the linker enough information you'd be able to prune a lot of unused code. That is far down work, however lots of little steps, of which this would be one.

@aistrych
Copy link

I don't understand why not just use Mono JIT and AoT compilers. Mono is used for C# version so it's already integrated. We just need GDScript to CIL bytecode compiler.

@OvermindDL1
Copy link

Mono JIT

Not allowed on some platforms like iOS and webasm.

AoT

This is what I'm a proponant for! The OCaml compiler would be nice for that (GDScript could even compile 'to' it, with some caveats that vanish with typed gdscript), interpreter for runtime testing and debugging, compilation to C-speed machine code otherwise.

@aistrych
Copy link

Not allowed on some platforms like iOS and webasm.

Yes, I know but it can be JIT-ed on other platforms and AOT compiled on iOS and webasm. Or it can be JIT compiled in development mode and AoT compiled for production. Mono has experimental version for WebAsembly already.
It can be also OCaml compiler but it means we will be adding another compiler to the stack. Question is: what is easier to implement (Mono compiler?) and what will be more performant (OCaml compiler?).

@DriNeo
Copy link

DriNeo commented Jul 11, 2018

I don't understand why not just use Mono JIT and AoT compilers. Mono is used for C# version so it's already integrated. We just need GDScript to CIL bytecode compiler.

GDNative is already integrated too. Mono is integrated in C# version only.
I guess it's rare to find a developer without a C compiler installed on his machine.
I don't understand where is the problem to add a so common dependency.
For Android export you must enter the path of JDK in the settings, and other paths for external things, why not enter the path of GCC or Mingw ?

@malucard
Copy link

malucard commented Jul 11, 2018

The problem is that it wasn't a dependency. They were talking about using LLVM or libclang in Godot. Those are libraries, not programs, and they would directly increase the executable size of Godot. Like you said, simply running a compiler is a more lightweight solution. I support that. But you don't need to enter the path, just run cc or c++ (and cl on Windows, I think) which point to the default C and C++ compiler respectively. Of course, allowing a custom path is nice, but it doesn't have to mandatory.

@vnen
Copy link
Member

vnen commented Jul 11, 2018

Can this LLVM dependency compile to all target platforms from the same host? If not, then it breaks expectation in the cross-platform export system. Though it is somewhat a concern, the size is not much of the problem, what gets it is that Godot is supposed to be cross-platform, and usually compilers have a single target. So we have to ship compilers to all platforms, which is not really feasible.

I don't have any problem if it's an optional external dependency. As I said before: it's easier to just convert it to source code and let the user provide the toolchain on their own. This way it can be used with different compilers and linkers.

@Two-Tone
Copy link

they would directly increase the executable size of Godot

Isn't clang, by itself, 40MB-60MB? Yes, that basically doubles Godot's size, but when Unity is over 10GB in size and UE4 is 20GB for the same level of platform support that Godot has, it's still absolutely tiny.

Can this LLVM dependency compile to all target platforms from the same host?

Yes, LLVM supports cross-compiling. It's what UE4 uses for cross compiling.

@m4nu3lf
Copy link
Contributor

m4nu3lf commented Nov 13, 2018

I had the same idea and I thought my idea was original... what a delusion :(
The main problem with a JIT to me is that I think it would be a ton of work to get right.

We could still have this feature provided as optional ofc. If you don't transpile to C++ you would still use the the interpreted version.
Is having gcc/clang as an optional dependency that bad? In practice all you need to do is to provide the name of the command to Godot.

I also think it would be easy to add a feature to selectively transplile files from the editor. It may be useful to speed up GDScript during development, not only during exporting.

The pro of this is technique is that you may transpile to readable C++ and if you realize you need more control over the code, keep working on the C++ module and drop the GDScript one

@pchasco
Copy link

pchasco commented Nov 13, 2018

I am working on an experimental version of this that will compile gdscript bytecode to GDNative class files. Hopefully in the next month or so I will have an alpha version available.

The challenge will be ensuring that the GDNative scripts inter-operate with GDScripts that are not compiled (ensuring all public interface remains compatible). Translating the bytecode to C++ is a problem that has been solved for other VMs many times over, so I don't expect any show-stoppers there.

The problem I haven't solved yet is yield and resume. This will require support from the engine itself and cannot be entirely solved by transforming the input GDScript into a state machine. The common way to translate a coroutine to C/C++ would be to transform the method into a structure. All locals get promoted to members of the struct and the statements themselves are separated by the yield:

Some psuedo-ish code to illustrate what I mean:

class SomeMethodCooroutine : public GDNative::Cooroutine
{
protected:
    int _currentState; // From GDNative::Cooroutine

public:
    gd2cpp_generated_class *self;
    Variant result;

    int x;
    int y;

GDNative::Cooroutine resume() {
    switch (_currentState) {
        case 0:
            // var x = 1;
            x = 1;
            // var y = 2;
            y = 2;
            // member_variable = x + y
            int _a = x + y;
            owner->set_member_variable(_a);
            // yield($Button1, "pressed")
            return yield_signal(1, "Button1", "pressed");
        case 1:
            // Do some other stuff
            return complete("some return value");
    }
}

GDNative::Cooroutine yield_signal(int next_state, String node_path, String signal) {
    _currentState = next_state;
    // Set up whatever is needed to inform godot that we are waiting for the signal
    return this;
}

@m4nu3lf
Copy link
Contributor

m4nu3lf commented Nov 13, 2018

@pchasco IMO it would be much easier to emit C++ from the GDScript AST! Have you considered that?

@pchasco
Copy link

pchasco commented Nov 13, 2018 via email

@m4nu3lf
Copy link
Contributor

m4nu3lf commented Nov 13, 2018

@pchasco I had a look at the implementation right now. It is generating an internal AST it just isn't exposed with any API. No writing another parser is not the way to go. What I was suggesting was to have another implementation of the GDScript compiler which takes the AST and produces the C++ code, so yes, implement something like gdscript_compiler_cpp as part of the GDScript module. Not sure which approach works best now

@pchasco
Copy link

pchasco commented Nov 13, 2018 via email

@neikeq
Copy link
Contributor

neikeq commented Nov 13, 2018

You can use this public API:

class GDScriptParser {
public:
struct ClassNode;
struct DataType {
enum {
BUILTIN,
NATIVE,
SCRIPT,
GDSCRIPT,
CLASS,
UNRESOLVED
} kind;
bool has_type;
bool is_constant;
bool is_meta_type; // Whether the value can be used as a type
bool infer_type;
bool may_yield; // For function calls
Variant::Type builtin_type;
StringName native_type;
Ref<Script> script_type;
ClassNode *class_type;
String to_string() const;
bool operator==(const DataType &other) const {
if (!has_type || !other.has_type) {
return true; // Can be considered equal for parsing purpose
}
if (kind != other.kind) {
return false;
}
switch (kind) {
case BUILTIN: {
return builtin_type == other.builtin_type;
} break;
case NATIVE: {
return native_type == other.native_type;
} break;
case GDSCRIPT:
case SCRIPT: {
return script_type == other.script_type;
} break;
case CLASS: {
return class_type == other.class_type;
} break;
case UNRESOLVED: {
} break;
}
return false;
}
DataType() :
has_type(false),
is_constant(false),
is_meta_type(false),
infer_type(false),
may_yield(false),
builtin_type(Variant::NIL),
class_type(NULL) {}
};
struct Node {
enum Type {
TYPE_CLASS,
TYPE_FUNCTION,
TYPE_BUILT_IN_FUNCTION,
TYPE_BLOCK,
TYPE_IDENTIFIER,
TYPE_TYPE,
TYPE_CONSTANT,
TYPE_ARRAY,
TYPE_DICTIONARY,
TYPE_SELF,
TYPE_OPERATOR,
TYPE_CONTROL_FLOW,
TYPE_LOCAL_VAR,
TYPE_CAST,
TYPE_ASSERT,
TYPE_BREAKPOINT,
TYPE_NEWLINE,
};
Node *next;
int line;
int column;
Type type;
virtual DataType get_datatype() const { return DataType(); }
virtual void set_datatype(const DataType &p_datatype) {}
virtual ~Node() {}
};
struct FunctionNode;
struct BlockNode;
struct ConstantNode;
struct LocalVarNode;
struct OperatorNode;
struct ClassNode : public Node {
bool tool;
StringName name;
bool extends_used;
StringName extends_file;
Vector<StringName> extends_class;
DataType base_type;
String icon_path;
struct Member {
PropertyInfo _export;
#ifdef TOOLS_ENABLED
Variant default_value;
#endif
StringName identifier;
DataType data_type;
StringName setter;
StringName getter;
int line;
Node *expression;
OperatorNode *initial_assignment;
MultiplayerAPI::RPCMode rpc_mode;
int usages;
};
struct Constant {
Node *expression;
DataType type;
};
struct Signal {
StringName name;
Vector<StringName> arguments;
int emissions;
int line;
};
Vector<ClassNode *> subclasses;
Vector<Member> variables;
Map<StringName, Constant> constant_expressions;
Vector<FunctionNode *> functions;
Vector<FunctionNode *> static_functions;
Vector<Signal> _signals;
BlockNode *initializer;
BlockNode *ready;
ClassNode *owner;
//Vector<Node*> initializers;
int end_line;
ClassNode() {
tool = false;
type = TYPE_CLASS;
extends_used = false;
end_line = -1;
owner = NULL;
}
};
struct FunctionNode : public Node {
bool _static;
MultiplayerAPI::RPCMode rpc_mode;
bool has_yield;
bool has_unreachable_code;
StringName name;
DataType return_type;
Vector<StringName> arguments;
Vector<DataType> argument_types;
Vector<Node *> default_values;
BlockNode *body;
#ifdef DEBUG_ENABLED
Vector<int> arguments_usage;
#endif // DEBUG_ENABLED
virtual DataType get_datatype() const { return return_type; }
virtual void set_datatype(const DataType &p_datatype) { return_type = p_datatype; }
FunctionNode() {
type = TYPE_FUNCTION;
_static = false;
rpc_mode = MultiplayerAPI::RPC_MODE_DISABLED;
has_yield = false;
has_unreachable_code = false;
}
};
struct BlockNode : public Node {
ClassNode *parent_class;
BlockNode *parent_block;
List<Node *> statements;
Map<StringName, LocalVarNode *> variables;
bool has_return;
Node *if_condition; //tiny hack to improve code completion on if () blocks
//the following is useful for code completion
List<BlockNode *> sub_blocks;
int end_line;
BlockNode() {
if_condition = NULL;
type = TYPE_BLOCK;
end_line = -1;
parent_block = NULL;
parent_class = NULL;
has_return = false;
}
};
struct TypeNode : public Node {
Variant::Type vtype;
TypeNode() { type = TYPE_TYPE; }
};
struct BuiltInFunctionNode : public Node {
GDScriptFunctions::Function function;
BuiltInFunctionNode() { type = TYPE_BUILT_IN_FUNCTION; }
};
struct IdentifierNode : public Node {
StringName name;
BlockNode *declared_block; // Simplify lookup by checking if it is declared locally
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
IdentifierNode() {
type = TYPE_IDENTIFIER;
declared_block = NULL;
}
};
struct LocalVarNode : public Node {
StringName name;
Node *assign;
OperatorNode *assign_op;
int assignments;
int usages;
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
LocalVarNode() {
type = TYPE_LOCAL_VAR;
assign = NULL;
assign_op = NULL;
assignments = 0;
usages = 0;
}
};
struct ConstantNode : public Node {
Variant value;
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
ConstantNode() { type = TYPE_CONSTANT; }
};
struct ArrayNode : public Node {
Vector<Node *> elements;
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
ArrayNode() {
type = TYPE_ARRAY;
datatype.has_type = true;
datatype.kind = DataType::BUILTIN;
datatype.builtin_type = Variant::ARRAY;
}
};
struct DictionaryNode : public Node {
struct Pair {
Node *key;
Node *value;
};
Vector<Pair> elements;
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
DictionaryNode() {
type = TYPE_DICTIONARY;
datatype.has_type = true;
datatype.kind = DataType::BUILTIN;
datatype.builtin_type = Variant::DICTIONARY;
}
};
struct SelfNode : public Node {
SelfNode() { type = TYPE_SELF; }
};
struct OperatorNode : public Node {
enum Operator {
//call/constructor operator
OP_CALL,
OP_PARENT_CALL,
OP_YIELD,
OP_IS,
OP_IS_BUILTIN,
//indexing operator
OP_INDEX,
OP_INDEX_NAMED,
//unary operators
OP_NEG,
OP_POS,
OP_NOT,
OP_BIT_INVERT,
//binary operators (in precedence order)
OP_IN,
OP_EQUAL,
OP_NOT_EQUAL,
OP_LESS,
OP_LESS_EQUAL,
OP_GREATER,
OP_GREATER_EQUAL,
OP_AND,
OP_OR,
OP_ADD,
OP_SUB,
OP_MUL,
OP_DIV,
OP_MOD,
OP_SHIFT_LEFT,
OP_SHIFT_RIGHT,
OP_INIT_ASSIGN,
OP_ASSIGN,
OP_ASSIGN_ADD,
OP_ASSIGN_SUB,
OP_ASSIGN_MUL,
OP_ASSIGN_DIV,
OP_ASSIGN_MOD,
OP_ASSIGN_SHIFT_LEFT,
OP_ASSIGN_SHIFT_RIGHT,
OP_ASSIGN_BIT_AND,
OP_ASSIGN_BIT_OR,
OP_ASSIGN_BIT_XOR,
OP_BIT_AND,
OP_BIT_OR,
OP_BIT_XOR,
//ternary operators
OP_TERNARY_IF,
OP_TERNARY_ELSE,
};
Operator op;
Vector<Node *> arguments;
DataType datatype;
virtual DataType get_datatype() const { return datatype; }
virtual void set_datatype(const DataType &p_datatype) { datatype = p_datatype; }
OperatorNode() { type = TYPE_OPERATOR; }
};
struct PatternNode : public Node {
enum PatternType {
PT_CONSTANT,
PT_BIND,
PT_DICTIONARY,
PT_ARRAY,
PT_IGNORE_REST,
PT_WILDCARD
};
PatternType pt_type;
Node *constant;
StringName bind;
Map<ConstantNode *, PatternNode *> dictionary;
Vector<PatternNode *> array;
};
struct PatternBranchNode : public Node {
Vector<PatternNode *> patterns;
BlockNode *body;
};
struct MatchNode : public Node {
Node *val_to_match;
Vector<PatternBranchNode *> branches;
struct CompiledPatternBranch {
Node *compiled_pattern;
BlockNode *body;
};
Vector<CompiledPatternBranch> compiled_pattern_branches;
};
struct ControlFlowNode : public Node {
enum CFType {
CF_IF,
CF_FOR,
CF_WHILE,
CF_SWITCH,
CF_BREAK,
CF_CONTINUE,
CF_RETURN,
CF_MATCH
};
CFType cf_type;
Vector<Node *> arguments;
BlockNode *body;
BlockNode *body_else;
MatchNode *match;
ControlFlowNode *_else; //used for if
ControlFlowNode() {
type = TYPE_CONTROL_FLOW;
cf_type = CF_IF;
body = NULL;
body_else = NULL;
}
};
struct CastNode : public Node {
Node *source_node;
DataType cast_type;
DataType return_type;
virtual DataType get_datatype() const { return return_type; }
virtual void set_datatype(const DataType &p_datatype) { return_type = p_datatype; }
CastNode() { type = TYPE_CAST; }
};
struct AssertNode : public Node {
Node *condition;
AssertNode() { type = TYPE_ASSERT; }
};
struct BreakpointNode : public Node {
BreakpointNode() { type = TYPE_BREAKPOINT; }
};
struct NewLineNode : public Node {
NewLineNode() { type = TYPE_NEWLINE; }
};
struct Expression {
bool is_op;
union {
OperatorNode::Operator op;
Node *node;
};
};
enum CompletionType {
COMPLETION_NONE,
COMPLETION_BUILT_IN_TYPE_CONSTANT,
COMPLETION_GET_NODE,
COMPLETION_FUNCTION,
COMPLETION_IDENTIFIER,
COMPLETION_PARENT_FUNCTION,
COMPLETION_METHOD,
COMPLETION_CALL_ARGUMENTS,
COMPLETION_RESOURCE_PATH,
COMPLETION_INDEX,
COMPLETION_VIRTUAL_FUNC,
COMPLETION_YIELD,
COMPLETION_ASSIGN,
COMPLETION_TYPE_HINT,
COMPLETION_TYPE_HINT_INDEX,
};

@reisraff

This comment has been minimized.

@RetroZvoc
Copy link

Alright. I see the issue. What makes Godot Engine stand out from all other engines is that Godot isn't some old or proprietary or GPL/ShareAlike or non-commercial piece of software, but that it's an actual actually true truly open-source and free (free=NO STRINGS ATTACHED!) game engine that can do many things that aren't just a simple game engine such as a text editor, media player, etc.. One might as well make a simple BSD-powered permissive-licensed MIPS SoC devboard running Godot Engine as a desktop environment! I'm actually thinking about that. Imagine if we called it GodotOS or GodOS. LOL.

So, to get serious, here's my point. Both the gamers/users and the developers want to have something that will work out of the box on howevermuch old or new machine they might have. Meaning that it's not the disk space that's such a big issue, but the real issue is the execution speed, stability, consistency and RAM usage. One can have 16GB of disk space for a superpowered Godot with all sorts of JIT compilers for every single CPU and chipset and platform and OS, but they cannot all have 64GB of RAM!

Yes, 16GB is surely a huge issue, but the thing is that if these 16GB mean that it's not something that will break when you install a different Visual C++ Redistributable or some dependency or get a Windows update, then it's worth it. At the same time, there are users who want to have it very simple as how the Godot engine is (cca 50MB iirc). So that way, I guess it would be the best if every user could choose which version of Godot they'd like to download.

Now, recoding the whole Godot infrastructure is a big problem for developers because it would create so much delay while everyone gets synchronized into where to commit which changes and where to do what and that would slow down bugfixes, security patches, new feature releases, it would spawn a ton of new bugs and security patches and people might lose interest in Godot in an instant!

This is why the layers of "bloat"/abstraction should go like this:
Dynamically loaded scripts (like missions; GDscript), your game code (GDscript, C++, C#) > Godot code (GDscript, C++, C#) > JIT-ter (C++) > Any platform you want the code to run or to compile to.

This way every one of these layers will not have to be submerged and choked and quenched by another. Instead, everyone will be doing their job and making sure that their part is done well. If the JIT compiler is perhaps made from scratch, or imagine if some of professors and college students and doctors and senior developers who are reading this or some of you who have access to talk to such people could spread the news about this and possibly get involved into making a Godot-dedicated JIT compiler that might even be a much greater alternative to all LLVMs and things that we can see on the internet. I mean, if Terry Davison could make a 2MB OS with his own programming language and everything all by himself and if it took him 10 years, then imagine how much it would take those who truly love the true open-source no-strings-attached free software development software to make this Godot dream became a reality.

I hope that I've inspired people a huge lot and that the Godot Engine causes a total disruption in the gamedev software and software development software market in such a way that even the copyright laws start legalizing and indemnifying and extending Fair Use and memes and fan works (fan art, fan fiction, fan characters, video game music remixes, etc.) and other transformative works in which case the avalanche of copyright/patent liberalization might even kill some of our biggest enemy licenses; causing them to be forced to release their MIT-license-incompatible software under the MIT license.

This would be the disruption of the century! And we've been waiting for this for so long! Waiting for Godot! And we shall await him!

@pchasco
Copy link

pchasco commented May 12, 2019 via email

@RetroZvoc
Copy link

Well, I'm sorry. I wrongly said. JIT compiling should only be for loading things in runtime such as mods, plugins, customized multiplayer features (like custom characters with custom abilities), etc.. AOT compiling should be for everything that can be compiled at export time.

Now, how far has Godot went with this JIT/AOT feature?

JIT is a waste of time because it’s not supported on some important platforms. It is easier to compile GDScript ahead of time than to build a JIT. Precompiled code is also more efficient and performs better in nearly all cases.

On Sat, May 11, 2019 at 9:19 PM RetroZvoc @.***> wrote: Alright. I see the issue. What makes Godot Engine stand out from all other engines is that Godot isn't some old or proprietary or GPL/ShareAlike or non-commercial piece of software, but that it's an actual actually true truly open-source and free (free=NO STRINGS ATTACHED!) game engine that can do many things that aren't just a simple game engine such as a text editor, media player, etc.. One might as well make a simple BSD-powered permissive-licensed MIPS SoC devboard running Godot Engine as a desktop environment! I'm actually thinking about that. Imagine if we called it GodotOS or GodOS. LOL. So, to get serious, here's my point. Both the gamers/users and the developers want to have something that will work out of the box on howevermuch old or new machine they might have. Meaning that it's not the disk space that's such a big issue, but the real issue is the execution speed, stability, consistency and RAM usage. One can have 16GB of disk space for a superpowered Godot with all sorts of JIT compilers for every single CPU and chipset and platform and OS, but they cannot all have 64GB of RAM! Yes, 16GB is surely a huge issue, but the thing is that if these 16GB mean that it's not something that will break when you install a different Visual C++ Redistributable or some dependency or get a Windows update, then it's worth it. At the same time, there are users who want to have it very simple as how the Godot engine is (cca 50MB iirc). So that way, I guess it would be the best if every user could choose which version of Godot they'd like to download. Now, recoding the whole Godot infrastructure is a big problem for developers because it would create so much delay while everyone gets synchronized into where to commit which changes and where to do what and that would slow down bugfixes, security patches, new feature releases, it would spawn a ton of new bugs and security patches and people might lose interest in Godot in an instant! This is why the layers of "bloat"/abstraction should go like this: Dynamically loaded scripts (like missions; GDscript), your game code (GDscript, C++, C#) > Godot code (GDscript, C++, C#) > JIT-ter (C++) > Any platform you want the code to run or to compile to. This way every one of these layers will not have to be submerged and choked and quenched by another. Instead, everyone will be doing their job and making sure that their part is done well. If the JIT compiler is perhaps made from scratch, or imagine if some of professors and college students and doctors and senior developers who are reading this or some of you who have access to talk to such people could spread the news about this and possibly get involved into making a Godot-dedicated JIT compiler that might even be a much greater alternative to all LLVMs and things that we can see on the internet. I mean, if Terry Davison could make a 2MB OS with his own programming language and everything all by himself and if it took him 10 years, then imagine how much it would take those who truly love the true open-source no-strings-attached free software development software to make this Godot dream became a reality. I hope that I've inspired people a huge lot and that the Godot Engine causes a total disruption in the gamedev software and software development software market in such a way that even the copyright laws start legalizing and indemnifying and extending Fair Use and memes and fan works (fan art, fan fiction, fan characters, video game music remixes, etc.) and other transformative works in which case the avalanche of copyright/patent liberalization might even kill some of our biggest enemy licenses; causing them to be forced to release their MIT-license-incompatible software under the MIT license. This would be the disruption of the century! And we've been waiting for this for so long! Waiting for Godot! And we shall await him! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11068 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUFIAHQKJ3FWAS5SHQDWPLPU55CBANCNFSM4D2E5KZA .

@pchasco
Copy link

pchasco commented May 12, 2019 via email

@RetroZvoc
Copy link

I hope I can directly write C++ stuff to Godot inside of Godot IDE's text editor since I'm making a desktop environment for BSD in Godot and it will run on a MIPS SoC devboard that won't have enough time for bytecode bloat (or should I say bloatcode). Can I do that?

No work has been done on JIT, and I don’t think that it will be. I have been working on an AoT module and it is very rudimentary. http://blog.moblcade.com

On Sun, May 12, 2019 at 9:59 AM RetroZvoc @.> wrote: Well, I'm sorry. I wrongly said. JIT compiling should only be for loading things in runtime such as mods, plugins, customized multiplayer features (like custom characters with custom abilities), etc.. AOT compiling should be for everything that can be compiled at export time. Now, how far has Godot went with this JIT/AOT feature? JIT is a waste of time because it’s not supported on some important platforms. It is easier to compile GDScript ahead of time than to build a JIT. Precompiled code is also more efficient and performs better in nearly all cases. … <#m_-7560708296574458465_> On Sat, May 11, 2019 at 9:19 PM RetroZvoc @.> wrote: Alright. I see the issue. What makes Godot Engine stand out from all other engines is that Godot isn't some old or proprietary or GPL/ShareAlike or non-commercial piece of software, but that it's an actual actually true truly open-source and free (free=NO STRINGS ATTACHED!) game engine that can do many things that aren't just a simple game engine such as a text editor, media player, etc.. One might as well make a simple BSD-powered permissive-licensed MIPS SoC devboard running Godot Engine as a desktop environment! I'm actually thinking about that. Imagine if we called it GodotOS or GodOS. LOL. So, to get serious, here's my point. Both the gamers/users and the developers want to have something that will work out of the box on howevermuch old or new machine they might have. Meaning that it's not the disk space that's such a big issue, but the real issue is the execution speed, stability, consistency and RAM usage. One can have 16GB of disk space for a superpowered Godot with all sorts of JIT compilers for every single CPU and chipset and platform and OS, but they cannot all have 64GB of RAM! Yes, 16GB is surely a huge issue, but the thing is that if these 16GB mean that it's not something that will break when you install a different Visual C++ Redistributable or some dependency or get a Windows update, then it's worth it. At the same time, there are users who want to have it very simple as how the Godot engine is (cca 50MB iirc). So that way, I guess it would be the best if every user could choose which version of Godot they'd like to download. Now, recoding the whole Godot infrastructure is a big problem for developers because it would create so much delay while everyone gets synchronized into where to commit which changes and where to do what and that would slow down bugfixes, security patches, new feature releases, it would spawn a ton of new bugs and security patches and people might lose interest in Godot in an instant! This is why the layers of "bloat"/abstraction should go like this: Dynamically loaded scripts (like missions; GDscript), your game code (GDscript, C++, C#) > Godot code (GDscript, C++, C#) > JIT-ter (C++) > Any platform you want the code to run or to compile to. This way every one of these layers will not have to be submerged and choked and quenched by another. Instead, everyone will be doing their job and making sure that their part is done well. If the JIT compiler is perhaps made from scratch, or imagine if some of professors and college students and doctors and senior developers who are reading this or some of you who have access to talk to such people could spread the news about this and possibly get involved into making a Godot-dedicated JIT compiler that might even be a much greater alternative to all LLVMs and things that we can see on the internet. I mean, if Terry Davison could make a 2MB OS with his own programming language and everything all by himself and if it took him 10 years, then imagine how much it would take those who truly love the true open-source no-strings-attached free software development software to make this Godot dream became a reality. I hope that I've inspired people a huge lot and that the Godot Engine causes a total disruption in the gamedev software and software development software market in such a way that even the copyright laws start legalizing and indemnifying and extending Fair Use and memes and fan works (fan art, fan fiction, fan characters, video game music remixes, etc.) and other transformative works in which case the avalanche of copyright/patent liberalization might even kill some of our biggest enemy licenses; causing them to be forced to release their MIT-license-incompatible software under the MIT license. This would be the disruption of the century! And we've been waiting for this for so long! Waiting for Godot! And we shall await him! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11068 (comment) <#11068 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUFIAHQKJ3FWAS5SHQDWPLPU55CBANCNFSM4D2E5KZA . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11068 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUFIAEFLXSTHCM6VTA5UI3PVAWDNANCNFSM4D2E5KZA .

@pchasco
Copy link

pchasco commented May 12, 2019 via email

@KoBeWi
Copy link
Member

KoBeWi commented May 28, 2020

Closing this because we are moving proposals out of this repo to a new proposal repository. Also this feature is very likely coming in the upcoming GDScript rewrite, so there's no need to make a new proposal.

@Two-Tone
Copy link

Two-Tone commented May 29, 2020 via email

@Xrayez
Copy link
Contributor

Xrayez commented May 29, 2020

@Two-Tone GIP is likely not an appropriate place to "track" proposals. For example, I wanted to keep such a tracker proposal here: godotengine/godot-proposals#569, but was suggested to close it unfortunately.

Meanwhile, feel free to use Godot Ideas repository for such things (it's unofficial though).

@vnen
Copy link
Member

vnen commented May 29, 2020

Actually, reduz has an idea for this and said he will come up with a proposal of how it would work. I'm not against opening a GIP for this in any case. You do need to follow the template though.

@Xrayez my comment there is that it was more like a bug than a proposal, so it should be treated as such. If you want to track all issues with module, you can create a tracker here listing all related reports (as we already have a few).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests