Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix missing script time for some functions in profiler #75623

Merged
merged 1 commit into from
Dec 19, 2023

Conversation

reach-satori
Copy link
Contributor

@reach-satori reach-satori commented Apr 3, 2023

This PR adds the (toggleable in settings) ability to profile non-gdscript functions , and makes it so the time spent in them no longer disappears

Since it's a pretty big pr and touches a lot of code, I'll briefly explain what the problem was and how I solved it:

Essentially, the script profile information is saved only in GDScriptFunction. But when something that's not a GDScriptFunction is called from there, the code assumed it would profile normally anyway and subtracted the time spent in it to get a self-time, and this wrong information caused the bug.

I've added an additional hashmap attribute to GDScriptFunction::Profile that instead saves the information on any function directly under it in the stack that has no capability to profile itself. In profiling_get_frame_data these sub-profiles then get converted to normal profile frames and sent off as usual.

The changes in ScriptLanguage and GDScriptLanguage are just to get the information on whether profiling is enabled or not from the settings to the VM, following the same mold of the profiler enabling/disabling. I thought this was necessary because collecting more information naturally slows down the execution during profiling.

pr_img

Fixes #23715, fixes #40251, fixes #29049

@reach-satori reach-satori requested review from a team as code owners April 3, 2023 17:19
@YuriSizov YuriSizov added this to the 4.1 milestone Apr 3, 2023
@reach-satori reach-satori force-pushed the profiler_for_builtins branch 4 times, most recently from b990e5c to eb97445 Compare April 10, 2023 10:13
@reach-satori reach-satori force-pushed the profiler_for_builtins branch 2 times, most recently from ff67d1c to 5ad630f Compare April 11, 2023 13:40
@vnen
Copy link
Member

vnen commented Apr 11, 2023

Overall looks okay but I do have some nitpicks.

The major one is calling this "builtins" which I think isn't clear. In general they are called "native". I would prefer to use profiler_include_native_calls than profiler_include_builtins as the editor setting name for instance.

Comment on lines 1683 to 1686
// In some cases, non-verified builtin functions end up in OPCODE_CALL.
// When that happens, base_class is empty, so we can use this to stop
// the time from being subtracted in that case
//
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why this would be the case. They should still have a base Object which will always have a class name (unless they are freed or it's null). The only exception are calls on built-in types (Vector2, String, etc.) which don't have a base Object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, that was my mistake. I can't figure out a nice way to distinguish native from user GDScript code at this point in the VM without bigger changes. After thinking about it for a while and testing some things, the best I could find was:

if (!base_obj || !base_obj->get_script_instance() || !base_obj->get_script_instance()->has_method(*methodname))

But I don't know, there seem to be a ton of potential corner cases. Can you think of a scenario where this would fail? Or a decent alternative? I'll spend some more time testing this more thoroughly regardless.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there are any good solutions. At this point the VM does not really know who owns the function being called (whether a script or a native class). It only delegates to Variant which does the actual dynamic dispatch.

I now wonder if this profiling should be done by Variant instead of the GDScript VM.

@reach-satori reach-satori marked this pull request as draft April 12, 2023 14:07
@reach-satori reach-satori force-pushed the profiler_for_builtins branch 3 times, most recently from 7fbbb80 to 1409fc4 Compare April 12, 2023 21:36
@reach-satori reach-satori force-pushed the profiler_for_builtins branch 2 times, most recently from a1349bb to d405cea Compare April 21, 2023 15:38
@reach-satori
Copy link
Contributor Author

As per @vnen's comment, there seems to be no clean way to do this properly without bigger changes that would probably require a proposal first (and/or would be over my head by quite a bit atm), but I think I managed to get some of the way there using ClassDB. As it is, according to my testing, this works as intended with straightforward native calls from GDScript, but fails in a few important cases.

Those cases are: call_deferred(), call(), and new(), because I couldn't find a way to find out if what they're calling ends up in native code or GDScript code. So I've left them unchanged and they'll miss time, same as the current profiler does for all native functions.
There might be others of the same kind I'm missing, or other cases where this approach fails that I couldn't find or think of.

In case someone eventually wants to look into it, here's also the small testing project I was using. It's essentially just a loop that calls one from a selection of function configurations, with a menu, so you can look at the profiler behaviour.
prof_proj.zip

Although, if it seems like nothing useful can come of this, feel free to close this PR.

@reach-satori reach-satori force-pushed the profiler_for_builtins branch from d405cea to 28d3ba3 Compare April 21, 2023 17:53
@reach-satori reach-satori marked this pull request as ready for review June 6, 2023 10:25
@YuriSizov YuriSizov requested review from vnen and Faless June 6, 2023 11:58
@YuriSizov YuriSizov requested a review from a team June 14, 2023 17:09
@YuriSizov YuriSizov removed request for a team December 5, 2023 12:22
@YuriSizov
Copy link
Contributor

YuriSizov commented Dec 5, 2023

Please don't merge master into your feature branch. Prefer rebasing, so your PR only contains changes relevant to your work and nothing else.

See this documentation, if you need help with rebasing.

@reach-satori reach-satori force-pushed the profiler_for_builtins branch from 48e3cee to b168e37 Compare December 5, 2023 12:39
@adamscott
Copy link
Member

Hi @reach-satori! Thanks for your first PR.
It seems that you still have problems with your merge, as it shows still 3,313 files changed.

Don't hesitate to join our developer chat! We could help you with git and your PR.

@reach-satori reach-satori force-pushed the profiler_for_builtins branch from b168e37 to fa70a40 Compare December 5, 2023 12:44
@reach-satori
Copy link
Contributor Author

reach-satori commented Dec 5, 2023

@adamscott
Sorry about that, just had a little bit of git trouble with the rebase, should be fine now

@adamscott
Copy link
Member

Happy to see that you resolved your issue!
And the invitation to join our developer chat still stands!

Copy link
Member

@adamscott adamscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed in today's GDScript meeting. This PR seems fine, we don't see how else the job could be done. It would be better to improve the code based on this PR than not.

Thank you for your PR!

Copy link
Member

@AThousandShips AThousandShips left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good otherwise!

modules/gdscript/gdscript_vm.cpp Outdated Show resolved Hide resolved
modules/gdscript/gdscript.cpp Outdated Show resolved Hide resolved
modules/gdscript/gdscript.cpp Outdated Show resolved Hide resolved
modules/gdscript/gdscript.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@Faless Faless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work 🏆 !
I just have some nitpicks on the parameter name for profiling_set_save_native_calls, but it's looking great otherwise!

Thanks a lot for working on this! 💙

core/object/script_language.h Outdated Show resolved Hide resolved
core/object/script_language_extension.cpp Outdated Show resolved Hide resolved
doc/classes/ScriptLanguageExtension.xml Outdated Show resolved Hide resolved
modules/gdscript/gdscript.cpp Outdated Show resolved Hide resolved
modules/gdscript/gdscript.cpp Outdated Show resolved Hide resolved
modules/gdscript/gdscript.h Outdated Show resolved Hide resolved
modules/mono/csharp_script.h Outdated Show resolved Hide resolved
Fixes the issue by adding a mechanism by which the functions that were
previously disappearing can be profiled too. This is optional with
an editor setting, since collecting more information naturally slows the engine
further while profiling.

Fixes godotengine#23715, godotengine#40251, godotengine#29049
@YuriSizov YuriSizov force-pushed the profiler_for_builtins branch from f55a271 to f1cc14d Compare December 19, 2023 18:43
@YuriSizov
Copy link
Contributor

Applied suggestions and rebased on your behalf.

@YuriSizov YuriSizov merged commit 549cb96 into godotengine:master Dec 19, 2023
15 checks passed
@YuriSizov
Copy link
Contributor

Thanks, and congrats on your first merged Godot contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants