Defer clearing of ResourceUID cache #67128

KoBeWi · 2022-10-09T12:00:58Z

EditorFile dock was clearing the UID cache, but it was happening before plugins, autolads etc. are loaded, resulting in error spam. I deferred the call, so the editor stuff can load properly. This also makes it possible to use UIDs directly in editor plugins instead of paths.

Fixes #64881

bitbrain · 2022-10-09T22:19:02Z

Just tested this change on my game and it seems the issue described in #64881 is still happening for my game:

The .tscn files containing the uid:// strings were already pre-existing so perhaps due to this bug some of the resource ids were created wrongly and are now permanently broken. The only way to fix it currently is to manually edit every single .tscn file and remove the broken uid:// bits from there to get rid of these warnings.

KoBeWi · 2022-10-09T22:22:47Z

You can actually fix this without opening the scenes (manually). Just run this script in your project:

@tool
extends EditorScript

var files: Array[String]

func _run() -> void:
	files = []
	
	add_files("res://")
	
	for file in files:
		print(file)
		var res = load(file)
		ResourceSaver.save(res)

func add_files(dir: String):
	for file in DirAccess.get_files_at(dir):
		if file.get_extension() == "tscn" or file.get_extension() == "tres":
			files.append(dir.path_join(file))
	
	for dr in DirAccess.get_directories_at(dir):
		add_files(dir.path_join(dr))

It will open and re-save all text resource files, to make sure they are up to date. It fixes formatting, uids, old classes etc.

bitbrain · 2022-10-09T22:37:40Z

@KoBeWi nope - running that makes it even worse on restart:

https://gist.github.com/bitbrain/d3870f0c52cfdb20038a0066a1251a5a

I also attempted to delete the .godot folder and remove all *.import files and re-import but same issue. The game works fine and resources load as expected, it's just that it still spams when opening the project. Perhaps somebody else can reproduce this on this branch?

KoBeWi · 2022-10-09T22:40:38Z

Are the UIDs correct though? Compare the ext resources in scenes with their actual ids; maybe it's a different issue.

bitbrain · 2022-10-09T22:54:12Z

@KoBeWi example when loading my game project:

WARNING: res://ui/savegame/LoadGameContainer.tscn:6 - ext_resource, invalid UUID: uid://k810555el837 - using text path instead: res://worldobjects/player/player.png
     at: ResourceLoaderText::load (scene\resources\resource_format_text.cpp:448)

Looking into res://ui/savegame/LoadGameContainer.tscn the affected line is this:

[ext_resource type="Texture2D" uid="uid://k810555el837" path="res://worldobjects/player/player.png" id="4"]

However, this id is present within res://worldobjects/player/player.png.import:

[remap]

importer="texture"
type="CompressedTexture2D"
uid="uid://k810555el837"
path="res://.godot/imported/player.png-0c8880c21a1280c073ecfb89c7ec2ba3.ctex"
metadata={
"vram_texture": false
}

[deps]

source_file="res://worldobjects/player/player.png"
dest_files=["res://.godot/imported/player.png-0c8880c21a1280c073ecfb89c7ec2ba3.ctex"]

[params]

compress/mode=0
compress/lossy_quality=0.7
compress/hdr_compression=1
compress/bptc_ldr=0
compress/normal_map=0
compress/channel_pack=0
mipmaps/generate=false
mipmaps/limit=-1
roughness/mode=1
roughness/src_normal=""
process/fix_alpha_border=false
process/premult_alpha=false
process/normal_map_invert_y=false
process/hdr_as_srgb=false
process/hdr_clamp_exposure=false
process/size_limit=0
detect_3d/compress_to=0

KoBeWi · 2022-10-09T23:00:03Z

Ok I checked again and the warnings seem to still appear. The PR fixes the warning before editor load, but there seem to be more warnings after it's loaded. Maybe caused by scan?

EDIT:
lol yes. The cache is cleared because "Will be updated on scan.", but then scan will spam warnings caused by lack of cache, omg xddd

So either a solution would be not clearing the cache (which would make it grow infinitely, storing entries for deleted files) or add a fallback similar to #67124 EDITEDIT: No, fallback makes no sense, it's a reverse situation xd I think we could just silence the warning by verifying the UID with ResourceLoader.

bitbrain · 2022-10-09T23:04:43Z

We might get away with just merging this PR (as it brings value already) but this means #64881 won't be fixed yet.

KoBeWi · 2022-10-10T00:03:14Z

I added a brutal silencer for the warning. If the UID does not exist in cache, ResourceLoader will validate whether UID and path still match. @bitbrain check again if it fixes the warnings for you.

The deferred call from the original fix is still needed for using UIDs in plugins (otherwise they fails, not just warn).

bitbrain · 2022-10-10T06:35:34Z

No more warnings happening here!

core/io/resource_format_binary.cpp

akien-mga · 2022-10-10T11:56:25Z

Thanks!

InfiniteProductions · 2023-02-23T11:39:25Z

I still have those weird warnings for a while now, using RC til the newly released RC3:

I don't remember when they were displayed first, never change anything, and not tried the tool script provided by @KoBeWi, I don't know what t think about this, it doesn't looks like #64881 has been actually fixed or maybe #67124 created those warnings in the first place.

Although everything looks like to work as expected despite the name of the loaded scene instead is not.

KoBeWi · 2023-02-23T11:57:12Z

Do you have any minimal reproduction project? Does saving the problematic scenes again fix the issue?

InfiniteProductions · 2023-02-23T13:25:01Z

Do you have any minimal reproduction project? Does saving the problematic scenes again fix the issue?

No I don't and yes, saving the scenes again makes the warning no longer displayed.

Thanks.

Zimmabue · 2023-03-06T12:31:55Z

I'm having these warnings from rc if I remember right, and they still appear in 4.0
Every time I open the project the warnings are from different resources. If I run the game or save the scene they still appear.

KoBeWi · 2023-03-06T12:33:47Z

Is the UID always the same?
The warnings come from external resources, so you need to save the resource file, not the scene.

Zimmabue · 2023-03-06T13:00:44Z

Ok I saved the tscn and reimported glb (in which I have external materials setted) files and warnings disappeared. Why do they appear? May I did something wrong?

KoBeWi · 2023-03-06T13:03:13Z

When were the files created?
The warnings might appear if you do some changes to the file outside Godot. Although this shouldn't be happening when you modify the glb file, so maybe it's some importer bug.

Zimmabue · 2023-03-06T13:08:39Z

If I remember right I delete a plug-in from the file system (wrong move, I did it without thinking too much). I had warnings related to the missing plug-in that I managed to resolve and then this warnings appeared. But I don't know if they are related

KoBeWi · 2023-03-06T14:43:35Z

If you do changes to files that affect UIDs, like moving the file, renaming etc, it will cause UID mismatch. Each UID corresponds to a specific path, which is stored in a cache file. Godot updates it automatically when you manipulate the files (it even updates all usages of an UID inside scenes/resources), but if you e.g. externally duplicate or move the file, it will get detected as a new resource and cause all kinds of weird things.

Some related issues:
#73916
#68661

eh-jogos · 2023-04-05T11:57:40Z

You can actually fix this without opening the scenes (manually). Just run this script in your project:

@tool
extends EditorScript

var files: Array[String]

func _run() -> void:
	files = []
	
	add_files("res://")
	
	for file in files:
		print(file)
		var res = load(file)
		ResourceSaver.save(res)

func add_files(dir: String):
	for file in DirAccess.get_files_at(dir):
		if file.get_extension() == "tscn" or file.get_extension() == "tres":
			files.append(dir.path_join(file))
	
	for dr in DirAccess.get_directories_at(dir):
		add_files(dir.path_join(dr))

It will open and re-save all text resource files, to make sure they are up to date. It fixes formatting, uids, old classes etc.

I get this warnings a lot not on the editor but on exported projects. This snippet really helped to automate "fixing" it Thanks a lot!

But there are some strange things:

The amount of changes on git after running this script doesn't match the amount of files I receive warnings for. And I've noticed it sometimes that the UID in the scene's "external resources" listing and in the external resource itself match perfectly, and then just opening the scene and saving usually fixed the warning even though the uid itself didn't change.
I also have to re import all my pngs as I receive warnings for almost all of them when exporting the game as well.

Am I doing something wrong, or is this normal?

I don't think I edited these files externally since last export, I might have moved the images and a lot scenes/resources as I reorganized the project folder but I always do that through the editor.

Sometimes I do save scenes and scripts in VScode, as I use it for git, but it's usually throwing away debug prints when reviewing changes before commit, or sometimes when I want to split some changes between commits, I commit some lines and not other from files.

In the end everything gets committed, but there might be some file saving between them. Could that be it?

pennyloafers · 2023-07-21T02:25:01Z

@KoBeWi 's editor script worked for me on 4.1. I had to also add ".material" resources as well. Thanks

dzsqxhy · 2024-08-20T16:45:05Z

You can actually fix this without opening the scenes (manually). Just run this script in your project:

@tool
extends EditorScript

var files: Array[String]

func _run() -> void:
	files = []
	
	add_files("res://")
	
	for file in files:
		print(file)
		var res = load(file)
		ResourceSaver.save(res)

func add_files(dir: String):
	for file in DirAccess.get_files_at(dir):
		if file.get_extension() == "tscn" or file.get_extension() == "tres":
			files.append(dir.path_join(file))
	
	for dr in DirAccess.get_directories_at(dir):
		add_files(dir.path_join(dr))

It will open and re-save all text resource files, to make sure they are up to date. It fixes formatting, uids, old classes etc.

Thanks very much. This resolved an issue that had been bothering me for a long time.

KoBeWi added bug topic:editor topic:plugin labels Oct 9, 2022

KoBeWi added this to the 4.0 milestone Oct 9, 2022

akien-mga approved these changes Oct 9, 2022

View reviewed changes

KoBeWi force-pushed the call_deferred_always_works branch from b3ecfee to 799a545 Compare October 10, 2022 00:01

KoBeWi requested a review from a team as a code owner October 10, 2022 00:01

akien-mga reviewed Oct 10, 2022

View reviewed changes

core/io/resource_format_binary.cpp Outdated Show resolved Hide resolved

Defer clearing of ResourceUID cache and silence warnings

14b82c3

KoBeWi force-pushed the call_deferred_always_works branch from 799a545 to 14b82c3 Compare October 10, 2022 09:20

akien-mga merged commit 5616898 into godotengine:master Oct 10, 2022

KoBeWi deleted the call_deferred_always_works branch October 10, 2022 12:01

akien-mga mentioned this pull request Oct 10, 2022

Fallback to ResourceLoader if can't find UID #67124

Merged

TokisanGames mentioned this pull request Apr 6, 2023

Add documentation on how to fix corrupt UIDs in a project (workaround inside) #68661

Open

KoBeWi mentioned this pull request Nov 8, 2023

Excessive warnings about invalid UUID in ext_resource #64881

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Defer clearing of ResourceUID cache #67128

Defer clearing of ResourceUID cache #67128

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022 •

edited

Loading

bitbrain commented Oct 9, 2022 •

edited

Loading

KoBeWi commented Oct 10, 2022

bitbrain commented Oct 10, 2022

akien-mga commented Oct 10, 2022

InfiniteProductions commented Feb 23, 2023

KoBeWi commented Feb 23, 2023

InfiniteProductions commented Feb 23, 2023 •

edited

Loading

Zimmabue commented Mar 6, 2023

KoBeWi commented Mar 6, 2023

Zimmabue commented Mar 6, 2023 •

edited

Loading

KoBeWi commented Mar 6, 2023

Zimmabue commented Mar 6, 2023 •

edited

Loading

KoBeWi commented Mar 6, 2023

eh-jogos commented Apr 5, 2023

pennyloafers commented Jul 21, 2023

dzsqxhy commented Aug 20, 2024

Defer clearing of ResourceUID cache #67128

Defer clearing of ResourceUID cache #67128

Conversation

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022

bitbrain commented Oct 9, 2022

KoBeWi commented Oct 9, 2022 • edited Loading

bitbrain commented Oct 9, 2022 • edited Loading

KoBeWi commented Oct 10, 2022

bitbrain commented Oct 10, 2022

akien-mga commented Oct 10, 2022

InfiniteProductions commented Feb 23, 2023

KoBeWi commented Feb 23, 2023

InfiniteProductions commented Feb 23, 2023 • edited Loading

Zimmabue commented Mar 6, 2023

KoBeWi commented Mar 6, 2023

Zimmabue commented Mar 6, 2023 • edited Loading

KoBeWi commented Mar 6, 2023

Zimmabue commented Mar 6, 2023 • edited Loading

KoBeWi commented Mar 6, 2023

eh-jogos commented Apr 5, 2023

pennyloafers commented Jul 21, 2023

dzsqxhy commented Aug 20, 2024

KoBeWi commented Oct 9, 2022 •

edited

Loading

bitbrain commented Oct 9, 2022 •

edited

Loading

InfiniteProductions commented Feb 23, 2023 •

edited

Loading

Zimmabue commented Mar 6, 2023 •

edited

Loading

Zimmabue commented Mar 6, 2023 •

edited

Loading