Add documentation on how to fix corrupt UIDs in a project (workaround inside)

[RevoluPowered]

Godot version

4.0 dev

System information

all

Issue description

In some projects and others people are reporting this issue:

WARNING: res://object.tscn:4 - ext_resource, invalid UUID: uid://somebrokenuid- using text path instead: res://assets/models/somepath.tres
     at: load (scene/resources/resource_format_text.cpp:448)

I believe it's a bug between our alpha and beta godot version, we should fix it, as it appears to not just be The Mirror that has the bug.

A workaround I found I would like to share

I found a working fix for this which can resolve the problem for an entire project, its quite slow to do, but it has worked in our case to clear up all our UID issues. (this goes without saying but I'll say it: MAKE BACKUPS before you do this)

1. Use a regex find and replace (in my case I used CLion with the regex box checked) to remove all the UIDs with a space at the start of uid:// (ENSURE you keep the space at the start of uid= regex pattern)

 uid=\"uid://.*?\"

2. Then use this pattern without the space to nuke it from all the other places its used (ensure no space at start of regex pattern)

uid=\"uid://.*?\"

3. Open the engine with your project.
4. Set message queue to a very large size (required for large projects) - I used 819200kb because memory is cheap.
   
5. Reboot editor and project
6. File -> Quick Open -> Select all scenes with shift select the top and bottom for all scenes.
7. Wait about 5 minutes until all the scenes open.
8. File -> Save all scenes
9. You're not done yet.
10. Open the file browser -> Search for all .tres files
11. Click each .tres individually and click the save icon individually for each file (this took me ages but it worked)
12. Now it's important, repeat step 8 again
13. You should have a bunch of changes to .import/.tres reboot your editor and check for any UID errors in the console they should be gone.

Steps to reproduce

Unsure how the bad UIDs were introduced but they seem to disappear by doing my workaround.

Minimal reproduction project

No response

[MikeSchulze]

Hello, I now have the same problem with my update process.
This error is really frustrating!

For my use case I have an integrated plugin upgrade tool where a new version is installed when it is available.
Since the introduction of this uid the plugin starts after the update with these ominous uid problems

For more background, the uid's are a kind of cache to reference resources.
You have to in account the cache can be deleted, and the uid's must be regenerate.
Means if you delete all imported images rm -rf *.svg.import you will run into this issue.

And important when you deliver a plugin the cached resources are never be part of the plugin and you will run always into this issue.

As example i have a animated texture resource

+[ext_resource type="Texture2D" uid="uid://0d4wn0t2wnq5" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress7.svg" id="1"]
+[ext_resource type="Texture2D" uid="uid://djsa452vjyeg8" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress5.svg" id="2"]
+[ext_resource type="Texture2D" uid="uid://bk7ejnw132eyv" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress1.svg" id="3"]
+[ext_resource type="Texture2D" uid="uid://conhui4b6ubsn" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress8.svg" id="4"]
+[ext_resource type="Texture2D" uid="uid://cgemfcu6qacvx" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress2.svg" id="5"]
+[ext_resource type="Texture2D" uid="uid://bd2ro6pysahsb" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress6.svg" id="6"]
+[ext_resource type="Texture2D" uid="uid://cekd6xs0756i0" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress3.svg" id="7"]
+[ext_resource type="Texture2D" uid="uid://dovxd5j4scybs" path="res://addons/gdUnit4/src/ui/assets/spinner/Progress4.svg" id="8"]
 
 [resource]
 frames = 8
 speed_scale = 2.5
 frame_0/texture = ExtResource("3")

Each included texture (Progress*.svg) has a uid but the uid will never match when you install the plugin at first time.

Please fix this bug as soon as possible, this kind of error is confusing the user.

[Lippanon]

Deleting an .import file (the editor re-imports it) will produce this warning ("invalid UUID - using text path instead") for every scene that uses that file, clogging the output.
I have binary scenes (.scn) so the regex workaround doesn't work. Is there no way to fix these corrupt UUIDs in the editor? I've tried deleting .godot, saving scenes, none of my attempts have worked so far.

[TokisanGames]

Thanks for the workaround. Here is an improved method that works on git bash or any unix shell in windows or linux. This will find all tscn files, run perl to search and replace on the file to strip out UIDs.

$ find . -name \*.tscn -exec perl -i -pe 's/ ?uid=\"uid:\/\/[a-zA-Z0-9]+?\"//g' {} \;

You can change perl -i to perl -i~ and it will leave a backup copy of the files named .tscn~. If using git, that is unnecessary.
I only had UID problems with tscn files. If you want to also get .tres files you can change it above and run it twice, or change the part before exec: find . -name \*.tscn -o -name \*.tres

@Lippanon manually save your scenes as tscn, then revert. Or leave them as tscn. You should be using git, even as a solo dev, and things that change all the time like scenes will both fill up your git server, and not allow you to review your changes before committing. I recommend using git and tscn, tres, no .material, .scn. Only use binary files for things that will not change, such as glb and texture files.

[Lippanon]

Right clicking a file in the editor gives an option to copy the UID of that file. This can be used to manually edit the invalid/corrupt UID in scenes that use that file.
It would be nice if the editor had an option to fix invalid UIDs associated with a file, since it can already track dependencies (for example when trying to delete a file it shows where that file is in use).

[KoBeWi]

I don't know if this changed since then or was always like that, but when you re-save the scene/resource, all stored UIDs will get updated.
Godot generally updates them automatically if you change files from inside the editor.

[Lippanon]

I don't know if this changed since then or was always like that, but when you re-save the scene/resource, all stored UIDs will get updated. Godot generally updates them automatically if you change files from inside the editor.

Can confirm. I think it changed at some point. Re-saving the scene/resource fixes the invalid UIDs.

[TokisanGames]

This is still a problem, even in 4.0-stable with entirely new UIDs, new cache directories and a new UID database.

I used my command to strip out UIDs:
find . -name \*.tscn -o -name \*.tres -exec perl -i -pe 's/ ?uid=\"uid:\/\/[a-zA-Z0-9]+?\"//g' {} \;

Then I used the script below to have Godot automatically load and resave all .tres/res files in the project. It can also work on other file types or specific directories by changing the parameters.

It upgrades the format types for .tres/.res files without having to manually open them and click the save icon one by one.

-[gd_resource type="ArrayMesh" load_steps=2 format=2]
+[gd_resource type="ArrayMesh" load_steps=2 format=3 uid="uid://cr1p1x33ubgn2"]

You could add .tscn to the extensions and see if it will work with that, however in my testing it appears insufficient compared to opening all scenes and saving them.

Create a test scene, attach this script, save, close, and reopen the scene. It will process everything upon reopening, since it's a tool.

@tool
extends Node


const START_DIR = "res://"
const EXTENSIONS = [ "tres", "res" ]


func _ready():
	process_dir(START_DIR)
	
	
func process_dir(dir_name: String) -> void:
	var dir := DirAccess.open(dir_name)
	if dir:
		dir.list_dir_begin()
		var file_name = dir.get_next()
		if dir_name.ends_with("/"):
			dir_name = dir_name.trim_suffix("/")
		while file_name != "":
			if dir.current_is_dir():
				print("----- New directory: " + dir_name + "/" + file_name + " -----")
				process_dir(dir_name + "/" + file_name)
			elif file_name.get_extension() in EXTENSIONS:
				print("Processing file: " + dir_name + "/" + file_name)
				var res := ResourceLoader.load(dir_name + "/" + file_name)
				ResourceSaver.save(res)
				
			file_name = dir.get_next()
	else:
		print("An error occurred when trying to access the path: ", dir_name)

[naturally-intelligent]

I've been having non-stop problems with UIDs since upgrading from v3 to v4.

I had to not only copy all .import files from one machine to another, but also the entire .godot folder, or the project could not re-generate from a clean slate (editor crashes). Which is really disturbing... because I thought those files were unnecessary to track... but copying .godot folder and .imports from one machine to another fixed the issue, so I moved on, not wanting to deal with it

Anyhow as time went on I began to create more assets like .PNG files and commit them, along with their .import files and now on different machines the UIDs are conflicting because Godot generates different UIDs in the .tscn files on different machines, and slowly my project is filling up with yellow resource warnings in the console, and I have to comb through diffs to see if files really changed or is it just the UIDs changing again....

@TokisanGames is this the same issue you are solving with your script above?

[naturally-intelligent]

What is the standard practice with .import and .godot folder? Do we commit them or not?

Seems like either choice causes headaches

[TokisanGames]

Commit *.import files.

.import folder in the project is gd3 and can be deleted.

.godot folder in the project contains gd4 local caches and should not normally be committed. It normally can be deleted and everything will be regenerated. If your system can't generate it, that suggests a problem with your system or your assets. Divide and conquer to find out which asset causes the crash.

However there is a bug #75388 that basically means .godot/global_script_class_cache.cfg must be committed since it doesn't generate properly in certain instances.

[KoBeWi]

You should commit .import files (e.g. icon.png.import), but not the .import (3.x) and .godot (4.x) folders. A minimalist .gitignore file for Godot is just this:

.godot/

I followed this workflow and don't have any UID problems in my project.

If you experience some UID-related bugs, try making a minimal project that reproduces it.

[TokisanGames]

I followed this workflow and don't have any UID problems in my project.

Do you have any Image subresources? Do you have a custom theme? The UID bug described in this issue is still undoubtedly present. My theme and subresources are still changing every day even after wiping all UIDs from every scene and resource file and the cache.

The workaround in this ticket does remove UIDs. It does not workaround the bug at all as UIDs still keep changing.

Original issue #62258, which was closed but not fixed by #72257
Related to #64881, which was closed but not fixed by #67128
Related #68672
Cause of all of this #50786

Make a new theme, attach it to a scene, and save it. Restart, save the scene again, UIDs changed.

MRP 4.0.1. Backup new_theme.tres. Open the project and control.tscn. Save it. Diff new_theme.tres with the backup.

test_uid.zip

 [sub_resource type="ImageTexture" id="ImageTexture_vm0ms"]
-image = SubResource("Image_yfsjn")
+image = SubResource("Image_0toyg")

[naturally-intelligent]

@KoBeWi Have you tried adding new resources to your project from different machines? I believe that's where most of my UID issues come from.

I also get UIDs in .tres tileset resources changing from machine to machine, but it seems like a separate issue.

[KoBeWi]

@TokisanGames Why does your theme have binary Image data saved as text?
It looks like you have a font embedded inside your theme. You should not embed fonts in your theme, see this comment:#62258 (comment)

Also this is unrelated to UIDs. UIDs identify resource files, sub-resource ids are a different thing.

[TokisanGames]

This isn't my game's theme. It's the default theme saved to a file. You asked for an MRP to highlight the problem. Whether there is binary data converted to text in the file (which is fully supported by Godot) is irrelevant. The problem is Godot changes the IDs on every save. The problem is not limited to themes, I experience it in countless files. Making a theme off of the default is just the fastest way to consistently show the issue.

Also this is unrelated to UIDs. UIDs identify resource files, sub-resource ids are a different thing.

This is the first time you've mentioned this though we've been talking about it since October as noted on the other ticket. I didn't know UIDs and sub resource IDs are different. Nevertheless, the sub resource IDs change every day and it's a problem for Git and all of us developers.

So what do you want to do? There are many developers with this problem. If you've fixed all of the UID problems, then all of us experiencing problems have subresource ID problems. There are already several tickets related but closed, a few that are open. Do you want to start another brand new ticket with the subresource ID issue or continue talking about it in the ongoing discussions?

[KoBeWi]

The Image does not change its id. The problem in the MRP you attached is that the font generates new image every time. It's a different image, hence it has different id. The comment I linked explains it.

It's the default theme saved to a file.

Then it's a problem with the default theme. It should not be embedding Fonts, because it leads to the aforementioned issue.

Do you know any bug with sub-resource ids that doesn't involve fonts?

So what do you want to do? There are many developers with this problem. If you've fixed all of the UID problems, then all of us experiencing problems have subresource ID problems.

Sub-resource ids don't just change randomly. If it happens, it's either caused by some buggy logic involving a specific resource or some wrong usage of resources. I think every such case should be reported as a separate issue.

[TokisanGames]

The Image does not change its id. The problem in the MRP you attached is that the font generates new image every time. It's a different image, hence it has different id.

I see. I experience it on a resource that my enemies use for their stats. Stats is a resource table marked local_to_scene, so each enemy can have different stats without affecting the template. Godot is regenerating a new ID on every save, even though the resource is identical to the old one, just like in the themes. That's still a problem for devs who use Git. It should retain the ID.

eg.

[ext_resource type="PackedScene" uid="uid://bhiovpobmq11l" path="res://src/units/enemy/pig/Pig.tscn" id="34_326jl"]
[ext_resource type="Script" path="res://src/units/stats/Stats.gd" id="27_usq0v"]

-[sub_resource type="Resource" id="Resource_78sg4"]
+[sub_resource type="Resource" id="Resource_2a1du"]
resource_local_to_scene = true
script = ExtResource("27_usq0v")
gold = 0
max_health = 3
max_stamina = 0
attack = 1
defense = 1
level = 1
max_level = 0
experience_given = 2
speed = 5.0
sprint_speed = 7.0
jump_speed = 7.0
rotate_speed = 0.15

[node name="Pig" parent="Animals" instance=ExtResource("34_326jl")]
transform = Transform3D(0.449431, 0, 0.893315, 0, 1, 0, -0.893315, 0, 0.449431, 105.575, 7.21529, 116.409)
-_stats = SubResource("Resource_78sg4")
+_stats = SubResource("Resource_2a1du")

Edit: My theme and these Stats are probably 95-100% of my subresource id issues. Now that I know the cause, I can probably eliminate all of my subresource id issues on my own.

[KoBeWi]

AFAIK local_to_scene will duplicate the resource each time a scene is instantiated. We'd need some logic to ensure it happens only once 🤔

[TokisanGames]

I was able to workaround the issues with my subresource IDs.

We learned today that UIDs are for external files, and have likely been fixed. However you may need to strip out all UIDs in all scenes and resources and let Godot generate new ones using the workarounds in this ticket.

Subresource IDs are for resources saved in the scene file. Godot regenerates new IDs for them on every save. This problem should be fixed in the engine by reusing the same ID on resources that haven't changed. However, I was able to work around them on my project with the following.

External resources marked "resource_local_to_scene"
I don't really need this, so I removed the flag. Godot kept re-enabling it on some of my scenes and resources, so getting it to stick was a challenge at first.

Themes
My custom theme was generated by importing defaults within Godot then customizing it and saving it as an external file. However, this process will create forever changing IDs as there are a lot of embedded icons and widget textures that get included. I went through my file and identified the resources that included textures saved in the file. Fonts, icons, and styleboxes with textures. Then I went through the theme editor and deleted the imported controls with those elements so my UI will use the embedded defaults. If I want to customize a control that includes a font/icon/stylebox, that item needs to be saved to disk, outside of the theme resource file or it will become a changing subresource.

So far this seems to have addressed everything. Git still reports that Godot has changed line endings on many resources and scenes, so the files show as changed. Though they disappear when staged.

Cc @lesleyrs

[Rindbee]

AFAIK local_to_scene will duplicate the resource each time a scene is instantiated. We'd need some logic to ensure it happens only once thinking

This is mainly because the copy of the resource is not associated with the information of the original resource (uid, id), so the resource information cannot be saved as the original resource when the copy is saved. It forgets where it came from.

[Koyper]

This PR exposes the rename_dependencies() method to ResourceLoader that can remove the non-existing UID paths.
#73884

for check_resource in ResourceLoader.get_dependencies("res://MyScene.tscn"):
	if not ResourceLoader.exists(check_resource) and check_resource.substr(0, 4) == "uid:":
		prints("UID missing:", check_resource)
		ResourceLoader.rename_dependencies("res://MyScene.tscn", {check_resource:""})

Renaming the UID path to an empty string seems to cause the UID reference to be completely removed from the external reference, replacing it with the path instead.

You can preferably pass the Dictionary argument with all of the bad UID's in a single call.

This gives a clue as to what's happening to cause the invalid UID errors: the get_dependencies() call only returns the UID path if it exists in the external reference text, and skips the path name.

I wonder why get_dependencies can't check to ensure it returns a valid UID and return the path instead after removing the bogus UID?

[Koyper]

What should be implemented is at the location where the error is thrown regarding the missing UID, rename_dependencies is called to automatically remove the offending UID from the scene file. The error can then be a warning instead.

[jitspoe]

Just wanted to note that this is also happening with assets from the AssetLib. I tried Ridiculous Coding on the 4.1 RC3 and got a bunch of spam about UID's. I tried opening the tscn files to rename them, but Godot said the files were corrupt. Interestingly, the addon still works aside from all the UID spam.

https://github.com/jotson/ridiculous_coding/tree/godot4

Edit: Might be a problem with that specific asset because it doesn't have the .import files checked in.

[dtesniere]

Hello,
I may have bad practices, but I made a minimal reproduction project.
I just need 3 scenes : "a, b, c"
And make some "link" between them though a "clickable" Area2D and the "get_tree().change_scene_to_packed" on an exported "packedScene".

I do not really understand what is behind this bug reproduction, but I share this as there are actually no minimal reproduction example for the moment.

When scene_c are loaded from scene_b, so scene_c seems to not load the link back to scene_b

Important note :
This bug seems to "randomly" corrupt the ".tscn" file
Importing the joined project is possible, but Godot won't open scene_b and scene_c
But it is possible to run the project.

To see the error, running the project is enough.
But you can try the "link" between scene by clicking on the "Godot red Icone" and then in the "godot normal icon".

Here is the project : issues68661.zip

(PS : I tried the "workarround" proposed in the previous comment, but, my project seems to "re-generate" the bugs. So it doesn't work here)

[KoBeWi]

@dtesniere Scene B depends on Scene C and vice-versa. You created a cyclic dependency, which is not supported.
Seems unrelated to this issue.

[MrJox]

This still happens and leads to headaches when multiple people work on the project and use Git.
It can lead to merge conflicts that cannot be auto-resolved if multiple people touched the same file.

[KoBeWi]

Merge conflicts are inevitable, but they are easy to resolve now, because every sub-resource in scene has unique ID instead of sequential number.

[MrJox]

I figured the culprit why we had this issue in the first place.
Our .gitignore file excluded all *.import files from repo. Meaning, everyone on the project had their own local copies of *.import files with their own UIDs, which didn't match the UIDs referenced in scene (*.tscn) files that were hosted in Git. This led to a mismatch between UIDs, obviously and caused those merge conflict headaches.

I don't know whether it was our mistake to exclude import files or whether Godot generated the .gitignore file that way.

[naturally-intelligent]

This still happens and leads to headaches when multiple people work on the project and use Git. It can lead to merge conflicts that cannot be auto-resolved if multiple people touched the same file.

The only way I can keep sanity is to have ONLY ONE machine generating and adding new .import files. The other machines get spammed with yellow warnings but at least the repo stays clean.

There must be something that is causing same-project UIDs to be generated differently on different machines, maybe Godot is using home/absolute directory or something machine-dependent as the seed?

[Morokiane]

Going from 4.2.2 to 4.3RC3 is causing this error on a few of my scenes.