Cannot import any resource from any folder other than res://

[TwistedTwigleg]

Godot version:
3.0.2

OS/device including version:
Windows 10, 64 bit

Issue description:
There does not seem to be any way to import/load any resource file outside of res://

I've tried importing files with various methods (see the example project for three methods for importing texture files). I've tried importing .png, .jpg, .dae, and .obj files from user:// and other folders (not in res://) with no avail.

According to the docs, the host file system can be used, but I've had no luck.

(Not sure if this is needed, but I'll explain anyway)
The reason I want to load textures/models from folders outside of res:// is to make a PBR texture creation tool similar to Surforge. I did some initial tests and it seems it's possible to make such a system in Godot (at least a simple version), and having the ability to load textures/models would be extremely helpful.

Steps to reproduce:
Try to load any resource outside of res://

Minimal reproduction project:
LoadFileExample.zip

[LinuxUserGD]

Could be the new issue for #17748 (exactly 100 issues before)

[TwistedTwigleg]

I looked at that, but even in user:// it was not working for .png, dae, or .obj files.

I just tried with the example project, and it doesn't load images from user://, even with code similar ot that posed in issue #17748 (the code changed slightly to work with images).

---

Kinda off topic, but...

Is there a way to load meshes like in the issue linked above? I looked but I didn't see anything for loading meshes from code.

[LinuxUserGD]

It seems like the .data property doesn't exist in VideoStreamWebm and I don't know any replacement.

[TwistedTwigleg]

No worries! I found a way around the issue for my project.

[LinuxUserGD]

So it's possible to load videos that are not packaged? Or do you mean a way around in general?

[TwistedTwigleg]

I mean in general. In the end I decided to split my model viewer into it's own project, where I dump the textures/models/scenes/others I want to load into that project's resource folder.

Sorry. I wish I had a better solution to offer.

[TwistedTwigleg]

Well, I found a interesting way around the problem, it is just a bit more involved.
You can convert files into something that is readable in Godot by using the get_var and store_var functions provided in the File class.

All it requires is placing the image in a Godot project, and then saving it out using get_var into a custom file. Then you can load it from anywhere and in any project. Here is the code I used to save and load image files:

# You can use this function in any Godot project, as long as it has a image file.
func create_file_selected(filepath):
	var image_file = load("res://test_image.jpg");
	image_file = image_file.get_data();
	
	var new_file = File.new();
	new_file.open(filepath, new_file.WRITE);
	new_file.store_var(image_file);
	new_file.close();

# Now delete test_image.jpg, it's .import file, and it's files in the .import folder!
# Then you can load and use the image from anywhere in the file system using the following code:
# (It can even be a completely different project!)

func load_file_selected(filepath):
	var GImage_File = File.new();
	
	if (GImage_File.file_exists(filepath)):
		GImage_File.open(filepath, GImage_File.READ);
		
		var g_texture = GImage_File.get_var();
		var new_img = ImageTexture.new();
		new_img.create_from_image(g_texture);
		
		GImage_File.close();
		
		get_node("Sprite").texture = new_img;

I can provide a test project if you guys want. I think you can use the same method to save almost any resource in Godot. I've tried saving meshes this way, but I haven't quite figured out how to do it.

This is probably more trouble than it's worth, but for anyone looking for a way to load files from anywhere in the file system, this method seems to work nicely.

[TwistedTwigleg]

I know this is totally off topic from this issue, and I promise this is the last off-topic post I'll make on this issue. I just wanted to share an update for anyone trying to load files outside a project, as I found it interesting and it may help others with similar interests/problems

---

Since I last posted, I figured out how to save and load meshes. Turns out it works almost exactly the same as saving images. You just need to use file.store_var(mesh_instance.mesh) to save meshes, and mesh_instance.mesh = file.get_var() to load meshes using the File class.

So, after figuring this out, I decided to take a small detour and see if I can save/load entire scenes using the File class. Turns out, you can!

You can save ANY node to a file using store_var and retrieve it using get_var. Just to be safe, I duplicate the nodes using node.duplicate(true) just to make sure I am not editing the nodes actually in the scene, but I do not think that it is necessary per say (though I have not tested either!)

All of the properties from the node will be saved, with only resources being lost on export. You can get around this by saving the resources as well (like Image resources on Sprite nodes) and then reassigning them once you have loaded the file.

One interesting thing to note is that nodes will forget their name when they are stored using get_var. This means you will need to store the names of all the nodes in the scene somewhere so you can reassign them once you have loaded the nodes using get_var.

Another interesting problem to work around is rebuilding the scene tree. I found a way around this by storing a JSON dictionary with the names of the children for each node in the scene. Then it is just a matter rebuilding the node tree using the JSON dictionary and a depth-first algorithm.

---

So after a few days of tinkering, I have successfully made a editor plugin that saves and loads .dscn (Deep-Copy Scene) files to and from the editor. (Made entirely in GDScript!)

Right now the plugin only has resource support for Sprites and Polygon2D nodes, but some nodes already work without needing resource support. (Like AnimationPlayer nodes).

In theory I should be able to support most, if not all, of the default nodes Godot provides, as only a few nodes need custom resource support.

Using .dscn files (or files made using the same method), you should be able to do the following, which I think is not currently doable using normal .tscn or .scn files:

• Open scene files anywhere on the file system.
  • .tscn and .scn files can only be opened in res:// and user://. Because .dscn files use the File class, you can load scenes anywhere the File class can access.
  • In theory you may be able to send and receive .dscn files using the high level networking API, which would allow for easy sharing of .dscn files, but I'm not 100% sure if this will work or not.
• Save and load user made scene files (from anywhere File can access) while the game is running.
  • This allows for super easy mod support!
• Easily change/patch scene files without having to export a new .pkg file.
  • Users just have to download a new .dscn file and replace the old .dscn file to use the patch.

The one big downside with .dscn files is they are a tad more heavy when it comes to data storage. A simple .tscn scene is 3kb, while the same scene as is 22kb as a .dscn files. Most of the added size comes from embedding resources, like images, into the .dscn file, and from data that has to be stored to parse the .dscn file.

A minor downside is that in order to make a .dscn file, you have to use File, which means you have to have whatever you want to save in a Godot project. This means you cannot load image files without first converting them to .dscn files in a Godot project. This is not a huge issue for me, but it's worth mentioning.

So if anyone is looking to load scene files from anywhere in the file system, I have a plugin that (hopefully) will do exactly that. Right now the code messy, poorly documented, and there are still a lot of nodes missing resource support. I do not really know when I will be able to release it, but I'm hoping for sooner rather than later.

Right now I am struggling to get scripts from nodes in the open scene in the editor. It seems get_script() does not work with nodes in the editor, or I am going about it the wrong way. If anyone knows how to get scripts from nodes using a EditorPlugin, please let me know, as it would be awesome to be able to save/load scripts as well!

[KoBeWi]

Related: #14879

[bitdom8]

https://github.com/TwistedTwigleg Is the solution you find eligible also for windows?

[TwistedTwigleg]

@bitdom8 It should work on Windows. It utilizes Godot’s File class, so as long as the File class works the same across all platforms, it should work.

Here is a link to the repository, if you want to try it: https://github.com/TwistedTwigleg/DSCN_Plugin

[bitdom8]

@TwistedTwigleg Thanks, I just want to make my filedialog available for users to import from windows file system. DSCN not needed I guess

[smartin015]

AFAICT this solution still doesn't work for importing assets from user:// that were never previously imported by Godot, unless there's a "from_buffer" style method that lets you dump raw data into a container object.

I have a similar use case as the one here (trying to load a mesh at runtime from Google Poly). If I knew how .msh files are stored I could try to rewrite the OBJ format into .msh... but that all seems rather silly since Godot clearly knows how to import OBJ files and will happily load them from res://.

[Calinou]

@smartin015 Some people wrote custom OBJ loaders to do this, like this one: https://github.com/Ezcha/gd-obj

[smartin015]

asks for thing

gets link to exactly the thing

I'm not used to that from commenting on years-old github issues :)

Thanks! I somehow missed this when looking around to see if it'd been done before.

[fire]

If it helps, this issue also show alternatives to loading assets from the user. #24768

[pwab]

I was banging my head against the table the last three hours because I "simply" wanted to load a jpg or png file from user://. I read all the related issues (#2319, #14879, #17748, #18367 and #24768) and had to understand the difference between loading and importing a file.

Thanks to @LinuxUserGD 's #18367 (comment) I was able to implement this functionality and it works perfectly.

Was this functionality implemented somewhere else in the meantime?
Was there any discussion that this is not a needed feature (because I often build apps/programs - not games - where loading external stuff is important) or is this just a matter of documentation?

@fire:
Are there any other types of Importers developed at this time that could be used at runtime?

[fire]

There are three in-memory importers.

• png
• jpg
• I don't remember the third one.

godot/core/image.h

Line 136 in 35e700e

static ImageMemLoadFunc _png_mem_loader_func;

[LinuxUserGD]

There was also one for ogg (see #17748 (comment) ), an .escn runtime importer (all assets in one file) from @TwistedTwigleg and this OBJViewer (https://github.com/GoldenThumbs/Godot-OBJ-viewer ).

[TwistedTwigleg]

I think .escn is the Blender exporter, if I recall correctly.

I made a .dscn (deep-copy scene file) file and plugin (Github), which uses Godot's file class to save and load resources. It unfortunately requires the files to be converted through the plugin before it can be used, in which case storing the assets in .pck files might be more useful.

[Castlemark]

I had a similar issue when trying to load images from outside the .pck once the game was compiled, but I managed to solve it with the following code:

static func get_path(path : String) -> String:
	return ProjectSettings.globalize_path(path)

static func _read_img(direction : String) -> Image:
	var path : String = get_path(direction)

	var file_exists := File.new().file_exists(path)
	if not file_exists:
		print("Error loading image at -> " + path + " <- Please check that the image exists and the name is correct")
		return null

	var img : Image = Image.new()

	img.load(path)
	return img

I don't think it's the same exact use case, but i'm gonna post it here in case someone needs it or finds it interesting.

[pwab]

Thanks @Castlemark for your input. As I wanted to display the image I had to create an ImageTexture from it. So I used your functions like this:

func _ready():
	var img = _read_img("C:/test.jpg")
	var tex = ImageTexture.new()
	tex.create_from_image(img)
	$texture_rect.texture = tex

(Project file for testing: issue-17848.zip)

Could be compressed into one function and would fit my needs perfectly (getting a texture from an external image file). I'll create an issue in the docs repository to document those functions and plugins mentioned in the comments since I asked my question. Thanks guys for your input. 👍

[Castlemark]

The code I shared was just a snippet. I have an Utils class in my project to load external images for 3D & 2D, as well as reading external json files and scanning external directories. You can find it here: https://github.com/Castlemark/JRPG-Builder/blob/master/scripts/Utilities/Utils.gd

I was thinking about maybe polishing it up some more and publishing it on the AssetLib as a standalone utility when I'm done with my project.

[bitdom8]

Your repo is remarkable and deserves asset library use

[pwab]

I'll create an issue in the docs repository

There already is one and it pretty much sums it up: godotengine/godot-docs#2148

[edoartworks]

@Castlemark sorry to bother... I'm having an issue using the snippet you suggested above.
It's working fine when I play in the editor... it correctly loads the jpg texture from a custom system path.
But. When I export the project and run the exe, that exact same texture fails to load.
This is the error I see in the exe output >

ERROR: texture_set_data: Condition "!read.ptr()" is true.
At: drivers/gles3/rasterizer_storage_gles3.cpp:836

Is there something obvious I'm missing? (I'm a Godot newbie). Some export settings it doesn't like..? I dug around all issues related to this one but I'm lost.

Tested in Godot v3.2.2, also tested on a different PC and still occuring.
Project files > [ i_17848.zip ]

[pwab]

Hey @edoartworks just a side note for you: The Github issues do not work well for helping out other people. Here you can find better alternatives: https://godotengine.org/community

Apart from that you could edit your comment to add your system info, your used Godot version and upload your project (or a minimal version of it that is reproducing this behaviour) as a zip file so anyone can take a look at it and spot possible issues.

[edoartworks]

Noted. Thanks for telling me nicely :)

[pwab]

Hey @edoartworks I looked into your project files and I think your problem can easily be solved by changing line 6 in your script from

var BG_file_path = OS.get_executable_path().replace("godot.exe", "BG.jpg")

to

var BG_file_path = OS.get_executable_path().get_base_dir() + "/BG.jpg"

Your executable_path changes from your godot.exe to the exported executable when you run an exported version of your project. The godot engine is then packed into your exe so if it isn't named 'godot.exe' your code will fail (it does because your exe is named 'i_17848.exe').

So instead of replacing the part 'godot.exe' manually you can use the builtin capabilities of the String object which is able to navigate in a path so you don't have to care how your exported exe is named as long as the BG.jpg file is next to it.

[edoartworks]

That worked! Thank you very much for taking the time <3

[fire]

Seems like people's issues are resolved. This is either a documentation problem or a feature request in the godot-proposals [for example runtime importers].

Edited:

It seems to be working as designed, maybe the hacks can be removed..