Variables bigger than a certain size are displayed as null in the editor debugger

[ghost]

---

Bugsquad note: This issue has been confirmed several times already. No need to confirm it further.

---

Godot version:

0eaefa7

OS/device including version:

Win10 64-bit GLES2

Issue description:

I have a loop that will take random PackedScenes out of an Array and copy them into a new Array. (A shuffling thing.)

It works fine until it gets up to 10 in the iteration. Then the debugger shows that it goes null. However, a print() will show about 11 of the 30 expected elements. It will behave as null from there on and crash later.

	var list = []

	var set  = sets[0].set
	var size = set.size()

	set.append(set[0])

	for n in p_number:
		
		var ridx = RNG.rand_int()
		var tscn = set[ridx % size]

		list.push_back(tscn)

	return list

Prints some of the elements, even though it claims it is null.

Goes null during 10th execution...

@akien-mga Has there been any GScript commits lately that could be causing a regression and I can check against?

@vnen Might you know of a way to better isolate this?

Steps to reproduce:
Use the project below and set breakpoints to inspect the array. Not sure yet of an operation that will crash it all the time, as the values are in some sort of limbo, sometimes they behave null, sometimes they work, even when the debugger claims null.

Observe the enemies Array, and the first and last element.

1. The Array enemies is null.
2. The first element, first_tscn has a value.
3. The last element, last_tscn is null.

Minimal reproduction project:

Reducing the total enemies in the wave property below 7085 will get normal behavior.

NOTE: Use breakpoints to inspect results in wave.gd

Arrays Going Null.zip

[akien-mga]

Has there been any GScript commits lately that could be causing a regression and I can check against?

There might be, but I'd need to know an estimate of what was the last working release/commit for you to look over the changelog/bisect.

[ghost]

The oldest built version at hand is from f5a3d34.

Unfortunately still has the problem though. So either further back, or I've just never run into before until I hit the magic combination. I'll try building something from weeks ago.

It is very bizarre. If I lift the code out in the same project, and load the scene with a literal path on the same file, it works fine in that context.

[ghost]

Well seems unlikely to be a regression of anything recent, and just something that happened for the first time today.

I went back as far as:
fc5dad4

Still having the issue. Another thing that is odd about it, is why it completes the loop when there are push_back() calls in it, and it says it the Array has gone null.

[ghost]

@akien-mga With some luck, was able to reproduce it. The magic number of Array elements in this MRP is 8075. Using 8074, all is fine, add one more and the Array goes into some limbo state between null and functional.

[ghost]

Another odd observation, upon reloading and testing it, the last element is appearing, though the Array is still reporting null. Hopefully others are able to reproduce at least some part of it.

In contrast to:

[akien-mga]

Observe the enemies Array, and the first and last element.

1. The Array enemies is null.
2. The first element, first_tscn has a value.
3. The last element, last_tscn is null.

Tested and I can confirm the issue with the current 3.2 branch (784e854).

However, AFAIK the fact that last_tscn is null at this breakpoint is not a bug, it's just that the breakpoint happens at the start of the line, so last_tscn hasn't been initialized yet. Stepping to the next statement makes it get a value as expected.

On the other hand the enemies Array does become [null] in the debugger representation. This might just be a debugger bug though and not an Array or GDScript issue. Do you have evidence of it actually behaving like null in game logic, or is it only displayed as [null] in the debugger but still working properly?

I tried adding var is_array = typeof(enemies) == TYPE_ARRAY and this is true in the debugger even when enemies is shown as [null]. The fact that first_tscn and last_tscn can be initialized without error also means that the array is still valid since .front() and .back() work.

CC @Faless as this might be a debugger-specific issue.

[akien-mga]

Yeah this seems to confirm that it's a debugger-specific issue, as the remote scene tree shows the Array properly and can inspect it:

Most likely the debugger can't serialize the big array and returns [null] instead of a clearer error label.

[ghost]

Do you have evidence of it actually behaving like null in game logic, or is it only displayed as [null] in the debugger but still working properly?

Thanks for reviewing.

Currently, no evidence for elements to be treated as null. You're right the MRP isn't reproducing it.

I have a tag to an old commit, but my memories on this issue have grown quite cold. I recall that it came to my attention because instance() was crashing and showing the variable containing some later element as null.

If this memory of mine is trustworthy, evidence may exist, but I wouldn't have the time to reacquaint myself with how to forcefully recreate the problem in the project. It was very nettlesome to trigger originally, as it doesn't happen in most play throughs.

[KoBeWi]

Can this be reproduced in the latest master? MRP is outdated.

[developingwoot]

I'm having the problem using 3.4.4 Stable. I have a folder that has a bunch of mp3's in it and I wanted to add them to an array and then play random songs.

extends Node2D

var path = "res://Songs"

onready var directory = Directory.new()

export var song_list = []
var file_name
var loaded_path

func _ready():
	song_list = []
	directory.open(path)
	directory.list_dir_begin()
	while true:
		file_name = directory.get_next()
		if file_name == "":
			break
		elif !file_name.begins_with(".") and !file_name.ends_with("import"):
			print(file_name)
			loaded_path = load(path + '/' + file_name)
			song_list.push_back(loaded_path)
	directory.list_dir_end()

Hopefully this might help test the issue

[Zireael07]

@developingwoot is the local song_list in ready intended? because as things are, you are shadowing the global (exported) song_list...

[developingwoot]

@Zireael07 thanks for pointing that out I didn't realize it worked like that. No that's not intended I added it during troubleshooting this issue. Once I realized (based on this thread) that the issue was related to the load() call I changed my code to below and everything works, I just call the load() method when I assign the song in the AudioStreamPlayer and everything works great.

extends Node2D

var path = "res://Songs"

onready var directory = Directory.new()

export var songs = []

# Called when the node enters the scene tree for the first time.
func _ready():
	directory.open(path)
	directory.list_dir_begin()
	while true:
		var file_name = directory.get_next()
		if file_name == "":
			break
		elif !file_name.begins_with(".") and !file_name.ends_with("import"):
			songs.append(path + '/' + file_name)
	directory.list_dir_end()

[yatesm4]

Its still an issue in Godot 3.5, just throwin that out there. Been chugging away at a WFC impl and this god damn array literally just vanishes into a null after a loop for no discernable reason.

Either that, or I really just need to go to sleep but I assure you thats not the case. I've been sitting here for an hour absolutely dumbfounded.

https://gitlab.com/yesterday_development/future-shit/-/blob/master/scripts/wfc/WfcModel.gd#L63

You can check it out yourself if you want.

[KoBeWi]

Doesn't seem to happen, but maybe I'm testing wrong. I did this:

var model = preload("res://WfcModel.gd").new()
model.weights = [1]
model.t = 1
model.fmxXfmy = 1000
model._initialize()

Some instructions how to reproduce would be useful.

[yatesm4]

@KoBeWi apologies, as I alluded to I was exhausted so I didn't best describe the problem

https://gitlab.com/yesterday_development/future-shit/-/blob/master/objects/WfcTest.gd

Here is a test script, with an accompanying scene/node to run, which creates an OverlappingModel. During this:

	var model = OverlappingModel.new()
	model.construct(img_data, width, height, 3, destWidth, destHeight, true, true, 2, 102)
	model._initialize()

	var finished = model._generate(RandomNumberGenerator.new())

and the image used is the flowers.bmp image in my resources folder.

[KoBeWi]

That code is confusing. It stops on this line:

comp is null, because compat[t2] is null, because it gets a value from array full of nulls. Pretty sure you just accidentally put some null where it shouldn't be.

[yatesm4]

Yea so working our way up the pipeline, if you put a break point on line 67:

	compatible = []
	compatible.resize(fmxXfmy)
	
	compatibleh_2 = [] // Ignore this, added for debugging
	compatibleh_2.resize(fmxXfmy)

	for i in range(fmxXfmy):
		wave[i] = []
		wave[i].resize(t)
		
		compatible[i] = []
		compatible[i].resize(t)
		
		compatibleh_2[i] = []
		compatibleh_2[i].resize(t)
		
		if compatible == null:
			print("WTF")
		
		if i == (fmxXfmy - 1):
			print("Last call") // compatible array is null here, even though it wasn't null at start of loop
		
		for tt in range(t):
			compatible[i].append([0, 0, 0, 0])

At this point, I wouldn't worry about it. Switched over to Mono version and used a C# implementation and the problem is gone. 3am frustration version of myself just wanted to make sure there was some exposure on the issue

[akien-mga]

Is this still reproducible in the latest 4.x builds?

[baptr]

Is this still reproducible in the latest 4.x builds?

Yeah, I spent tonight independently discovering and debugging this at 4.0rc6+, it's definitely still an issue.

#74148 isn't so much a solution as surfacing some hint of what's going on, but in my case a large complex scene reference was sufficient to bypass the 1MiB debugger serialization limit, which I suspect was the issue for OP as well given the behavior.

[Wiggan]

I just ran into this too. Stuff looks like null after loading mp3s.

[lazaromenezes]

I just ran through this same/similar issue with v4.1.2.stable.official [399c9dc]

I'm trying to load a bunch of words tinto a PackedStringArray and in the end of the loop, my PackedStringArray shows null in the editor, even though printing it show me results, and also using it

extends Resource
class_name WordDictionary

@export_file("*.txt") var dictionary_file: String

var _dictionary: PackedStringArray = PackedStringArray()

func initialize():
	var dictionary: FileAccess = FileAccess.open(dictionary_file, FileAccess.READ)
	
	while dictionary.get_position() < dictionary.get_length():
		_dictionary.append(dictionary.get_line())
		
		assert(_dictionary != null) # doesn't fail
	
	dictionary.close() 

       	print(_dictionary) # prints the words
       	
func has_word(word):
	var idx = _dictionary.bsearch(word) # works properly
	
	return word == _dictionary[idx]

[SundaePixel]

I can still confirm that this still exists in 4.2 version

[Dragonsdoom]

I repro when loading a 13.6 MB file into an array in 4.2.1, but not when loading a 2.95 MB file. Does anyone know a workaround?

Update: I seem to repro when loading a 1.14 MB file as well, I guess the file size isn't relevant.

[chaojian-zhang]

In v4.2.1.stable.official [b09f793] I am encountering similar issue when continuously adding to a String. The string variable becomes null when the length becomes 1742383.

However, performing any string functions (e.g. substr) on this string produces valid result.
Adding this string variable as a value to a dictionary causes the operation to fail silently.

[Faless]

This is indeed a debugger issue.

If a variant is too big, the debugger will send it as null (to avoid stalling waiting for network), and will show the value as "null" in the interface.

We have a hint flag for that (PROPERTY_HINT_OBJECT_TOO_BIG) which we use for the debugger inspector so we should probably use it for the stack variables too, and change the way they displayed to something like "Variant too big".

The in-game variables, are not actually null, and will work as they should.

We can also probably make the max size adjustable (right now it should be 1MiB).

I have updated the title accordingly.

[Dragonsdoom]

I repro when loading a 13.6 MB file into an array in 4.2.1, but not when loading a 2.95 MB file. Does anyone know a workaround?

Update: I seem to repro when loading a 1.14 MB file as well, I guess the file size isn't relevant.

Turns out I was conflating this with a different error, my findings should be disregarded.

[Khalaq04]

Hello everyone, I am looking to make my first contribution and would like to take a shot at fixing this if no one is working on it yet. It would be really helpful if someone could help me reproduce this issue in the latest version. Also I would really appreciate some context, references and, resources that would help me fix it.

[RyanCross]

Arrays of a relatively small size (< 1000 elements) cause this null issue for me as of 4.2.x. This is a huge pain. :(