Increasing lag, seems to disappear on save and reload

[Marrim]

Describe the bug

When doing long activities, the game takes 1-3 seconds per 5 in-game minutes which feels pretty slow already. However, if you've been traveling around it becomes several times slower.
If you save and quit, then reload, the lag reverts to the "normal" 1-3 seconds per 5-minute interval.

Steps To Reproduce

Make a survivor, drive a car around for a few in-game hours, exploring the overmap. Then try a long action, like batch-crafting 1000 nails.
Does not reproduce if you stay in place.

Expected behavior

I understand the activity can run slower if there are more creatures nearby (or underground), because pathfinding, but I expect it to not slow down just because I had wandered the map for a while.

Versions and configuration

Consistent between versions (can get exact version numbers when I get home).
OS Ubuntu 18.04 LTS.
Terminal.
Default mod configuration (base game + no npc needs)
Wander spawns on.

[ghost]

Check your memory utilization (% free -h), map memory only gets flushed when quitting game so its possible after exploring for a while (especially with z-levels and on 2GB or less) you are running on swap and wander spawn process outside of reality bubble - thus having to swap. Its annoying to no end when playing on a toaster.

[TimPollard]

I'm pretty sure @1n17 is correct on this one; I've just got in the habit of saving and reloading (I don't need to quit) if I've been moving around a lot.

It would obviously bee good to fix this; some sort of in-game garbage collection, or perhaps auto-saving.

[anothersimulacrum]

I can trigger the performance improvement by just autosaving, but I don't notice anything too different when I run free before and after.

[ghost]

I'm pretty sure @1n17 is correct on this one; I've just got in the habit of saving and reloading (I don't need to quit) if I've been moving around a lot.

Well, the option to save (to then reload) is called 'Save and Quit', sorry if I wasn't clear enough. Although quitting the game process itself would make sure to also free any potential memory leaks.

It would obviously bee good to fix this; some sort of in-game garbage collection, or perhaps auto-saving.

Some dynamic map purging would be awesome, I wonder if the existing 'map memory' system could be utilized to efficiently implement this

[Marrim]

I checked memory usage and lack of memory doesn't seem to be the case; the process is only ever using about 2gb out of 8gb available.

[scorpion451]

As I understand it, it's not necessarily lack of memory so much as an ever-growing "things you've been near recently" list. The memory footprint and processor load might not be huge, but when there's 10 million things in that list, the couple nanoseconds it spends flipping past each item every turn starts to add up.

[Marrim]

Interesting that I just tried to load the 0.D save from my old laptop. And the game runs super fast when crafting; like 2-3 in-game hours per second.

The save itself is an underground bunker filled with goodies, farm patch, power grid "vehicle" and surroundings on the surface mostly stripped clean. (It is a very advanced save.)

EDIT: I think the general slowness has to do with the underground lab tunnels that seem to permeate everything in my current save. I keep getting messages about stuff, hearing turrets firing from below etc. wherever I walk. Is this normal?

[ghost]

Yeah just experienced this, z-level lag can certainly be a thing in certain places.

in my case however, it wasnt a z-level thing and it definitely wasnt RAM getting full up accessing swap, it was probably as scorpion451 says, I had just travelled a long distance, and then gone up a radio tower to scout a huge area, and then read an area map too to reveal a huge area.

Then went back to my base which is not on top of any z-level areas like labs or anthills, and which was previously running completely lag-free.

AFter coming back from my trip however, it was running so slow while waiting or crafting, then after a save/load , it was fine again.

Its definitely a caching thing, im thinking.

[kevingranade]

I found the culprit:

Cataclysm-DDA/src/game.cpp

Lines 1520 to 1531 in 64b5e04

	for( auto &elem : MAPBUFFER ) {
	tripoint sm_loc = elem.first;
	point sm_topleft = sm_to_ms_copy( sm_loc.xy() );
	point in_reality = m.getlocal( sm_topleft );
	submap *sm = elem.second;
	const bool in_bubble_z = m.has_zlevels() || sm_loc.z == get_levz();
	for( auto &veh : sm->vehicles ) {
	veh->idle( in_bubble_z && m.inbounds( in_reality ) );
	}
	}

In order to allow off-map-but-connected-to-on-map vehicles to keep updating, this code walks the entire tree of loaded submaps to find vehicles and processes them, this does NOT scale since on average a single step the player makes toward new map areas loads 20 new submaps, a few hours of exploring can load hundreds of thousands of submaps.

This should be totally manageable by maintaining a list of vehicles that are either on the map or connected to an off-map vehicle.

[kevingranade]

Forgot to mark #36917 as closing this.