Towns have too many zombies and wrecked vehicles

[waifusd]

Describe the bug

Literally in every new town I go to there are a massive amount of zombies and wrecked vehicles. This makes the game completely unplayable. Vanilla and default world options.

Attach save file

Chatom-trimmed.tar.gz

Steps to reproduce

-Load the save file provided above
-Turn on quick setup (for convenience)
-Just visit any unexplored town

Expected behavior

According to discord, there shouldn't be this many zombies. I concur.

Screenshots

Versions and configuration

• OS: Windows
  • OS Version: 10.0.22621.2134 (22H2)
• Game Version: f666af0 [64-bit]
• Graphics Version: Tiles
• Game Language: System language []
• Mods loaded: [
  Dark Days Ahead [dda],
  Disable NPC Needs [no_npc_food],
  Portal Storms Ignore NPCs [personal_portal_storms],
  Slowdown Fungal Growth [no_fungal_growth],
  Bionic Professions [package_bionic_professions],
  Stats Through Skills [StatsThroughSkills],
  Translate Complex Dialogue [translate_dialogue]
  ]

Additional context

No response

[anoobindisguise]

I've also seen something like a thousand zombies every block in every part of town. I don't think it makes the game unplayable per se but it's definitely a lot higher than it used to be and I can't imagine it's intentional. Especially if I was on a new char and not one who had already established some resources and skills.

[waifusd]

I've also seen something like a thousand zombies every block in every part of town. I don't think it makes the game unplayable per se but it's definitely a lot higher than it used to be and I can't imagine it's intentional.

I didn't mean unplayable as in "the game constantly crashes" (even though having this many zombies around leads to a lot of errors and the game did crash a couple of times) but it's just not fun. Short of using plenty of minukes clearing this little town will take me 10 hours in real life. I like killing zombies, that's why I'm here, but apparently I don't like it this much.

If that argument is not persuasive enough... it's just not realistic. Why would a tiny town of around 15 single story buildings have a population in the multiple thousands of zombies?

[ZhilkinSerg]

I can't reproduce this on new world or new overmaps on existing world from the savegame, but already explored overmaps seems to be affected.

Can you try to load your save, debug teleport to overmap 20,20 and see if there are still too many monsters and car wreck in town on that new overmap?

[waifusd]

I can't reproduce this on new world or new overmaps on existing world from the savegame, but already explored overmaps seems to be affected.

Can you try to load your save, debug teleport to overmap 20,20 and see if there are still too many monsters and car wreck in town on that new overmap?

I loaded the save I attached to the report and teleported to 20,20 but the the closest town I could see with reveal map was at 56,49. Posting the screenshots of that place. Issue persists.

[ZhilkinSerg]

That's overmap with coordinates 0,0, not 20,20. You need to use 4 option in debug teleport.

[waifusd]

Yes, teleporting to another overmap and going to a random town I don't see the issue anymore.

[ZhilkinSerg]

Do you remember doing anything unusual during you run while you were on those monster bloated overmaps? Fiddling world settings mid-game, changing mods mid-game, playing with calendar in debug menu? Anything of that won't explain this amount of car wrecks, but it could possibly explain monster amount.

[waifusd]

Do you remember doing anything unusual during you run while you were on those monster bloated overmaps? Fiddling world settings mid-game, changing mods mid-game, playing with calendar in debug menu? Anything of that won't explain this amount of car wrecks, but it could possibly explain monster amount.

I haven't done any of that. With the exception that I added the "Translate Complex Dialogue" after meeting Rubik 50 hours or so in the savefile. I have used a little bit the debug menu but just to spawn a few items or to teleport myself, always within the same overmap and always in places that I had already explored. Never changed world settings, nor fiddled with the debug menu any deeper.

[anoobindisguise]

Do you remember doing anything unusual during you run while you were on those monster bloated overmaps? Fiddling world settings mid-game, changing mods mid-game, playing with calendar in debug menu? Anything of that won't explain this amount of car wrecks, but it could possibly explain monster amount.

Nope, nothing of the sort, but the spawns are only in cities, any place else that spawns monsters of any kind is totally unaffected. it's specifically just cities that have 50 morbillion wrecks and zombies.

[Standing-Storm]

Testing something totally different (#67840 for Magiclysm) and when I teleported into a town to check out one of the potion fruits:

Fresh world, version: b080355 [64-bit]
Mods loaded: [
Dark Days Ahead [dda],
Disable NPC Needs [no_npc_food],
Portal Storms Ignore NPCs [personal_portal_storms],
Slowdown Fungal Growth [no_fungal_growth],
Magiclysm [magiclysm]
]

[PatrikLundell]

In my case the "change" I made was to install the (at the time yesterday) latest experimental, copy the save from the experimental from about 1 week ago, remove portal storms (as has to be done every time if you hate them), restore settings to what you use (metric system, disable auto save, etc.), and restore the key bindings you want (vehicle control and debug menu).
Apart from that, it was a matter of traveling to the closest (small) town to start to clear out monsters uncovering the unsightly mess when previously unseen parts of the town was revealed.

Tested teleporting around. Eventually I found a radio tower, teleported to it, climbed it, and revealed a large, previously unrevealed, area, and teleported to a town in this new area. I encountered the same kind of mess there.

• OS: Windows
  • OS Version: 10.0.19045.3324 (22H2)
• Game Version: 5ebd5af [64-bit]
• Graphics Version: Tiles
• Game Language: System language []
• Mods loaded: [
  Dark Days Ahead [dda],
  Disable NPC Needs [no_npc_food],
  Portal Storms Ignore NPCs [personal_portal_storms],
  Slowdown Fungal Growth [no_fungal_growth]
  ]

[Dlightfull]

In my case, I also copied the save folder to a new game folder. I didn't do anything unusual besides that, did not fiddle with the debug menu at all.

[NetSysFire]

For the overpopulation we had #58946 but it got staled. The tl;dr of that issue is that place_monsters gets used a lot.

[PatrikLundell]

While the general issue of over population was attempted to be addressed by #58946 (but then left to rot, for whatever reason), that doesn't really explain the newly introduced mess. While it could well be that the rogue wreck generation bits might possibly use the wrong monster placement command for accompanying monsters, removing, scaling down (way down!), or repairing the rogue wreck placement to restore the previous behavior or improve on it is what's needed for this particular case. If the choice is to repair the wreck placement logic/weights it might well benefit from using the correct monster placement command as part of that, of course.

[ZhilkinSerg]

For the overpopulation we had #58946 but it got staled. The tl;dr of that issue is that place_monsters gets used a lot.

Yes, that is an issue and monster count probably should be proportional to building count, but recent spawn spike looks like something different. Vehicle wreck spawns are also spiked, so it must be not place_monster fault.

[PatrikLundell]

I managed to use git bisect to find which commit caused the issues. The results are:
aa6cccf is the first bad commit
commit aa6cccf
Author: Procyonae [email protected]
Date: Sun Aug 20 22:50:30 2023 +0100

No duplicate linear predecessors (#67773)

* Can't figure out save migration

* Scraps making it a flat_set

* Removes the functions for the hack

* Unborked the cherry pick

* Moves comment to a more sensible line

src/mapgen.cpp | 13 +++----------
src/mapgendata.cpp | 15 ---------------
src/mapgendata.h | 2 --
src/overmap.cpp | 6 +++++-
4 files changed, 8 insertions(+), 28 deletions(-)

I don't know how to continue from here, though, but the first option would be to roll back that commit, while the second one would be to figure out how it broke things and fix that (which wasn't intuitively obvious from my quick look at it).

(Method use to determine if a commit was good or bad:

• Copy my save from the last working version I've used:
• Use the bisect command to find and compile game versions (starting with a freshly downloaded master, to ensure everything was available).
• Start the game with that save, teleport to the town where I discovered the bug happening. Look at the scenery to see of it's a mess or not, and then teleport to the end of my vision down the street to make sure new area is uncovered. When there was a mess just south of the original teleport point there was also a mess further away when teleported there.
• Kill the game.)

[NetSysFire]

@Procyonae, could you look at the above comment?

[ZhilkinSerg]

So, possibly predecessors are not getting cleared and road mapgen is applied multiple times for same overmap terrain square, spawning monsters and wreck each time.

[PatrikLundell]

I tried to revert the change the offending PR made regarding removal of special handling of "linear", compiled it, and it removes the zombie/wreck mess. I did not change the place where new "linear" special handling was introduced, and I don't know if that has some other effect elsewhere.

The change I made was:
"predecessor_md.pop_last_predecessor();" at line 5131 (of my current master) in mapgen.cpp to
" // Gets around connections changing terrain type
if( ter.has_flag( oter_flags::line_drawing ) ) {
predecessor_md.clear_predecessors();
} else {
predecessor_md.pop_last_predecessor();
}
"
plus restoration of the clear_predecessors operation.

I can make a PR with that change, but I can't test it beyond verifying the mess disappears when applied, as I don't really understand what problem the offending PR was trying to fix (but hoping that's actually fixed by the other change made in the PR).

[Dlightfull]

Every town I discover has this problem.

Hopefully somebody can solve it, I have invested several hundred hours in this save and would hate to ditch it...

Id really like to be able to get into towns or to drive through them but with this bug it's almost impossible. I would need many many hours and resources to clear the towns from zombies and wrecks to be able to at least pass through...

[PatrikLundell]

The hack I made seems to work for me, but until there is a fix you'll have to stay away from new town tiles so as to not get any more mess created. Every town tile you've seen will remain a mess and a fix won't correct that, so feel free to clean up those for the time being (I only had two tiles, and it was bad enough... I will be a long time before I try to clear up the wreck mess, although it's a prime candidate for explosive tests (once I have any explosives)). It should be possible to debug hack a game if you know what you're doing and feel OK with "cheating". I know you can debug kill monsters, and I would assume you could also debug remove vehicles, although I've never tried to do that.

For whatever reason the person who made the change that caused it is incommunicado. There's really three ways to solve the issue:

1. Have the author fix it.
2. Roll back the change that caused the issue.
3. Apply my hack, preferably by someone who can figure out if it's reasonable. I can do it if given a go ahead (sans the sanity check, of course).

[NetSysFire]

I just pinned this since it affects all new games. Could avoid duplicates, too.

[PatrikLundell]

It affects all games that are "updated" to a broken experimental as well.

[RenechCDDA]

I managed to use git bisect to find which commit caused the issues

Did you test on a new save, a save from a previous version, or something else?

[PatrikLundell]

I tested it on my last save from before I moved to a broken experimental (a week or so older: I can look it up if the magic number for that version is important, but it probably isn't). I install each experimental as a "current" installation after renaming the previous one and then copy the save, so I tend to have the older saves available (modern hard drives take a while to fill up with junk...).

[Karol1223]

For whatever reason the person who made the change that caused it is incommunicado.

For the record, Procyonae said that they are currently in the process of moving house and are incapable of working on CDDA at the moment, but that they will return once everything there is settled. Waiting until they come back is a viable approach but it may take a while. Just throwing this out here in case this issue is being waited out with the hope of the author fixing it.

[vivyir]

Just chiming in to say that until Procyonae can restart work on CDDA we could probably apply a temporary fix, so when they come back they can do their change and test it out as well with ease of mind and crush this bug. (I've been holding back my playthrough because of this issue and I can't feasibly compile the game myself or I would D:)

[NetSysFire]

I think it would make sense to revert said PR. The changes are small enough for that.

[AlbertNewton]

I am commenting to draw attention back to this, as a solution to the issue appears to have been found.

[Procyonae]

Oh dear what have I messed up now >_>
I'm 'back' now as far as I'm moved in and have setup my PC but I'm not sure how much time I'll have in the near future to do CDDA stuff in general, I'll take a look at this now though and revert my PR if I can't figure out a solution.

[Procyonae]

I just pinned this since it affects all new games. Could avoid duplicates, too.

I can't reproduce this on a new save, as far as I can tell it should only affect saves that started between 37a45ad and the problem commit or older saves that updated during that period as the problem commit reverts the hacky solution I used in 37a45ad but ails to accommodate for saves already using the hack.
If I'm correct I'm not sure how to deal with that outside of leaving my hack on top of the cleaner + more encompassing solution (like Patricks solution above) which doesn't sound ideal, or save game migration stuff that I haven't been able to work out so I think I'll just completely revert the PR for now unless I'm mistaken and it is affecting new saves but I did test #67773 quite a bit so I'd be surprised if I messed up that badly

Nvm I can't repro this at all on new or old saves so that's helpful >o<

[Procyonae]

I can't think of a reason why this would only be affecting some peeps, I'm on windows 10 the same as the OP and can't repro on their save or a fresh exp save so I don't feel comfortable fixing this myself.
I'd prefer Patricks's hack described above to a full revert because the problem PR is needed for other stuff I'm working on and I can't think of any issue the hack would create but obviously it could do with some testing by someone who can repro the issue.

[PatrikLundell]

I downloaded and compiled the "current" master as of 10 minutes ago or so (as of when I began writing this, further testing took place since, so make it an hour or so). I then did a quick start and teleported around, seeing the same as @Procyonae , i.e. no wreck messes (although the first teleport had a weird zombie concentration of 7*8 tiles with only a few tiles without any zombie). After teleporting around a bit, including into a new overmap and not seeing anything weird(er),

I then switched to my save (which has been played with my hack), and see the mess again everywhere I look in areas visible on the overmap, but not ones which weren't revealed on the overmap before. I'm not completely sure this applies to the unrevealed areas of the current overmap, but it definitely seems to apply to other overmaps (which were completely unrevealed).

Finally I went back to my save from before my migration, and while I saw some mess that seemed worse than it should be (including a rectangle of zombies again), it wasn't the complete mess, APART from the tile I've encountered this problem on in the first place, which should be a tile I've passed close to, but just outside of the reality bubble by 1 or two overmap tiles.

Thus, it seems overmap information has been damaged somehow, although I have no idea of how that could happen or what information could have been altered to achieve this.

[waifusd]

I just did a fresh install of the latest experimental, loaded the save I uploaded here, teleported to a random town in the same overmap that I had not previously explored, and while there are possibly slightly less wrecks around, the zombie numbers are still completely out of whack.

[MacroLP]

my wrecks seem to be duplicating into each other. Although I have dragged this save through experimental's but just a few. no changes to mods or game settings. ALSO getting shittons of zombies that is taking me 10 hours to clear.

I backup all the time. I could load an older save in an older version and see the wrecks are still here.
Game version 2023-08-17-0236

zombie pops seem more... manageable

[akrieger]

@MacroLP can you upload the save and specify exactly what coordinates you teleported to, to reproduce/validate? I'm going to be bored on an airplane for many hours tomorrow and can try to bisect this but having a specific test case would be useful for testing fixes.

[akrieger]

Although I suppose I could create a game on that version and see if I can do the same experiment.

[akrieger]

Definitely bisects to #67773. I created a new save from ~version 2023-08-17-0236, saved, then to bisect I would load that save, debug map reveal, debug teleport to a far away city street intersection. It still shows lots of repeat omt types in the debug info, but also causes all the zombies and cars to respawn over and over each other. I don't think it actually did what it was supposed to do.

[akrieger]

I set a breakpoint here but this code doesn't even get hit when I teleport to the town.

Cataclysm-DDA/src/overmap.cpp

Lines 2890 to 2897 in 65c3f65

	oter_id &val = layer[p.z() + OVERMAP_DEPTH].terrain[p.xy()];
	if( id->has_flag( oter_flags::requires_predecessor ) ) {
	const oter_type_id id_type = id->get_type_id();
	// Stops linear fallback_predecessor maps (roads etc) spawning over themselves
	if( !( id_type->is_linear() && id_type == val->get_type_id() ) ) {
	predecessors_[p].push_back( val );
	}
	}

[akrieger]

Ohhh wait ok. This affects saves created prior to the PR, because those saves have the repeatedly-applied omt transforms serialized in actual save. The PR removed the on-demand logic to deduplicate that, causing the overload of wrecks and zombies as a result. So I think the right fix would be to implement save-load-time deduplication, which I think was mentioned in the PR but not implemented.

Saves created on or after the PR won't have bad data in the save file and won't reproduce the issue. It's only a forward compatibility issue, if I'm understanding right. But I haven't poked around in the code to confirm this.

[akrieger]

I have a fix that seems to make things 'better' but still not 'perfect'. For example here there's like 3 roads spawning over each other in an intersection.

[akrieger]

Nevermind, seems like it might actually have worked as it is supposed to. I was confused because I still see multicar pileups and maybe more zed density than is called for, but not the extreme amounts we were seeing before. This picture shows 3 but I saw a pile of 7 vehicles all over each other the previous time I tested this intersection.

[PatrikLundell]

3 car pileups OUGHT to be quite rare but probably not impossible. At least we don't see 5 other multi-car pileups all around it.

[akrieger]

You can check my screenshots in #68324. I think this is an incomplete fix.

[MacroLP]

@MacroLP can you upload the save and specify exactly what coordinates you teleported to, to reproduce/validate? I'm going to be bored on an airplane for many hours tomorrow and can try to bisect this but having a specific test case would be useful for testing fixes.

My bad. I knew I subscribed to this after my comment, just never refreshed the page... Absolutely.

(Gahanna)buildnumber2023-08-17-0236.zip

(Gahanna)buildnumber2023-09-06-1648.zip

You can really check anywhere between these two saves. or Level 0, 0'57, 0'167 for a good case study here.

[MacroLP]

Please let me know if that helps :). Had to scrap a ~400hr game, waved the towel and went back to stable for now.

[akrieger]

There is no recovering a save where the terrain has already been generated and monsters/wrecks placed etc. Potentially affected saves where the terrain has not been generated, ought to have the negative effects mitigated for newly revealed tiles.