View Issue Details

IDProjectCategoryLast Update
0021288AI War 2[All Projects] Gameplay IssueAug 3, 2019 7:10 pm
ReporterKesselethAssigned Tox4000Bughunter 
Severitymajor 
Status resolvedResolutionfixed 
Product VersionBETA 0.866 Hotfix 
Fixed in Version0.878 Fleet Customization 
Summary0021288: Serious Slowdown (Chris notes: proto-simulation needed for background third-party fights.)
DescriptionI'm running a several year old computer but it has an i7 processor, an nVidia Geforce 970, and otherwise is a pretty competent machine. However, on large maps I experience enormous slowdown, significant enough that Badger finds it notable. I have a posted save which, on my own machine, is running at 65% while simply waiting around on an allied planet.
TagsNo tags attached.

Relationships

related to 0021429 resolvedx4000Bughunter Performance Boost: Sim Step Bundling For Background Battles 

Activities

Kesseleth

Jun 8, 2019 8:13 pm

reporter  

significant slowdown save.save (2,737,863 bytes)

BadgerBadger

Jun 8, 2019 8:32 pm

manager   ~0051903

Last edited: Jun 8, 2019 9:23 pm

View 3 revisions

Hey Chris, does the unit test program still work for you? I get weird errors about mono being out of sync with UnityEngine.CoreModule.dll

This is either a problem with my environment or fallout from the unity upgrade or both. Probably my environment. In any case, one of us needs to see what's going on here.

BadgerBadger

Jul 28, 2019 4:44 pm

manager   ~0052342

I disabled most of the minor faction code that was running on the main Sim thread with no major change in performance. The slowness seems to be just because it's a large galaxy with combat happening all over the place, from a cursory examination of the performance tracing.

x4000Bughunter

Jul 30, 2019 12:20 pm

administrator   ~0052344

I haven't tested the unit testing code in a super duper long time, and we've had several unity upgrades since then, so there's a fair bet it's busted.

In terms of the performance if there's a ton of combat all over the place... that's definitely something that will eat the CPU, for sure. Factions that are unseen and fighting one another in large-scale battles are a serious system stress that the game wasn't really designed to handle. It's certainly generally been proven to be ABLE to handle that, but there was actually a good reason that in both this game and the original Keith and I made things like the Dyson Sphere have just a few units that were super powerful, rather than a ton of regular-sized units. The sequel is vastly better at handling loads of combat happening at once, mainly due to the multithreading, but it still has to obey the laws of physics when it comes to processor speeds.

The way that I'd been thinking about solving that if it ever became a true issue, which so far it had not in a generalized sense, was to start shortcutting things when players can't see. The trick with simulations like this is that if the player can't see, then the battle doesn't really need to be faithful to what would happen if the player is there watching. As long as the results seem reasonable, whatever that means, it doesn't even need to use the regular battle code, let alone targeting code.

Example: rather than actually firing shots and simulating all the stuff if it's a planet where the players can't currently see, it can just do some simplified range-ignoring logic with ships attritioning every second or so based on the relative strengths at the planets. Or every sim step, whatever. The downside is that this makes for an entirely second combat model and unseen battles might have strange results (like if it doesn't know to not hurt an invincible ship), but the positive side is that we get to proto-simulate absolutely silly numbers of ships in combat if we want to, as far as the background goes.

The other downside is that this can only happen when players aren't looking, or things get very strange to see indeed. And so either the game needs to react to what planets the players are viewing (that's possible), or it needs to be only on planets that have not yet been scouted yet (that's simpler but would then not kick in nearly as much). It also can ONLY happen between third parties, or else players will notice that they get a different result if they are watching their personal ships that are fighting some enemy than when they are not watching. Aka it can't use the proto-simulation, ever, when a player is involved; it's just for the background stuff.

And this needs to be in there at the game-sim level, not at a specific faction level. People will implement all sorts of factions that have all sorts of crazy abilities and numbers of ships, and expecting them to understand the impact on performance is definitely not a fair thing, as well as the fact that it will cause absolutely huge amounts of recoding of the same thing over and over again.

In the first game, we did tons of things to avoid things happening out of sight of the player, although we never took it quite this far. But in The Last Federation we did a lot of things to simplify simulations as needed when they were on the background. When you get large enough, even an O(n) simulation is too costly since people like doing crazy things in terms of having so many battles going on.

Obviously we also want the main simulation to be as efficient as possible as well, but we may be hitting the point where there's simply a limit being hit when we're talking about galaxy-wide background combat.

x4000Bughunter

Jul 30, 2019 5:56 pm

administrator   ~0052357

Thought about it while driving today. Will be doing this: 0021429.

Way quicker to implement, and should be devastatingly effective.

Ribald

Aug 2, 2019 4:14 pm

reporter   ~0052385

Not sure if this should be a separate issue or not, but the extreme slow down I get is caused by a massive build up of ships at warden outposts. As soon as I enter a system with a warden outpost millions of ships start to spawn.

This can be observed by moving a fleet from the Columbus system (top left of galactic map) to the Adrian system (right of Columbus) in the attached save file or just entering the system and hovering over the warden outpost to see the extreme numbers.

E2.save (826,305 bytes)

x4000Bughunter

Aug 3, 2019 7:10 pm

administrator   ~0052395

Thank you!

* The game now tries to deploy AI reinforcements in stacks (of up to 100 per stack), rather than as individual units that might then have to be stacked after that.
** This keeps things like warden ninja hideouts from absolutely bogging down the simulation.

* If there are more than 10k units in a single reinforcement point somehow, then it will show an error message and reduce the number of units of that type in that point to be "merely" 10k.
** There were some cases where there were several million units in one stack in the warden ninja guard post, which was definitely an error. The error popups will probably be annoying, but we want to know and fix these things if they happen.
** The gap of 10k units in there keeps the error message from being absolutely constant in these cases, though.

* The ship counts in the escape menu now include ships that are contained in transports or in guard posts and other reinforcement points.
** The difference in the counts is pretty telling, often tens of thousands or even hundreds of thousands more ships.
** Also in the escape menu, it now shows the number of contained ships, and the number of containers, on the line right below stacks.

* Fixed a pretty darn severe bug where the ships in the warden fleet (or any other AI guards, really) could multiply like crazy if it tried to put a stack of units into a guard post.
** If there was a stack of 10 units, it would put 10 into the guard post but only destroy 1 of them. Then the next frame it would put 9 in and destroy 1, etc.
** Most of the time this was not severe enough to notice, but if you were aggroing an AI planet really heavily and it was having ships coming in and out for various reasons, then it could easily get into situations of adding more than 10k ships in under 5 seconds.
** This was probably slowing down the simulations of all sorts of savegames, but mainly those in civil war mode more than anything else. The stuff that reduces the reinforcement counts will help some, but older savegames are still going to be bloated most likely if they were problematic before.
** This doesn't affect most savegames, but for those that it did affect it's going to be a good idea to just try something less off in its data. It's unclear how much of a performance problem we'll have with civil war mode on in a fresh savegame with this off.

Issue History

Date Modified Username Field Change
Jun 8, 2019 8:13 pm Kesseleth New Issue
Jun 8, 2019 8:13 pm Kesseleth File Added: significant slowdown save.save
Jun 8, 2019 8:32 pm BadgerBadger Note Added: 0051903
Jun 8, 2019 9:22 pm BadgerBadger Note Edited: 0051903 View Revisions
Jun 8, 2019 9:23 pm BadgerBadger Note Edited: 0051903 View Revisions
Jul 28, 2019 4:44 pm BadgerBadger Note Added: 0052342
Jul 30, 2019 12:20 pm x4000Bughunter Note Added: 0052344
Jul 30, 2019 12:20 pm x4000Bughunter Summary Serious Slowdown => Serious Slowdown (Chris notes: proto-simulation needed for background third-party fights.)
Jul 30, 2019 5:55 pm x4000Bughunter Assigned To => x4000Bughunter
Jul 30, 2019 5:55 pm x4000Bughunter Status new => assigned
Jul 30, 2019 5:55 pm x4000Bughunter Relationship added related to 0021429
Jul 30, 2019 5:56 pm x4000Bughunter Note Added: 0052357
Aug 2, 2019 4:14 pm Ribald File Added: E2.save
Aug 2, 2019 4:14 pm Ribald Note Added: 0052385
Aug 3, 2019 7:10 pm x4000Bughunter Status assigned => resolved
Aug 3, 2019 7:10 pm x4000Bughunter Resolution open => fixed
Aug 3, 2019 7:10 pm x4000Bughunter Fixed in Version => 0.878 Fleet Customization
Aug 3, 2019 7:10 pm x4000Bughunter Note Added: 0052395