View Issue Details

IDProjectCategoryLast Update
0023809AI War 2Bug - OtherOct 7, 2020 4:54 pm
ReporterOzoneGrif Assigned ToChris_McElligottPark  
Severitymajor 
Status resolvedResolutionreopened 
Product Version2.601 Savegame Hotfix 
Fixed in Version2.608 MP Deduplication And Bugfixes 
Summary0023809: Multiplayer Feedback: ships are multiplying infinitely on the client player's side
DescriptionShips are multiplying infinitely on the other player's side, to the thousands, until it crashes

We barely started the game, see the picture for what the client player sees. Note that host shows correct numbers.
The game froze after I unloaded my ships (barely 150 ships). I guess the network tried to send 10k+ ships or something and it froze

We did another try later and the game remained in sync for 10 minutes, then we started to see the same issue with fleet numbers multiplying, strength showing crazy values, power is way above what the player produces. His side shows a brownout, no shields, while on the host everything looks fine. Engineers on his side were moving back and forth while on the host they were helping the production factory.
TagsNo tags attached.

Activities

OzoneGrif

Sep 26, 2020 7:21 pm

reporter  

thousands.png (147,402 bytes)   
thousands.png (147,402 bytes)   

Chris_McElligottPark

Sep 28, 2020 4:46 pm

administrator   ~0058730

Thanks!

* Fixed a bug in multiplayer where transported ships were being doubled on the client every second or so.
** This was causing all sorts of issues ranging from brownouts flipping on and off, to a memory leak and performance degradation.
** If nobody was transporting ships, or transporting ships for very long, then this problem would solve itself. This is how Suzara and Ipsum were able to win the first complete game against the AI in multiplayer for this game, where others were hitting a wall a bit sooner.

OzoneGrif

Oct 3, 2020 8:52 am

reporter   ~0058944

The multiplication issue is still present in the latest version.
It began with drones multiplying, then followed by ships.
The game started to hiccup badly after that, and a lot of things stopped working correctly.
I join two screenshots of the client-side.

OzoneGrif

Oct 3, 2020 8:53 am

reporter  

Drones.png (100,120 bytes)   
Drones.png (100,120 bytes)   
Ships.png (126,877 bytes)   
Ships.png (126,877 bytes)   

OzoneGrif

Oct 3, 2020 8:54 am

reporter   ~0058945

This seem to occur when manually loading/unloading ships.

OzoneGrif

Oct 3, 2020 9:00 am

reporter   ~0058946

Note, the game DEBUG file is in this other ticket (same game):
https://bugtracker.arcengames.com/view.php?id=23810

Chris_McElligottPark

Oct 3, 2020 10:49 am

administrator   ~0058947

Can I have a copy of the savegame where you're seeing this, if you have one? It sounds like it's a pretty constant issue for you, but others are not seeing it, so I'm thinking it's an issue with one particular feature (or two or three).

So far you've identified drones and loading and unloading transports as likely culprits. I'd like to be able to observe that and see the resolution happen if possible.

Someone else had reported the AI Eyes, and that was definitely a problem, but is probably now fixed. There may be a problem with things being rebuilt from remains getting duplicated on the client, but I'm not certain.

The more I can have savegames and steps to reproduce, the faster I'll be at knocking these out. :) But in any respect, I really really appreciate the reports and logs in the first place!

OzoneGrif

Oct 3, 2020 11:41 am

reporter   ~0058949

This is the save I made when the issue began to appear on the client.
This save was made on the host.

I don't really have steps to reproduce the bug; it's random. We can play for a few minutes, then it will start to multiply.
It's apparently linked to loading and unloading troops but we can't be sure of that.
MP.zip (78,420 bytes)

Chris_McElligottPark

Oct 7, 2020 3:08 pm

administrator   ~0059071

I can duplicate this issue just by unloading the client with the Harus fleet. Thanks for the easy reproduction case!

There are a couple of strange things going on here. First of all, creating 2x as many ships as it should is a bug I need to fix.

But more pressing (to me) is the fact that these ships are somehow allowed to continue to exist and only VERY slowly get whittled down. They are being removed, one by one, as "Ship Dels From Miss," which is very very strange. It should blaze through and cull them all immediately, and why it is not doing that is not clear to me at all.

So I'm going to fix that before I fix the extra ships thing, because something will always be able to create extra ships somehow (mods or who knows what with a bug), and the game should recover from this faster than it is.

And... actually, I think these aren't being found at all, somehow! These are being deleted as being over cap on the client, but I think that these are actually invisible ships to the sync check process. That's a whole concerning thing in its own way, and does explain why it's so slow to fix things. That "delete while over cap" is intentionally slow, since it's usually expecting that you had something happen in normal gameplay like losing a ship line doubler or something.

Another fun fact is that putting these back into the transport is wrong for a moment, but then the next time the fleet sync pops by it corrects that bad data. Then when you unload the transport, it's busted again from making duplicates that are invisible to the sync process.

These also don't decollide, so they stick around right under the transport. The decollision only runs on the host, IIRC, so since it doesn't know about these extra ships that makes sense.

Chris_McElligottPark

Oct 7, 2020 4:54 pm

administrator   ~0059075

Nailed it! Thanks. :)

* Fixed a severe multiplayer bug that was leading to a wide range of issues, mostly involving having a lot of extra ships that should not exist on the client.
** These involved unloading ships from transports, ships coming out of guard posts, ships spawning from waves, ships spawning in hacking responses, turrets being placed in bulk batches by the player, and assumedly other cases as well.
** Essentially in all of these cases, the client and the host wind up with different values for their PKIDs of those units, and thus on the client the units get deleted and the host sends over the proper copies that they have created.
** The problem was, the client was doing the deletions in INCREDIBLE slow motion. It would remove those ships at a rate of about 1 per 3 seconds, rather than "all of the ships that are erroneous go away every 3 seconds."
** The reason for this is how we are ruthlessly scrubbing out the existence of these ships; it was breaking out ability to loop over the full list of ships properly. So we instead gather up the list of ships first, then ruthlessly delete them, and all is well.
** There are still going to be spurious brownout notices that pop up at the moment, and extra ships that appear, but they should only last about 3 seconds and so should not cause any major lag or other problems, unlike what was happening before.
** Our PKID allocation chain is going to be revised in order to completely bypass this sort of problem in general, but we first wanted to fix this issue since whenever there IS a problem, the game is supposed to be able to repair itself, and so these problems sticking around longer and being easy to reproduce (in the end) were a golden opportunity for us to make sure all that works. Now it does.
*** When the PKID allocation chain is reworked, there won't be those spikes of deletions and recreations, or those spurious brownout notices. Until then, that will be an ongoing minor annoyance.

Issue History

Date Modified Username Field Change
Sep 26, 2020 7:21 pm OzoneGrif New Issue
Sep 26, 2020 7:21 pm OzoneGrif File Added: thousands.png
Sep 28, 2020 4:46 pm Chris_McElligottPark Assigned To => Chris_McElligottPark
Sep 28, 2020 4:46 pm Chris_McElligottPark Status new => resolved
Sep 28, 2020 4:46 pm Chris_McElligottPark Resolution open => fixed
Sep 28, 2020 4:46 pm Chris_McElligottPark Fixed in Version => 2.602 Transports Fixed In Multiplayer
Sep 28, 2020 4:46 pm Chris_McElligottPark Note Added: 0058730
Oct 3, 2020 8:52 am OzoneGrif Status resolved => feedback
Oct 3, 2020 8:52 am OzoneGrif Resolution fixed => reopened
Oct 3, 2020 8:52 am OzoneGrif Note Added: 0058944
Oct 3, 2020 8:53 am OzoneGrif File Added: Drones.png
Oct 3, 2020 8:53 am OzoneGrif File Added: Ships.png
Oct 3, 2020 8:54 am OzoneGrif Note Added: 0058945
Oct 3, 2020 8:54 am OzoneGrif Status feedback => assigned
Oct 3, 2020 9:00 am OzoneGrif Note Added: 0058946
Oct 3, 2020 10:49 am Chris_McElligottPark Note Added: 0058947
Oct 3, 2020 11:41 am OzoneGrif File Added: MP.zip
Oct 3, 2020 11:41 am OzoneGrif Note Added: 0058949
Oct 7, 2020 3:08 pm Chris_McElligottPark Note Added: 0059071
Oct 7, 2020 4:54 pm Chris_McElligottPark Status assigned => resolved
Oct 7, 2020 4:54 pm Chris_McElligottPark Fixed in Version 2.602 Transports Fixed In Multiplayer => 2.608 MP Deduplication And Bugfixes
Oct 7, 2020 4:54 pm Chris_McElligottPark Note Added: 0059075