View Issue Details

IDProjectCategoryLast Update
0024673AI War 2Crash/ExceptionApr 16, 2021 10:02 am
ReporterBadgerBadger Assigned ToChris_McElligottPark  
Severityminor 
Status resolvedResolutionfixed 
Product VersionBeta 2.773 Civvies Hotfix 
Fixed in Version2.802 Multiplayer Steams Onward 
Summary0024673: MP Issues April 10
DescriptionWe could not connect at all using Steam P2P. Connecting via Steam C-O was very flaky, it would often take multiple attempts but would usually work in the end.

Other problems. Sometimes a bunch of ships on a planet will disappear for 5 seconds or so, then reappear. In general it happens on a per-\
faction basis; like all the player ships will disappear (but the player-allied scourge wil lbe visible), or all the AI ships will vanish \
on a planet and the planet will show as unowned, then it will reshow as owned by the AI.

<This error happened a ton>
4/10/2021 9:10:11 PM 2.773 CLIENT Hit exception during HandleAutobuild debugCode 5300 System.NullReferenceException: Object reference not set to an instance of an object

<This error also happened frequently>
4/10/2021 9:05:43 PM 2.773 CLIENT BLARRRGH! Only authorized to execute through frame 11401 but have already executed through frame 11402
  at System.Environment.get_StackTrace () [0x00000] in <9577ac7a62ef43179789031239ba8798>:0
  at Arcen.Universal.ArcenDebugging.ArcenDebugLog (System.String Message, Arcen.Universal.Verbosity Verbosity) [0x00000] in <390e1ff3359141fbb587d51473832675>:0
  at Arcen.AIW2.External.SimPlannerImplementation.DoActualSimStep (System.Boolean& stalledWaitingOnConnections, System.Boolean& mayUpdateVisuals, System.String& reasonForNoVisualUpdates) [0x00000] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.Core.Engine_AIW2.ProcessSimStep (System.Boolean& stalledWaitingOnConnections, System.String& reasonForNoVisualUpdates) [0x00000] in <e5f790800d364aaeb915aa8a9973be99>:0
  at Arcen.Universal.Engine_Universal.OnUpdateFromMainThread () [0x00000] in <390e1ff3359141fbb587d51473832675>:0
  at Arcen.AIW2.Core.ArcenGameControllerBase.BaseUpdate () [0x00000] in <e5f790800d364aaeb915aa8a9973be99>:0
  at ArcenGameController.Update () [0x00000] in <c72e751393a74e1dac1816656870bf25>:0


4/10/2021 9:03:25 PM 2.773 CLIENT Error in thread for execution context 'executionContext'
NullReferenceException
Object reference not set to an instance of an object
===STACK FRAMES (with file info)===
FILE METHOD IL_OFFSET NATIVE_OFFSET LINE_NUMBER COLUMN_NUMBER
        <CheckForInternalShipDeployment_ReinforcementLocations>b__0 155 442 0 0
        DoForEntities 77 204 0 0
        CheckForInternalShipDeployment_ReinforcementLocations 37 267 0 0
        DoCombatSecond 63 167 0 0
        DoCombatSecond_FromSimBGThread 58 199 0 0
        <DoWorldSecondLogic_FromSimBGThread>b__4 1 55 0 0
        DoForPlanets 59 208 0 0
        DoForPlanets 22 91 0 0
        DoWorldSecondLogic_FromSimBGThread 474 1731 0 0
        DoWorld_Second_PerSecondLogic 101 315 0 0
        DoWorldStepLogic_FromSimBGThread 116 247 0 0
        Execute 186 448 0 0
===RAW STACK TRACE===
  at Arcen.AIW2.Core.PlanetFaction+<>c__DisplayClass81_0.<CheckForInternalShipDeployment_ReinforcementLocations>b__0 (Arcen.AIW2.Core.Gam\
eEntity_Squad entity) [0x0009b] in <e5f790800d364aaeb915aa8a9973be99>:0
  at Arcen.AIW2.Core.EntityCollection.DoForEntities (Arcen.AIW2.Core.GameEntity_Squad+ProcessorDelegate Processor) [0x0004d] in <e5f79080\
0d364aaeb915aa8a9973be99>:0
  at Arcen.AIW2.Core.PlanetFaction.CheckForInternalShipDeployment_ReinforcementLocations (Arcen.AIW2.Core.ArcenSimContext Context) [0x000\
25] in <e5f790800d364aaeb915aa8a9973be99>:0
  at Arcen.AIW2.Core.PlanetFaction.DoCombatSecond (Arcen.AIW2.Core.ArcenSimContext Context) [0x0003f] in <e5f790800d364aaeb915aa8a9973be9\


4/10/2021 9:13:31 PM 2.773 CLIENT Blank FleetMembership for ship of type Warping-In Marauder Outpost! Health: 5000 Removed: False
  at Arcen.AIW2.Core.GameEntity_Squad.DoEntitySecondLogic_FromSimBGThread (Arcen.AIW2.Core.ArcenSimContext Context) [0x00000] in <e5f7908\
00d364aaeb915aa8a9973be99>:0


<This error happened for everyone>
4/10/2021 9:35:46 PM 2.773 CLIENT Exception in DoPerSimStepLogic_OnMainThreadAndPartOfSim for faction AI Praetorian Guard(Index 37)\
, so will not run any more of those until a reload of the save.
System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.SpecialFaction_PraetorianGuard.DoPerSimStepLogic_OnMainThreadAndPartOfSim (Arcen.AIW2.Core.Faction faction, Arce\
n.AIW2.Core.ArcenSimContext Context) [0x00019] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.Core.Faction.Safe_DoPerSimStepLogic_OnMainThreadAndPartOfSim (Arcen.AIW2.Core.ArcenSimContext Context) [0x00024] in <e5f7\
90800d364aaeb915aa8a9973be99>:0
4/10/2021 9:35:46 PM 2.773 CLIENT Unknown Exception in DoPerSimStepLogic_OnMainThreadAndPartOfSim for faction AI Praetorian Guard(I\
ndex 37), so will not run any more of those until a reload of the save. Check error log to see prior recorded exception.


4/10/2021 9:54:43 PM 2.773 CLIENT Exception in DoPerSecondLogic_Stage3Main_OnMainThreadAndPartOfSim for faction Marauders(Index 12)\
, so will not run any more of those until a reload of the save.
System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.SpecialFaction_HumanMarauders.SpawnRaiders (Arcen.AIW2.Core.Faction faction, Arcen.AIW2.Core.ArcenSimContext Con\
text) [0x0029e] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.External.SpecialFaction_HumanMarauders.DoPerSecondLogic_Stage3Main_OnMainThreadAndPartOfSim (Arcen.AIW2.Core.Faction fact\
ion, Arcen.AIW2.Core.ArcenSimContext Context) [0x00c70] in <2ab78e49a75a492885740c8281f88683>:0

TagsNo tags attached.

Activities

BadgerBadger

Apr 11, 2021 12:39 am

manager   ~0061080

4/10/2021 9:05:36 PM 2.773 CLIENT Tech mouseover error at debugStage 2000. Error: System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.ShipListerUtils+<>c__DisplayClass4_0.<CalculateShipsThatYouCanCapture>b__3 (Arcen.AIW2.Core.Fleet+Membership mem) [0x00078] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.Core.Fleet.DoForMemberGroups (Arcen.AIW2.Core.Fleet+Membership+ProcessorDelegate Processor) [0x00027] in <e5f790800d364aaeb915aa8a9973be99>:0
  at Arcen.AIW2.External.ShipListerUtils+<>c__DisplayClass4_0.<CalculateShipsThatYouCanCapture>b__0 (Arcen.AIW2.Core.Fleet fleet) [0x00069] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.Core.World_AIW2.DoForFleets (Arcen.AIW2.Core.FleetStatus RequiredStatus, Arcen.AIW2.Core.Fleet+ProcessorDelegate Processor) [0x00063] in <e5f790800d364aaeb915aa8a9973be99>:0
  at Arcen.AIW2.External.ShipListerUtils.CalculateShipsThatYouCanCapture (System.Predicate`1[T] filterCriteria, Arcen.Universal.ArcenSparseLookup`2[K,T] result, Arcen.Universal.ArcenSparseLookup`2[K,T] shipsToExcludeOrNull, System.Boolean AlsoSort, Arcen.AIW2.External.IShipCounterByType CounterByType) [0x000e8] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.External.ShipListerUtils.CalculateShipsThatBenefit_ThatYouCanCapture (Arcen.AIW2.Core.TechUpgrade Tech, Arcen.Universal.ArcenSparseLookup`2[K,T] result, Arcen.Universal.ArcenSparseLookup`2[K,T] shipsToExcludeOrNull, System.Boolean AlsoSort, Arcen.AIW2.External.IShipCounterByType CounterByType) [0x0000e] in <2ab78e49a75a492885740c8281f88683>:0


4/10/2021 9:06:55 PM 2.773 CLIENT WriteFleetTooltip exception at stage 200:System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.Window_InGameSidebarFleets+btnFleet.WriteFleetTooltip (Arcen.AIW2.Core.Fleet fleet, Arcen.Universal.ArcenDoubleCharacterBuffer buffer) [0x00308] in <2ab78e49a75a492885740c8281f88683>:0
  at System.Environment.get_StackTrace () [0x00000] in <9577ac7a62ef43179789031239ba8798>:0
  at Arcen.AIW2.External.Window_InGameSidebarFleets+btnFleet.WriteFleetTooltip (Arcen.AIW2.Core.Fleet fleet, Arcen.Universal.ArcenDoubleCharacterBuffer buffer) [0x00000] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.External.Window_InGameSidebarFleets+btnFleet.HandleMouseover () [0x00000] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.Universal.ArcenUI_ImageButton.OnUpdate_Subclass (System.Single DeltaTime) [0x00000] in <390e1ff3359141fbb587d51473832675>:0
  at Arcen.Universal.ArcenUI_Element.UpdateFromUI (System.Boolean ShouldDoFullUpdate, System.Single DeltaTime) [0x00000] in <390e1ff3359141fbb587d51473832675>:0


4/10/2021 10:14:48 PM 2.773 CLIENT ArcenUI_Window.OnUpdateWindow Error: System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.Hacking_GrantShipLine_DontDestroyTarget.GetCanBeHacked (Arcen.AIW2.Core.GameEntity_Squad Target, Arcen.AIW2.Core.GameEntity_Squad HackerOrNull, Arcen.AIW2.Core.Planet planet, Arcen.AIW2.Core.Faction HackerFaction, Arcen.AIW2.Core.HackingType Type, System.String RelatedStringOrNull, System.Int32 RelatedIntOrNull, System.String& RejectionReasonDescription) [0x002b2] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.External.Window_InGameSidebarHacking+customParent+<>c__DisplayClass7_0.<OnUpdateHacking>b__0 (Arcen.AIW2.Core.GameEntity_Squad entity) [0x000ca] in <2ab78e49a75a492885740c8281f88683>:0
  at Arcen.AIW2.Core.EntityCollection.DoForEntities (Arcen.AIW2.Core.EntityRollupType Rollup, Arcen.AIW2.Core.GameEntity_Squad+ProcessorDelegate Processor) [0x00068] in <e5f790800d364aaeb915aa8a9973be99>:0

BadgerBadger

Apr 11, 2021 12:40 am

manager   ~0061081

Also, partway through the game we were all granted the Achievements related to winning our game with the current settings (like "Beat this AI type", "Beat the game with these factions enabled", etc... This did not seem to correspond to any other errors

Chris_McElligottPark

Apr 14, 2021 7:01 pm

administrator   ~0061112

This may help, but is not directly related to the things you reported:

* Fixed a fairly rare bug that was causing units to not serialize properly sometimes. It seemed to be related to the incomingshots list on units, which is something that really should only be sent in fully network syncs but should not be sent to disk or as part of smaller network syncs.
** Essentially, because of the many improvements made to multiplayer efficiency, this unfortunately had created a miniature version of Russian Roulette (some sort of fantasy chain gun with 2000 slots but only one with a bullet in it), where if you got tagged with it, it would corrupt your save.
** Thankfully, autosaves are a thing, and in the example save that had this issue, there was an autosave from literally 20 seconds earlier that did not have the problem. And with extensive testing on the same save, we couldn't get it to replicate.
** That said, some of the persistent intermittent errors that folks were seeing in multiplayer were a lot less rare (because multiplayer takes a lot more tries at Russian Roulette, not because the actual odds changed), and this is probably an accidental discovery of what those were.

Chris_McElligottPark

Apr 15, 2021 7:54 pm

administrator   ~0061115

Going from top to bottom in your list:

* Fixed a variety of exceptions that could happen in HandleAutobuild() mainly in multiplayer, on the host or clients, due to cross-thread racing. Technically it could also happen in solo play, but we haven't seen any evidence of it actually doing so.

Chris_McElligottPark

Apr 15, 2021 7:57 pm

administrator   ~0061116

* The "BLARRRGH! Only authorized to execute through frame" error is now silenced. That... really wasn't actually a problem that was worth even complaining about. This was something that was showing up lately in at least a few multiplayer sessions, though something else seemed to be going on with them in general.

Chris_McElligottPark

Apr 15, 2021 9:07 pm

administrator   ~0061117

This error that you describe, by the way, I suspect is a downstream error thing:

"Other problems. Sometimes a bunch of ships on a planet will disappear for 5 seconds or so, then reappear. In general it happens on a per-\
faction basis; like all the player ships will disappear (but the player-allied scourge wil lbe visible), or all the AI ships will vanish \
on a planet and the planet will show as unowned, then it will reshow as owned by the AI."

This sounds like some sort of error during faction sync causing some data loss. Given the many other errors, I'm not too worried about this being a persistent thing. Now that the Steam Sockets code is multi-port, this should be less of an issue in general, but I'll adjust it if I need to. Mainly just reducing the obvious exceptions should solve that one, though.

Chris_McElligottPark

Apr 15, 2021 9:09 pm

administrator   ~0061118

* Fixed an issue in CheckForInternalShipDeployment_ReinforcementLocations() that could happen on MP clients, which really did not need to be running that logic anyhow.

Chris_McElligottPark

Apr 15, 2021 9:14 pm

administrator   ~0061119

Hooookay... this one should not even be possible as it already stands:

if ( ArcenNetworkAuthority.DesiredStatus != DesiredMultiplayerStatus.Client )
                        ArcenDebugging.ArcenDebugLog( "Blank FleetMembership for ship of type " + Entity.TypeData.DisplayName + "!", Verbosity.ShowAsError );

But that happened on a client for you. I think a client was somehow having a desiredstatus of singleplayer, which would have been all sorts of trouble.

THAT said, the logging code is like this:

switch ( ArcenNetworkAuthority.DesiredStatus )
                    {
                        case DesiredMultiplayerStatus.SinglePlayerOnly:
                            fullMessageMaybeStack += "\tSINGLEP";
                            break;
                        case DesiredMultiplayerStatus.Client:
                            fullMessageMaybeStack += "\tCLIENT";
                            break;
                        case DesiredMultiplayerStatus.Host:
                            fullMessageMaybeStack += "\tHOST";
                            break;
                    }

So the fact that it says client in front of it must mean that it was in client mode. It is possible that this was a delayed message, and it had shifted in and out of client mode somehow over the span of a few milliseconds, but that would be very alarming. Basically if it was in solo mode when on the BG thread, and it queued the message for a delayed write a few ms later, and then it was back in client mode by then? That makes no sense at all, but if we see more signs of something like that, then I'll have to take more of a look.

For now I'm going to chalk it up to "I guess things get strange after errors." But I will keep an eye out.

Chris_McElligottPark

Apr 15, 2021 9:15 pm

administrator   ~0061120

You fixed this one:

<This error happened for everyone>
4/10/2021 9:35:46 PM 2.773 CLIENT Exception in DoPerSimStepLogic_OnMainThreadAndPartOfSim for faction AI Praetorian Guard(Index 37)\
, so will not run any more of those until a reload of the save.
System.NullReferenceException: Object reference not set to an instance of an object
  at Arcen.AIW2.External.SpecialFaction_PraetorianGuard.DoPerSimStepLogic_OnMainThreadAndPartOfSim (Arcen.AIW2.Core.Faction faction, Arce\

Chris_McElligottPark

Apr 15, 2021 9:16 pm

administrator   ~0061121

* Fixed an exception that could happen in SpawnRaiders() on Marauders in multiplayer clients. Again not something they needed to ever do, it's left to just the host now.

Chris_McElligottPark

Apr 15, 2021 9:24 pm

administrator   ~0061122

* Fixed a number of possible cross-threading issues in CalculateShipsThatYouCanCapture(), which were vastly more likely to happen on MP clients.

Chris_McElligottPark

Apr 15, 2021 9:31 pm

administrator   ~0061123

* Fixed several more cross-threading bugs that could happen in WriteFleetTooltip(), again way more commonly on MP clients than anywhere else.

Chris_McElligottPark

Apr 15, 2021 9:40 pm

administrator   ~0061124

Okay, that's the last of actionable things:

* Fixed another cross-threading bug that could happen in Hacking_GrantShipLine_DontDestroyTarget.GetCanBeHacked(), again mainly on MP clients. The obvious potential errors have been fixed, but then also it will now show in the ui an "error at debug stage x" message if it still runs into one. It will also silently log the error for us to look at more later, and to be able to further fix if need be.

Thanks so much for all of these, and the way that you pull them out! The whole achievements thing is very strange, but again I have to guess that it's probably a downstream error. Hopefully as these primary errors are put to pasture, the downstream ones will also just quiet down.

BadgerBadger

Apr 15, 2021 11:14 pm

manager   ~0061125

My goal when opening a bug is to make it as easy as possible for you to fix!

Chris_McElligottPark

Apr 16, 2021 10:02 am

administrator   ~0061131

That is greatly appreciated! :)

Issue History

Date Modified Username Field Change
Apr 11, 2021 12:33 am BadgerBadger New Issue
Apr 11, 2021 12:33 am BadgerBadger Status new => assigned
Apr 11, 2021 12:33 am BadgerBadger Assigned To => Chris_McElligottPark
Apr 11, 2021 12:39 am BadgerBadger Note Added: 0061080
Apr 11, 2021 12:40 am BadgerBadger Note Added: 0061081
Apr 14, 2021 7:01 pm Chris_McElligottPark Note Added: 0061112
Apr 15, 2021 7:54 pm Chris_McElligottPark Note Added: 0061115
Apr 15, 2021 7:57 pm Chris_McElligottPark Note Added: 0061116
Apr 15, 2021 9:07 pm Chris_McElligottPark Note Added: 0061117
Apr 15, 2021 9:09 pm Chris_McElligottPark Note Added: 0061118
Apr 15, 2021 9:14 pm Chris_McElligottPark Note Added: 0061119
Apr 15, 2021 9:15 pm Chris_McElligottPark Note Added: 0061120
Apr 15, 2021 9:16 pm Chris_McElligottPark Note Added: 0061121
Apr 15, 2021 9:24 pm Chris_McElligottPark Note Added: 0061122
Apr 15, 2021 9:31 pm Chris_McElligottPark Note Added: 0061123
Apr 15, 2021 9:40 pm Chris_McElligottPark Status assigned => resolved
Apr 15, 2021 9:40 pm Chris_McElligottPark Resolution open => fixed
Apr 15, 2021 9:40 pm Chris_McElligottPark Fixed in Version => 2.802 Multiplayer Steams Onward
Apr 15, 2021 9:40 pm Chris_McElligottPark Note Added: 0061124
Apr 15, 2021 11:14 pm BadgerBadger Note Added: 0061125
Apr 16, 2021 10:02 am Chris_McElligottPark Note Added: 0061131