View Issue Details

IDProjectCategoryLast Update
0023675AI War 2[All Projects] Gameplay IssueSep 25, 2020 9:53 pm
Reporterx4000BughunterAssigned Tox4000Bughunter 
Severityminor 
Status resolvedResolutionfixed 
Product Version2.500 Multiplayer Alpha Begins Now! 
Fixed in Version2.600 Multiplayer Alpha Full Announcement 
Summary0023675: Multiplayer: General work on some stutter with "waiting for players" that flashes by too frequently on many machine combos.
Description* Find a way to make the performance of the clients versus the host a little more decoupled so that we have more of a window where the "waiting for players" thing is not popping up.
** That will take some experimentation, but we have some ideas on how to handle this in a couple of different ways.
** Right now there's a fair bit of stutter that you can experience if your machines are not all well above the network minimum, and your CPUs are not all really in prime shape. Even with those conditions, you'll likely see that message pop up every few seconds. This is a case of us needing to optimize how different-powered machines compensate for one another, AND us needing to give a few more extra ms of leeway for certain connections in order for them to not wind up running out of authorized frames and stall.
** In short, this is annoying, and probably feels like the most pressing problem in terms of impairing enjoyment, but it's something that has several layers of solutions for different machine pairs or groupings, and we'll be able to take care of those increasingly over the next week or so.
TagsNo tags attached.

Activities

x4000Bughunter

Sep 14, 2020 8:45 pm

administrator   ~0058515

Update as of version 2.505: there are far fewer stutters of this type, and it remained pretty consistent for me over 18 minutes of a multiplayer game (realtime, 1x speed). A lot of the stutters previously were caused by bad data doing insane things.

However, on weak network connections, the volume of data is enough that it's worth keeping the sync data a bit lower, which will further reduce network-saturation-based stutter. There are several open issues on those, and none are too complicated. Even as it stands now, though, the overall network load from this game (as a client, or as a host in a two player game) is well under that of just watching an SD video for the same amount of time.

In 18 minutes, the host had transferred about 20MB of data to the client, and the client had transferred something like half a MB back. I can probably cut the numbers from the host in half, if not even better than that, without having to do anything crazy. Which will be great for even larger games or even worse connections, but it's still never really going to be the main source of lag.

The main source of lag is now the timing of sim frames, aka the client telling the host it is done with its work, and the host telling the client it can do more work (as well as giving it commands). There are several options for ways of running this a bit smarter, and I'll admit that right now I've basically put things into the most-likely-to-lag but least-command-latency mode at the moment. Letting players choose their latency mode is something that has some interest (for truly abymsal connections across continents to run smoothly), but in an ideal world most connections should never have to faff around with that.

Weaker CPUs can definitely get off with the timing a bit, and drag things down in really large games, which was the primary source of lag in AI War Classic. But honestly, AI War 2 really is just in a whole other league in terms of handling that. Four years of optimizing the single player code, and the fact that the "long range planning code" (periodic or continuous) doesn't even run on the client, really helps to make this better. That means that all of the stuff with the autotargeting and decollision and so on is only run on the host, as well as all the higher intelligence of the AI.

Still, that came at a cost, which is timing synchronization. In AI War Classic, we could afford to have a 400ms latency on command input, and it wasn't really something people noticed after playing for long at all. We kept that latency identical in single player and in multiplayer. In AI War 2, we currently have a 100ms latency on the host, and on the client that turns into 200ms. The difference is noticeable and a bit painful, and not something that we can really solve because of... well, physics and the speed of light, etc.

The main things that guard against periodic drops and waiting are having a larger input latency period so that we can do things like say "okay, give me the commands for the next cycle" when there's still one cycle left to go. Even with the existing latency settings, I might be able to make that happen, by making the client give an ack for the cycle it just completed at the time it STARTS that cycle. That would give us a 100ms window for the client to tell the host and the host to respond and auth the next frame. That's a freaking eternity, and even with packet loss situations it should be fine. For truly bad connections, I can make it so that people can choose to have more frames authed at once, widening that gap to 200ms or 300ms, getting us back to 400ms or 600ms of input latency on the client, and 200ms or 300ms of input latency on the host. That would be preferable to choppy network performance if the connection is that distant.

So the basics solution to this might be really simple, we'll see, and then some longer-term solutions might involve a mix of options for performance tuning, and further refinement of things to go later.

---

Another possibility is that I may wish to time-slice the framerate into alternating cycles that are run every 50ms (so 20fps) rather than every 100ms (10fps). This was the framerate that AI War Classic was locked to, and it was extremely limiting. The actual framerate of AI War 2 can be anything, higher or lower, and the simulation framerate tries to be this either way. But the main concern here is command latency, and commands basically have to be inserted into the queue at times when nothing else is happening. If I were to time-slice so that randomly planets and factions did their business every other half-frame, and there were two half-frames per main frame, with the ability to sync in gamecommands between each one, then that would probably improve responsiveness.

Very worst case with that approach, the tier 1 planets should also run at least their player ship logic on EVERY half frame, and run them at half speed, so that the commands take effect faster. Now we're getting kind of complicated. This might be something for post-launch, or might not be needed at all. The performance is already better than AI War Classic in terms of command lag, and right now the worst case of what I'm considering is making it equal to what the first game had as the default.

I guess the command lag is also noticeable in things like autotargeting and decollision, to be frank. That might be able to feel a bit better and be a slight bit more responsive even in singleplayer if I could time-slice into half frames. I dunno. For now I want to get the choppiness out of there, and have the option for people to turn up the command latency as a way to get better smoothness if their connection is bad enough that it's still choppy. Then I should probably focus on every other bug people are finding, and all the features they want, before I dive into this too deeply. I've been mulling this for a few years, sigh, and thankfully the situation is brighter than I had worried it was going to be -- those serialization efficiency improvements I made continue to pay dividends big time. Earlier this year it felt like a wasted month in a lot of respects, and I could tell folks were getting frustrated with me, but this is the sort of thing I was thinking about at the time, and thankfully I'm not in the position of having to take a month off to work on that NOW.

x4000Bughunter

Sep 25, 2020 9:53 pm

administrator   ~0058683

There will be more work needed to make the clients as smooth as possible, but there have been ENORMOUS strides in this in this version, to the point where the popup is not there and the stutter is minimized, as is command lag. There is definitely some visual fiddly-ness on the client, but it's in a different league from what it was.

Issue History

Date Modified Username Field Change
Sep 9, 2020 6:08 pm x4000Bughunter New Issue
Sep 9, 2020 6:08 pm x4000Bughunter Status new => assigned
Sep 9, 2020 6:08 pm x4000Bughunter Assigned To => x4000Bughunter
Sep 14, 2020 8:45 pm x4000Bughunter Note Added: 0058515
Sep 25, 2020 9:53 pm x4000Bughunter Status assigned => resolved
Sep 25, 2020 9:53 pm x4000Bughunter Resolution open => fixed
Sep 25, 2020 9:53 pm x4000Bughunter Fixed in Version => 2.600 Multiplayer Alpha Full Announcement
Sep 25, 2020 9:53 pm x4000Bughunter Note Added: 0058683