View Issue Details
| ID | Project | Category | Date Submitted | Last Update | |
|---|---|---|---|---|---|
| 0004063 | Valley 1 | Crash/Exception | Sep 27, 2011 4:06 am | Sep 27, 2011 4:33 pm | |
| Reporter | c4sc4 | Assigned To | keith.lamothe | ||
| Status | resolved | Resolution | fixed | ||
| Product Version | 0.501 | ||||
| Fixed in Version | 0.502 | ||||
| Summary | 0004063: Fatal Error on the world map | ||||
| Description | I was on the world map and I was uncovering some new tiles, the specific tiles that were just uncovered were some lava flats, and then a fatal error occurred. Exiting and restarting the game fixed it. I'm attaching the UnhandledErrors.txt file. Let me know if you want my whole world file or not, it did happen on my massive world. | ||||
| Tags | No tags attached. | ||||
| Internal Weight | Fix ASAP | ||||
|
|
UnhandledErrors.txt (5,721 bytes)
9/27/2011 12:59:27 AM (0.501) ----------------------------------- UnhandledErrors----------------------------------- 9/27/2011 12:59:27 AM (0.501) -----------------------------------Error-----------------------------------Log String: FixedUpdate: System.OutOfMemoryException: Out of memory at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at ArcenSerializationBuffer.ResizeTo (Int32 NewSize) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.ResizeForAdditionalLength (Int32 NeedsToAccomodateThisMuchMore) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.Add (Int32 Item) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.AddDelimiterAnd (Int32 Item) [0x00000] in <filename unknown>:0 at CityStructure.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at CityManagementSubData.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at MacrogameLocation.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at Region.SerializeTo (.ArcenSerializationBuffer Buffer, Boolean IsForSendingOverNetwork) [0x00000] in <filename unknown>:0 at World.SaveRegionsToDisk () [0x00000] in <filename unknown>:0 at Game.ProcessSimStep () [0x00000] in <filename unknown>:0 at Game.RunNextCycle (Boolean DoRendering) [0x00000] in <filename unknown>:0 at Game.RunFrame () [0x00000] in <filename unknown>:0 at MainCameraLogic.Update () [0x00000] in <filename unknown>:0 Stack Trace: at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at ArcenSerializationBuffer.ResizeTo (Int32 NewSize) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.ResizeForAdditionalLength (Int32 NeedsToAccomodateThisMuchMore) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.Add (Int32 Item) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.AddDelimiterAnd (Int32 Item) [0x00000] in <filename unknown>:0 at CityStructure.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at CityManagementSubData.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at MacrogameLocation.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at Region.SerializeTo (.ArcenSerializationBuffer Buffer, Boolean IsForSendingOverNetwork) [0x00000] in <filename unknown>:0 at World.SaveRegionsToDisk () [0x00000] in <filename unknown>:0 at Game.ProcessSimStep () [0x00000] in <filename unknown>:0 at Game.RunNextCycle (Boolean DoRendering) [0x00000] in <filename unknown>:0 at Game.RunFrame () [0x00000] in <filename unknown>:0 at MainCameraLogic.Update () [0x00000] in <filename unknown>:0 9/27/2011 12:59:29 AM (0.501) ----------------------------------- UnhandledErrors----------------------------------- 9/27/2011 12:59:29 AM (0.501) -----------------------------------Error-----------------------------------Log String: FixedUpdate: System.OutOfMemoryException: Out of memory at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at ArcenSerializationBuffer.ResizeTo (Int32 NewSize) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.ResizeForAdditionalLength (Int32 NeedsToAccomodateThisMuchMore) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.Add (Int32 Item) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.AddDelimiterAnd (Int32 Item) [0x00000] in <filename unknown>:0 at CityStructure.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at CityManagementSubData.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at MacrogameLocation.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at Region.SerializeTo (.ArcenSerializationBuffer Buffer, Boolean IsForSendingOverNetwork) [0x00000] in <filename unknown>:0 at World.SaveRegionsToDisk () [0x00000] in <filename unknown>:0 at Game.ProcessSimStep () [0x00000] in <filename unknown>:0 at Game.RunNextCycle (Boolean DoRendering) [0x00000] in <filename unknown>:0 at Game.RunFrame () [0x00000] in <filename unknown>:0 at MainCameraLogic.Update () [0x00000] in <filename unknown>:0 Stack Trace: at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at ArcenSerializationBuffer.ResizeTo (Int32 NewSize) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.ResizeForAdditionalLength (Int32 NeedsToAccomodateThisMuchMore) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.Add (Int32 Item) [0x00000] in <filename unknown>:0 at ArcenSerializationBuffer.AddDelimiterAnd (Int32 Item) [0x00000] in <filename unknown>:0 at CityStructure.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at CityManagementSubData.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at MacrogameLocation.SerializeTo (.ArcenSerializationBuffer Buffer) [0x00000] in <filename unknown>:0 at Region.SerializeTo (.ArcenSerializationBuffer Buffer, Boolean IsForSendingOverNetwork) [0x00000] in <filename unknown>:0 at World.SaveRegionsToDisk () [0x00000] in <filename unknown>:0 at Game.ProcessSimStep () [0x00000] in <filename unknown>:0 at Game.RunNextCycle (Boolean DoRendering) [0x00000] in <filename unknown>:0 at Game.RunFrame () [0x00000] in <filename unknown>:0 at MainCameraLogic.Update () [0x00000] in <filename unknown>:0 |
|
|
Yea, I'll need the entire world file somehow, that's an out-of-memory error. |
|
|
So, what happened, I think, is this: it went to write the world stuff to disk, but the heap was too high on transient allocations, and so of course the handy mono GC suicided instead of doing a collection. My plan: have the game look at current heap usage, and if it's more than... let's say 400MB... then it will do a collection before trying to write the world file or the region file. I guess if it's more than 600MB, it would do the same before writing a chunk file or similar. Keith, actually, I have enough on my plate for today; would you be able to add that in? |
|
|
I can do that, but I also need the world file to see what kind of permanent allocation is happening and if the way I've got it buffering the changes is doubling or tripling the "max height" of the heap footprint of world-saving, etc. |
|
|
Yep, that all makes sense. |
|
|
Keith, I can't upload my world to mantis, it is far to big, I'm going to email it to you. |
|
|
How big is it up to, now? |
|
|
The zipped folder is 24+MB. |
|
|
Wowee, goodness. What about world.dat and regions.dat unzipped? |
|
|
10894 regions World.dat is 630KB regions.dat is 2,764KB |
|
|
Cool, that's not really that bad, then. I think world.dat is compressed, but can't recall. But anyway, my GC suggestion there should be fine for handling a memory spike of writing these files. It's just the mono suiciding thing again, not really a full explosion of memory. |
|
|
I'm checking out the permanent allocation right now and yea, it's not looking "terminal" ;) Right after loading the game in the standalone: F3 reports the heap is about 90MB. Process Explorer reports the AVWW.exe process is holding onto 854MB of ram. I assume this is mostly textures in unmanaged memory. Watching F3 (which allocates a fair bit of transient by itself due to some of the debug stuff using string.concat for big strings), the heap increases by about 1 MB per 2.5 seconds. Waiting a while, the heap made it up to about 190MB before the GC kicked it in and brought it down to a bit over 45MB, and then it resumed counting up. So yea, it's holding the entire macrogame for 10,000 regions and the general game overhead in 45MB of managed heap... shouldn't be a problem ;) Now to plug in those GC.Collect calls... |
|
|
Yeah, no problems whatsoever with that. That's miniscule compared to AI War. Load a decent-sized game there and you're looking at 250MB of heap. Just loading the AI War program at all gets you to about 115MB of heap. That will creep up with AVWW as well as we add more ships and such, but still probably not past 120MB or so. Anyway, yeah, 45MB is trivial. And his world is about 4x oversized for his civ level due to it being on the really older seeding. So it wouldn't even hit that 10k regions mark until MUCH further into the game for any other players. And if we get folks going into the really insanely large worlds (say, in multiplayer servers), then we can always start segmenting it; either behind the scenes, or into literal multiple planes, or whatever. A problem for much later, if it comes up, as it should really be a while before anyone hits that (knocks on wood). |
|
|
Odd, I really can't get it to get higher than 200MB on the managed heap no matter what I do in terms of entering, exiting, walking around, revealing new world tiles, etc. Going to the escape menu to trigger the world-save tends to cause a GC already, I guess I could have it always do so instead of only doing it 80% of the time. |
|
|
I think that he may have been playing for a very long time in one sitting. And I don't think that the RegionInterior data is dumped from memory when it's dumped to disk, at present. So that might have caused it to creep up over time. I definitely don't want it to always GC collect, I think that the threshold check is still desired even if you can't hit that threshold in your testing. |
|
|
Yeah, I was playing for a fair bit of time before the error occurred. |
|
|
Ok, I put in some gc-if-over-threshold checks for 0.502. Perhaps I also need to go ahead and have it start dropping interior data to disk if it isn't used in, say, two consecutive minutes (only checking every 60 seconds since I guess it'd have to check every region, though it could probably keep a list of regions that have interior metadata loaded to cut down on that. Chris, if the thresholds seem to low to you, feel free to change them. The consts start with "MUST_GARBAGE_COLLECT_". |
|
|
I think that dropping the interiors after 2 minutes would be a great addition. And especially future-proofing for multiplayer. |
|
|
And those thresholds look great to me. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| Sep 27, 2011 4:06 am | c4sc4 | New Issue | |
| Sep 27, 2011 4:06 am | c4sc4 | File Added: UnhandledErrors.txt | |
| Sep 27, 2011 6:24 am | tigersfan | Internal Weight | => Fix ASAP |
| Sep 27, 2011 6:24 am | tigersfan | Assigned To | => keith.lamothe |
| Sep 27, 2011 6:24 am | tigersfan | Status | new => assigned |
| Sep 27, 2011 9:28 am | keith.lamothe | Note Added: 0013840 | |
| Sep 27, 2011 9:56 am | Chris_McElligottPark | Note Added: 0013849 | |
| Sep 27, 2011 10:08 am | keith.lamothe | Note Added: 0013850 | |
| Sep 27, 2011 10:18 am | Chris_McElligottPark | Note Added: 0013852 | |
| Sep 27, 2011 1:14 pm | keith.lamothe | Status | assigned => feedback |
| Sep 27, 2011 2:29 pm | c4sc4 | Note Added: 0013924 | |
| Sep 27, 2011 2:29 pm | c4sc4 | Status | feedback => assigned |
| Sep 27, 2011 3:11 pm | Chris_McElligottPark | Note Added: 0013947 | |
| Sep 27, 2011 3:13 pm | keith.lamothe | Note Added: 0013950 | |
| Sep 27, 2011 3:15 pm | Chris_McElligottPark | Note Added: 0013951 | |
| Sep 27, 2011 3:16 pm | c4sc4 | Note Added: 0013954 | |
| Sep 27, 2011 3:21 pm | Chris_McElligottPark | Note Added: 0013964 | |
| Sep 27, 2011 3:37 pm | keith.lamothe | Note Added: 0013969 | |
| Sep 27, 2011 3:47 pm | Chris_McElligottPark | Note Added: 0013976 | |
| Sep 27, 2011 3:49 pm | keith.lamothe | Note Added: 0013980 | |
| Sep 27, 2011 3:58 pm | Chris_McElligottPark | Note Added: 0013991 | |
| Sep 27, 2011 4:23 pm | c4sc4 | Note Added: 0014009 | |
| Sep 27, 2011 4:31 pm | keith.lamothe | Note Added: 0014013 | |
| Sep 27, 2011 4:31 pm | keith.lamothe | Status | assigned => resolved |
| Sep 27, 2011 4:31 pm | keith.lamothe | Fixed in Version | => 0.502 |
| Sep 27, 2011 4:31 pm | keith.lamothe | Resolution | open => fixed |
| Sep 27, 2011 4:32 pm | Chris_McElligottPark | Note Added: 0014015 | |
| Sep 27, 2011 4:33 pm | Chris_McElligottPark | Note Added: 0014016 | |
| Apr 14, 2014 9:27 am | Chris_McElligottPark | Category | Bug - Crash or Exception => Crash/Exception |
