Page 1 of 1

[Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Mar 22, 2019 1:39 pm
by ps666
Hello,
I don't know if this is important or if you care, but the RAM usage and the time to save the game increases over time. Factorio starts with ~400 BM. After loading my map it has around 700 MB and later in the day after hours of AFKing it is around 5-6 GB and the save time is much longer as in the beginning of my playing.
I mean the green bar is filled fast but the time until it starts to fill increases.

I play with mods and maybe that's their fault. I don't know....

Greetings

Re: [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Mar 22, 2019 1:50 pm
by posila
Hello, thanks for the report.
Does the memory usage stay high if you restart the game and reload the save? Can you upload the save?

Re: [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Mar 22, 2019 2:22 pm
by ps666
Yes, it goes down to normal values after I closed my map (no restart of the game needed).
My save with mods:
https://mega.nz/#!X7AhhaCZ!xRkgusyXucuo ... 02oRSf0gyE

Edit:
I'm not sure but maybe it depends on the game speed increase which I started to use a couple days ago. Then it's my fault I guess and not a bug :)

Edit 2:
I tested/observed it the half day without using a speed increase cheat and it rises too. Slow but steady.
Greetings

Edit 3:
https://imgur.com/awgSw99
Image

Re: [0.17.17] RAM usage and the time to save the game rises over time

Posted: Thu Apr 11, 2019 11:27 pm
by SNACKaJACK
I currently have the same issue on a headless linux server I manage. RAM usage starts at 480 MB and over a few hours it slowly grows to 2 GB then it crashes because the current server only has 2 GB of RAM. Sometimes that even ends up in the last save being corrupted. When you then restart the server again its back at 480MB. The server is on 0.17.28.
Whatever is causing the RAM usage to go up takes very little CPU power we usually hover around 20% usage with only a single core at 2.4GHz, except when the map is being saved then the cpu gets pinned at 100% for a few seconds while the progress bar is at 0% and as soon as the progress bar starts moving cpu usage starts to drop again as well.
Here is a link with nearly all the data from the server https://drive.google.com/drive/folders/ ... sp=sharing .
Since this problem also seems to slow down the saving of the map, one of the things that came to my mind is that it might be related with this issue: viewtopic.php?f=182&t=65826 . However i don't have any data to support that, its just something that came to my mind.

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 1:41 pm
by Rseding91
I looked into it and found the problem. The Disco Science mod is using tables as keys and is storing them forever (until the map exits). Every 5 ticks it reads force.current_research and stores it as a key in a local table.

There are a few things wrong with the mod:
  • It should not be using tables as keys - especially LuaObject tables which are different every time you read one from the game.
  • It should not be storing mutable game data outside of the global table
Remove that mod and your memory issues and save time issues will go away.

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 1:50 pm
by SNACKaJACK
Thanks for the help, I will remove that mod now. I will post back here in a few hours to confirm if that fixed the issues.

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 3:15 pm
by SNACKaJACK
You were right, the disco mod was causing the problems. After an initial period where RAM slowly went up a bit it now stabilized at 600MB. That is the expected amount of RAM usage for a map like this I think. Thanks again for the help!

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 6:09 pm
by danielbrauer
Thanks for looking into this, Rseding! Thanks Bilka for explaining the issues in detail. And thanks, SNACKaJACK, for letting me know on the mod portal!

I think I have addressed both of these issues in DiscoScience 0.0.10.

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 6:45 pm
by posila
danielbrauer wrote:
Fri Apr 12, 2019 6:09 pm
Thanks for looking into this, Rseding! Thanks Bilka for explaining the issues in detail. And thanks, SNACKaJACK, for letting me know on the mod portal!

I think I have addressed both of these issues in DiscoScience 0.0.10.
Thank you for fixing the mod!

Re: [Rseding91] [0.17.17] RAM usage and the time to save the game rises over time

Posted: Fri Apr 12, 2019 7:14 pm
by danielbrauer
posila wrote:
Fri Apr 12, 2019 6:45 pm
Thank you for fixing the mod!
I wasn't actually able to reproduce the memory leak on macOS (or at least not so as I could detect it with Activity Monitor), so I fixed the issue somewhat blindly. I think I have a pretty good understanding of what the problem was, but if anyone can confirm that it's no longer leaking, I'd be very grateful!