Performance and Scalability

[imahmood786]

Hi
Can you give some tips on how to improve step performance for a large scale simulation?

Regards

/Imran

[kushal-satya]

I am also looking for answers on the same. Is mesa-geo fine to work with large regions or populations, like few million agents simulating a country? Could you please let me know?

Thank you so much!

[imahmood786]

I have explored it deeply. It is very challenging to run even a 100 thousand agents with some basic features in this framework.

[kushal-satya]

@imahmood786
Thank you so much for the response. Do you suggest a safe limit we can work with? Also, if there is any any alternative to this to work with large number of agents?

[Harshpanday]

Hey,
Did you guys figure out a way to handle a large number of agents?
I'd be very grateful if you could let me know how to deal with a large number of agents if you figured out any.

With kindest regards,
Harsh Panday

[rht]

Sometimes, mesa-geo itself might not be the bottleneck. You can read the thread at projectmesa/mesa#1185 for how to profile your code.

[Harshpanday]

Thank you for the reply,
I didn't understand how I can store my agent attributes in a NumPy array.
I am adding my agents like this
ERHC = AgentCreator(erhc, {"model": self, "fa": 0})
agents_erhc = ERHC.from_GeoDataFrame(erhc_values, unique_id="id")
i=0
while(i<len(erhc_values)):
for agent in agents_erhc:
agent.longitude = erhc_data["longitude"][i]
agent.latitude = erhc_data["latitude"][i]
self.grid.add_agents(agent)
i = i + 1
self.schedule.add(agent)

The exact problem I am facing is that when I run the model with >8K agents, the MAP doesn't display any agents it's very unstable.
https://github.com/Harshpanday/Store-Closure this is my repo with the code If you have time and want to take a look.
I will be grateful for any suggestions.

[rht]

The exact problem I am facing is that when I run the model with >8K agents, the MAP doesn't display any agents it's very unstable.

So the slowness happens only in the GUI? If you run without GUI, it is fast?

[Harshpanday]

It runs as it's supposed to, but yeah the GUI is very slow, it takes the map 2-3 minutes to display the agents and when I zoom in or zoom out on the map the frames per second are also very low.

[rht]

Maybe you should just smoothen out the agent circles into a heatmap. You can wait until #67 is merged, and add a layer that is computed from the circles density or something.

[mrceresa]

Some initial consideration on performances here: wang-boyu/agents-and-networks-in-python#2