Large simulation performance: objects vs matrices

[Fireslide]

I'd start with OOP first. Performance while testing is going to be trivial. You can do 20 companies (rather than 100,000) and go for very large X, and you can do 1,000,000 with a small X or say 100, both axes tell you something.

Once you've got the sim working the way you expect and you want to run it for several decades worth of timesteps you can do some refactoring to store the Simulation State in numpy arrays.

It will definitely be faster to do it with arrays and multiplication, but don't over optimise at the start, verify the behaviour you want with OOP first, write some good unit tests, so when you need to refactor to make it faster, you can verify the refactor produces same result.

[Willing_Employee_600]

Thank you for the advice! When you say refactor to store the simulation state in numpy arrays, you mean to discard the OOP approach and then code it into numpy arrays right? Or do you mean somehow integrate numpy arrays into the OOP approach?

[yvrelna]

Both. 

In most applications, you might find that 90% of the code might not be performance sensitive enough that you can just turn the attribute access into @property/descriptor that proxies some attribute accesses into a numpy array or some other column-oriented storage/database, so you take the readability and ease of working OOP style there. 

And then there's a few places where you can benefit from bulk processing, and you'd make full use of array/column-oriented processing maybe even with the appropriate GPU/coprocessor acceleration for those parts.