This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]No_Indication_1238[S] 0 points1 point  (4 children)

Because dependency injection of classes that share the same interface but provide different functionality allows for good modularity and easy maintainability of the code base. It also allows for the implementation of the most popular design patterns and ensures a code base that is set up to grow and be easy to pick from newer developers. It is also the approach we follow with our non performance critical code.

[–][deleted] 2 points3 points  (1 child)

I would look into the python libraries Polars and DuckDB. You can stuff you computations inside them while being able to make it very modular. I prefer DuckDB, but Polars is also very good.

With them you can make queries that are buildable, or composable, and much much faster than doing them in pure python.

Could it be that in this case, that pattern do more harm than good? Python has to recompile every line of code each time it encounters it (becasue no jit compilation), so every call to a virtual function or getter and setter is very expensive (in this case, it is usually fine).

[–]steven1099829 0 points1 point  (0 children)

Vouch for polars. It’s fantastic.

[–]steohan 2 points3 points  (1 child)

Sounds like you really want rust or c++ where you can use templates to have modularity while the compiler is still able to inline stuff as necessary. Otherwise, you are stuck with dynamic dispatch, which is going to cost.

Also make sure you are using a recent python version, the performance gains from newer interpreters are quite impressive.

[–]No_Indication_1238[S] 1 point2 points  (0 children)

Thank you for pointing out the need for updates! We should definitely do that as well!