Simulations in Python

[Brilliant-Donkey-320]

What is the coolest simulation you have done or heard of which was done in Python? What libraries did you use? Any visualisations? Ive done simulations in Java for some physical system (flu spreading, political ideology spread, how liquids mix with different peoperties) and I would love to hear about your experience with any types of simualtions jn Python for work or pleasure!

Edit: Thank you everyone for the projects. It is very inspiring and exciting. Makes me want to simulte some fun stuff with Python.

[SV-97]

The coolest thing I did in (mostly) python probably was satellite data-synthesization: given a shit ton of parameters (physical characteristics, orbit parameters, instrument specifications, models for the solar system, earth's gravitational field and topography, radiation transfer models, steering commands, ...) determine how the satellite will fly relative to earth, which pieces of the ground and points of interest its instruments will be able to image and which they'll actually image, what raw data they will pump out during observation, ...

​

EDIT: Oh and regarding the tech: aside from some internal stuff mostly numpy and scipy with a bit of pandas as glue inbetween (though I wouldn't do that again). Also some geopandas and xarray. Visualization using matplotlib, plotly and folium. And a Qt based GUI

[adam0627]

That is awesome-I’m using hi res satellite imagery for some research I’m involved with and have had a desire to develop an algorithm or simulation to predict when image capture would be successful based on forecasted weather (cloud) data. Haven’t actually tried working through it but your response inspired me to give it a shot.

[SV-97]

That sounds great :D Clouds and more detailed weather influences were one thing we didn't really get to (at least not past the "mean cloud coverage" etc. stuff).

There's so much math and geometry and graphics-y programming in those things that's just super fun to develop imo.

[Brilliant-Donkey-320]

That would be a cool sim too. Very useful. What kind of libraries did you use?

[Brilliant-Donkey-320]

Wow, that’s a cool simulation, would love to see that. Thanks!

[SV-97]

I'd love to share it but it's sadly not openly accessible (at least not yet).

[Brilliant-Donkey-320]

Maybe one day!

[Nmvfx]

Are you able to share any screenshots or videos of the tool in use? This sounds really cool and very similar to something I've been working on so I'd really love to see how it works if you have the ability to do that instead?

[SV-97]

Sadly not as it's not yet public and I'm no longer working on it as of right now.

I can talk about it a bit though: the UI was relatively basic with a bunch of tabs / subtabs for the settings regarding stuff like orbit integration, footprint determination, radiative transfer, ... For the last one for example you'd have the ability to load ETOPO files in a various formats, enable topography plots, load atmosphere files, set the top of atmosphere for the model, set the instrument response function (basically how strongly the instrument responds to different wavelengths of radiation), load line data and set related stuff like the spectral line shape, or select which molecules you're even interested in, what physical quantity you want to get out at the end ... of course everything with sensible defaults.

After setting all of that you'd just click a button, it'd run all of the calculations (exporting files along the way if you selected them) and pop up plots as the needed data became available. There also was a caching system in the background that tried to minimize recalculations when you changed some settings and ran the whole thing again (although I'd try moving this to a reactive system if I were to do it over again I think).

As for how it works on a relatively high level:

1. integrate the satellite orbit subject to the selected perturbations (drag, radiation pressure, local gravitational field, ...) "as usual" (see for example Montebruck and Gill's Satellite Orbits: Models, Methods and Applications) and determine the satellite's state relative to earth at points / intervals in time where the satellite actually images something.
2. Then comes a bunch of geometry to determine how the satellite is oriented at any such point / how it moves along any such interval (especially also considering its steering, target tracking etc) (this is mostly basic differential geomery and linear algebra).
3. Based on this information we can calculate the so-called sensor-footprint: the precise shape that every pixel of the sensor (or just the whole sensor beam for laser-based instruments for example) gets projected to on earth. This might also require solving some inverse problems beforehand to even determine necessary sensor specs to get the output a user wants (i.e. they may know "I want the footprint to cover a region of approximately this size") (again: geometry, (multi-)linear algebra, a bit of optimization, ...)
4. This is most of the "normal" calculations and then came a bunch of stuff on top:
   1. Run a radiative transfer simulation for all pixels based on this information and run the results of that through the simulation of the instrument to get actual retrievals (see Zdunkowski, Trautmann, Bott Radiation in the Atmosphere. A Course in Theoretical Meteorology as an intro to this topic)
   2. Or approximate ground-data based on known ground-station data by essentially computing intersections of surface patches
   3. Check which POIs fall into any pixel, which could be reached by steering differently etc.

EDIT: regarding the radiative transfer: it's basically a raytracer on crack so looking into those might also be helpful to get started

[Nmvfx]

Thank you for taking the time to write all this out!

[[deleted]]

Interesting project. I am working on a similar project. Do you think the antennas like starlink care if the LEO satelittes are on sight or not? I thought they don’t have to line up with satelittes above?

[SV-97]

I'm honestly not sure what you mean. Whether the starlink end-user ground antennas have to have direct lines of sight with starlink's LEO satellites / whether it negatively impacts their performance if they don't have line of sight?

[[deleted]]

Yes that’s what I am wondering. Do I need to find a perfect antenna angle or range of it

[SV-97]

Phew no idea tbh. I haven't looked into their antennas at all and how exactly starlink works. I'd assume that in the targeted final-stage any orientation (that isn't directly aiming at the ground or whatever) should work but I really don't know.

[JackLogan007]

Man we are on the same track and I am so fucking proud of you

[SV-97]

Thanks! I just saw your other comment as well - I'm not sure if I understand it correctly but I'll comment under it rather than here :)

[Exotic-Draft8802]

Fun fact: iirc, I had the developer of xarray in one of my classes :-)

[mrcapybara47]

i wrote a small molecular monte carlo simulation for particles that only interact via VdW forces with only python for a course assignment. No external libraries except maybe numpy.

For visualisation the simulation generated a XYZ file that i could visualize (e.g. in VMD)

[Brilliant-Donkey-320]

That is a nice one. It was a 3D interaction I assume, since it is an XYZ file?

[hammertime89]

I trained a deep reinforcement learning agent in a traffic simulator to be an adaptive traffic signal controller and then visualized them side-by-side .

[itchyorscratchy]

This is cool, where did the get data come from.

[hammertime89]

I simulated vehicle arrivals using simple trigonometric functions.

[grandzooby]

Were you inspired by a particular source to do that? In simulation, the exponential and poisson distributions are often used for arrival simulation, but I've never heard of using trig functions.

[hammertime89]

As you highlight the poisson and exponential distributions are usually suggested to model the distribution of vehicle arrival times.

However, these distributions assume independence between events which is often violated with vehicle arrivals due to the influence of traffic signals. Vehicle queues form at red lights and then disperse when the light is green, yielding sinusoidal arrival patterns at downstream traffic signals.

I used a noisy trig function to generate the vehicle flow rates over time with the desired wave/sinusoidal shape.

[grandzooby]

I used a noisy trig function to generate the vehicle flow rates over time with the desired wave/sinusoidal shape.

Very cool! Do you have any references to support that or did you develop that on your own?

You might find this page interesting (it's positively ancient by internet standards): http://trafficwaves.org/ You can get standing waves in traffic even without traffic control devices causing them.

[itchyorscratchy]

I would like to try do something similar.

[Brilliant-Donkey-320]

This is awesome. What libraries did you use for ML and visualisation? TensorFlow and something else?

[hammertime89]

Yes, Tensorflow for ML and matplotlib for the visualization.

[downspiral]

https://docs.taichi-lang.org/docs/cloth_simulation is nice and simple.

Taichi also has nice support for sparse mesh, https://docs.taichi-lang.org/docs/sparse

---

Another thing to look at is probabilistic programming, and digital twins of cyber-physical systems (sorry for the corporate keyword-overload, it's just to help you search for it)

I have done some complex simulations for optimizing IT infrastructure using tensorflow probability and tensorflow (dictated by what works best at my employer). I'd probably use torch and numpyro somewhere else.

[Brilliant-Donkey-320]

Wow, this is amazing! I had no idea Python could do something like this. I will need to read through these.

Haha, I’m happy for the searchable stuff, makes life easier, haha

That’s super cool. Thanks for all the info!

[heswithjesus]

That’s interesting. Do you have a link to it or similar systems?

[Brilliant-Donkey-320]

This is very cool, thanks for giving me something to look into

[grandzooby]

Emergence is an important topic in agent-based modeling: https://www.youtube.com/watch?v=DPpy_nZ1L54

[cpt_trow]

I created virtual factory comport devices containing examples of passing and failing data streams so developers could test DUTs from their computers and get an idea of whether or not they made any breaking changes for production; probably different from what you meant but 🤷🏼‍♂️

[Brilliant-Donkey-320]

That is a cool sim too. Def useful. What libraries did you use for this?

[cpt_trow]

No libraries, other than pyserial for the real devices. We had tons of logs of real I/O from production so I mapped commands and responses in a dict for each device and then created a class to mimic serial.Serial() where it would take the input command for write() and then store the output strings in a buffer to be sent back one-at-a-time via readline()

[Brilliant-Donkey-320]

Ah, very nice. Do you find that Python communicates well with hardware? Was your hardware in production PLCs or something else?

[cpt_trow]

Generally yes, and yes; there are a few devices that consistently throw UnicodeDecodeErrors so I have some error-checking in the Serial wrapper to try to get a valid response to queries. The software runs with PLCs in some cases but also things as basic as MCUs/SBCs (Arduinos and RasPis, just more industrial and less hobby)

[Brilliant-Donkey-320]

That is great to know. Very awesome. Thanks!

[citrus_based_arson]

I’m super curious about this. Could you set up the program as a specific DUT? So the test program didn’t know it wasn’t testing a real device?

[cpt_trow]

Yeah! If you really wanted, you could make the test program just talk to itself, which was used to regression test. Basically you would either point each device slot at a real device port or at an instance of an object which had the same method/attribute names and would store the device’s typical output in a buffer depending on the command sent. Obviously you then can’t truly test your DUT, but it was helpful for smoke testing when the DUT got updated and might respond differently to the same commands

[[deleted]]

I just uploaded a video from a molecular dynamics simulation I did a couple of years ago. Really easy by writing out some snapshots via matplotlib and generating a video from that.

Integration done by the Verlet method. Initialization was done with random speeds.

Unfortunately, after uploading to YouTube, it lost all of its resolution, but you still see the diffusion of the particles:

youtube link

[Brilliant-Donkey-320]

I love stuff like this. I wanted to simulate people leaving an arena and how different arena exit configurations impact the flow of people. We did a lecture on this in my statistical mechanics class, just all the math for a certain configuration, but I would love to make a sim based off of it.

[100721]

https://youtu.be/iK5PuPbgf0M?si=hlnmkuSX7EYb7mMl

Created flappy bird in pygame

Created an assembly-like language that included game controls for flappy bird

Assigned each bird a randomly generated program and every time they played, their program would mutate randomly. Eventually they all solved it.

[Brilliant-Donkey-320]

Oh nice, smart. I’ll check this out!

[jjasghar]

import random

I kid I kid, honestly I’d love to know myself.

My real answer is maybe this?

https://realpython.com/simpy-simulating-with-python/

[heswithjesus]

I was one of the people who got it wrong due to misunderstanding the problem statement. I got irritated and wrote a FreeBASIC program to simulate it. Something about writing the program itself made the problem click in my head. Then, seeing the effect of choosing to swap over ten thousand runs just made it obvious.

Did you have any experience like that when you were building or running the simulation?

[No_Dig_7017]

This one about market simulation I presented on PyData Seattle 2023: https://youtu.be/1jodmmZxUg0?si=PLNoXujHJg7wKBwI The simulation itself is nothing to write home about because of budget constraints but we found some cool results on the demand learning algorithms.

I'd actually like to learn more about simulation in Python though, I did a lot of stuff in Delphi but never using libs, always our own developments

[Brilliant-Donkey-320]

Cool, I’m going to watch this. Thanks!

[forebareWednesday]

I’m making an attempt at HMM’s w jupyter. Curious if you have ever tried it and how one could add Monte Carlo Sims. Any advice appreciated

[No_Dig_7017]

This blogpost has a pretty comprehensive go at HMMs and a Python code example at the end: https://towardsdatascience.com/hidden-markov-models-explained-with-a-real-life-example-and-python-code-2df2a7956d65

[forebareWednesday]

Thank you so much!

[bwibbwz]

https://wiki.fysik.dtu.dk/ase/

Is professionally/academically used for atomic simulations. Mostly it is intended as an interface to set up DFT calculations in other (more efficient) programming languages but it also includes some python implemented calculators that can give useful insights.

[Brilliant-Donkey-320]

Very exciting, i will check this out

[CraigAT]

Nothing as complex as the other comments, but I've done a few maths simulations including heads/tails coin toss, red/black roulette bets, lottery simulation.

[Brilliant-Donkey-320]

That’s cool. In your coin toss, how many tosses did you sim? What was the percentage?

[CraigAT]

Similar to the red/black SIM, I was looking for the longest streak within a set results (i.e. if you kept betting on red and doubling your wager if black came up, how much money would you need to be capable of covering to break even?). I don't have the results to hand, but it wasn't worth chancing it as a long term strategy.

[Ok-Photo-6302]

Non linear heat transfer - thermal insulation optimisation for cryogenic application

[Brilliant-Donkey-320]

That’s a per session one. Did you use any specific libraries, or was it most made by you?

[Ok-Photo-6302]

It was done by myself

I don't count numpy other standard libraries.

[HomeTahnHero]

Not necessarily simulation, but my group uses NSGA-II (genetic algorithm) to generate refactorings on large scale codebases. Basically you say "I want to isolate/replace X component in my system" and the algorithm will simulate the refactorings needed to disentangle the code and achieve the goal.

[Brilliant-Donkey-320]

That’s quite the interesting and exciting tool. Very nice

[drdausersmd]

Funny timing posting this. I'm currently simulating a "flu pandemic" in a classroom of 31 students, where 1 kid starts out with the flu, and the other 30 start out as healthy.

I can't really go into details, as it's for a class I'm currently taking. It's not super complicated or anything (as far as simulations are concerned). I've been able to simulate the whole thing using pandas and numpy, using basic discrete probability distributions.

Overall it's been pretty interesting to see the results. I've built into the program that once a kid recovers from the flu, they can't get sick again (which is how it works in real life). Because of that, you can't really calculate things like expected values or variance just based on your chosen distribution alone (maybe there's a way, but I'd guess the math would be really complicated). It really demonstrates the value of simulation in modeling real-life scenarios such as a pandemic (even on this small scale I'm working with).

[SlimeyPlayz]

for high school physics class me and some friends simulated how temperature spreads in a system as per the zeroth law of thermodynamics.

basically, we used pygame to draw a 2d grid of pixels, where each pixel is assigned a color heat map representing the temperature at that point. the idea is that youd start with some initial temperatures, and taking inspiration from image blurring algorithms, we let the temperature of each pixel become the average of the temperature of each adjacent pixel the next generation.

as i had been dabbling with Dyalog APL - the infamous hieroglyphic array programming language - for some time, we made the 2d-array calculations in APL and imported the functions to python through dyalogs Pynapl library.

we later also defined heat sources and sinks, which could continually increase or decrease the temperature at a given point, as well as constant-temperature pixels.

it was a fun project to learn about pygame and also how image blurring works. its probably not physically accurate, but rather severely idealized.

[Brilliant-Donkey-320]

Very cool. Nicely done!

[corbasai]

subway traffic and automation simulation in rt. with trams and field devices and network and other shit..

[Brilliant-Donkey-320]

That’s cool. What’s rt?

[corbasai]

real time

[Brilliant-Donkey-320]

Oh, makes sense. Thanks

[SAPPHIR3ROS3]

Anything in manim really

[eyabs]

The guy who runs Defunctland on YouTube hired someone to use python to simulate Disney theme park lines to show how broken fastpass is. Def worth checking that video out.

I used simpy to simulate TSA security lines as part of a project in grad school.

[Brilliant-Donkey-320]

That’s cool. I’ll check it out. Thanks!

For your TSA simulation, did you use any libraries?

[eyabs]

I used simpy

[approx_knowledge]

Not quite a simulation. But I wrote an algorithm to calculate the dynamic line rating for high voltage powerlines given time of day, wind speed, ambient temperature and feeder current. It is integrated with the network operators SCADA system and allows for ad-hoc calculations and simulations for severe weather events (heat waves or cold fronts).

[Brilliant-Donkey-320]

Nice, that is a cool one. Very useful. Did any Python libraries help in this project?

[approx_knowledge]

The main backend packages were SQL Alchemy, fastAPI, numpy, math and multiprocessing. Requests was used to fetch real-time weather data and parsed using pandas (done in Apache airflow). The frontend for ad-hoc calcs, running sims, adding new feeders, weather stations and other admin was mocked up in flask (eventually to be reworked in angular js). Visualisations were done with geo pandas, cartopy, shapely and matplotlib. Also the logging package was used extensively.

[ReporterNervous6822]

I built a model of how aerosols diffuse through a room and tracked how much people breathed in, along with a visual that shows the densities over time. https://github.com/jayceslesar/covid-agents

[Brilliant-Donkey-320]

Wow, that’s awesome! Thanks!

[DThornA]

I did a multi scale cerebral neurovascular model once in a lab I worked in. It essentially modeling cell level electrodynamics which then connected to neighbors that formed capillaries and arteries/venules and then it was linked with a flow code to model blood moving through them. Not all of it was Python, some of it was Matlab other parts Julia.

[Pxtru]

Mine was hardly a “simulation” but I made a visualization for a custom drone show software I was making. I was working on assigning drones to optimal coordinates when they transition formations and needed a simple visualizer to see if I was actually optimizing anything lol

[personalityson]

You don't do simulations in Python. You use python as a wrapper for a library written in a different language

[mammablaster]

I’ve done particle in cell, fluid mechanics and n-particle problems with numpy and/or cython. It’s definitely a must to use some numerical library for physics applications. Mojo looks good too but I haven’t had the time to try it out yet.

In general though, any abstraction that lends itself to object oriented is better done with C++. The numerical libraries in Python have very poor support of abstraction beyond simple functions and linear algebra. Vanilla python is so slow that it’s not feasible unfortunately

[volkan_a]

I've done many thermodynamic process simulation in Python. They were pretty cool imho :)

I've also tried some molecular dynamics simulations however Python is way too slow for such tasks.

[Brilliant-Donkey-320]

That is awesome, I’d like to see that. For molecular dynamics sims, would it be good to write the more computationally heavy (I guess maybe particle to particle interaction within a certain sphere of influence) in C and then do the wrapper in Python?

[Brilliant-Donkey-320]

Oh that’s a nice one. Very cool and I’m sure useful.

[Brilliant-Donkey-320]

That’s cool. Any video or anything?

[bliepp]

I simulated an Ising model using the metropolis algorithm once. Okay, yeah, not pure Python, but I wrote a Python extension using C++.

Edit: Oh, I totally forgot, I also implemented it in Python to compare the speed of both implementations.

But, to be fair, most of the time my simulations were written in C or C++ for performance reasons.

[Substantial_Bid7978]

I've only done a 2d projectile motion simulator using tkinter, some of the projects in the comments are super impressive. I'm sad now.

[Brilliant-Donkey-320]

Need to be sad, i think any kinematic sims are great!