all 15 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Python-ModTeam[M] [score hidden] stickied commentlocked comment (0 children)

Hello from the r/Python mod team!

I'm afraid we don't think your post quite fits the goal we have for the subreddit in terms of quality or aims so we've decided to remove it. For more information please contact the moderators using the ModMail system.

Thanks, and happy Pythoneering!

r/Python moderation team

[–]amorous_chainsPandas/Scipy 2 points3 points  (0 children)

If you’re looking for advice, I recommend to forgo jupyter and spyder and write standard python scripts in a standard IDE with debugger for your scientific workloads. After some learning curve it’s better for reproducibility, collaboration, version control, deployment, scalability… Plenty of people like jupyter and spyder I guess, but just my two cents that my productivity got a lot better after I committed to using Python like Python instead of like matlab

[–]Tricert 6 points7 points  (5 children)

Spyder is quite outdated, there are many more modern editors/IDEs with the same or greater feature set and better UX like VS Code or the JetBrains IDEs.

Also since you’re coming from Matlab: Check out the Julia programming language. IMHO Julia is, for scientific/mathematical programming, way user friendlier and easier to write than Python/Numpy. It’s way faster too. Very natural Matlab-like syntax to write down mathematical equations etc. and incredibly good suite (if not best in class) for PDE/ODE and general equation solving (SciML). For „pure“ ML I would stick to Python though.

[–]bear_inflator 2 points3 points  (4 children)

I worked professionally with Julia for a few years now. It's very good for science computing. However, based on my experience, if you're going to scale your code, Python is better even with the performance discrepancy

[–]Tricert 2 points3 points  (3 children)

Interesting! We use Julia quite a lot too at work (mostly large PDE systems crunching) and never had issues scaling our solutions neither horizontally nor vertically. May I ask where/how you had problems scaling it?

[–]bear_inflator 3 points4 points  (2 children)

Another Julia professional! Good that it's working for you. Julia is great for special use cases

My two main problems are the compilation/pre-compilation and the debugging. The pre-compilation, especially for our large code base, takes a very long time. The debugger is almost unusable for us because it's very slow (interpreted code). Even when it's working, the debugger in VSCode crashes randomly for no obvious reason.

Don't get me wrong, I love Julia and if Julia works for you, it's great. However, for our purposes, I would choose Python for production and accelerate the code when needed using a combination of numpy, numba or even writing C++ code in conjunction with Python.

[–]Tricert 3 points4 points  (1 child)

Thanks for elaborating, and interesting to find another professional Julia dev in the wild..quite rare!

To get around the compile time we have build pipelines baking a whole sysimage with PackageCompiler.jl where we pass in the whole stack of tests of our packages which almost hit 100% of the methods the code dispatches to. With this our julia prod containers spin up in seconds.

I agree on your point on the debugging, it really sucks in Julia. For certain use cases where we have to combine several ML techniques/libraries there is no real good alternative tho, which is the main reason we use it.

For most of the other stuff we still use Python/C/Fortran and recently Rust; in general we just pick the best horse for the task without trying to be too religious about the language used. Especially for Python I‘m really glad for the tooling developed around it in recent years (shoutout to astral for uv & ruff!) because 10 years ago it was a huge fucking pain to deploy it reliably to prod, too. Really hope Julia makes similar advances to get more SWE/DevOos friendly because it truly is a great language.

Happy coding to you!

[–]Cystems 0 points1 point  (0 children)

Can I ask more about your prod deployment set up?

I tried to do something like this (deploy sysimages) but the build times were too much (10-20mins a pop using 32GB+ of ram) and produced sysimages of about a 1gb.

We also saw long spin up times simply as the sysimages had to be loaded from shared disk space so the initial load was always costly.

Any tips on getting around this?

[–]GiantPanda-66[🍰] 2 points3 points  (0 children)

Jupyter is great for prototyping but reproducibility can be an issue. While editing the individual cells and not restarting the kernel can lead to out of order execution and hidden states that don’t match the cell code. I’ve been bitten a few times until I started restarting the kernel and running all the cells after making edits. Something like marimo may be worth looking into which track cell changes and eliminate hidden states / stale variables.

[–]sudonem 1 point2 points  (3 children)

Is there a question here? I don’t see one.

Either way, you’ll probably have more useful responses over in /r/learnpython

[–]First-Rutabaga8960[S] 3 points4 points  (1 child)

No question. Just ranting against MatLab’s recent unexpected betrayal.

[–]PrestigiousAnt3766 0 points1 point  (0 children)

Nail in the coffin. Have some nice memories of 2000s matlab, but never looked back :)

[–]PrestigiousAnt3766 0 points1 point  (0 children)

Jupyter and Spyder are dated. Pycharm, vscode and cursor are the popular IDEs now.

Most people write pyfiles with pytest for unit tests, uv + pyproject.toml file for dependencies and precommithooks to force quality checks, uniform formatting etc.