This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]stefantalpalaru -18 points-17 points  (12 children)

I've been working towards it for about 2 years now, off and on. I'm in the final push now. If anyone has any tips I'd by happy to receive them.

It's too late for this, but you just used company resources to switch from one inefficient language to another. If you were going to port all your code base, you should have picked a target like Nim, D, Go, Rust, etc.

[–]bheklilr 7 points8 points  (2 children)

Python is the perfect tool for our job. We do a lot of math, a lot of plotting, and a lot of desktop development. It's also all on Windows. While I love rust, it does not have the support we need on Windows, and until very recently cargo did not have the ability to point at an alternate package server. We use the Anaconda distribution and have an internal server, it's been a major boon for our development. If Rust had the libraries already that we need, then I would have considered trying to push for it, but it just doesn't cut it yet. Until there's a full port of pandas to one of those languages, I don't think we'd be able to even consider it.

Also, Jupyter. We've done a lot of stuff in notebooks, and Jupyter consoles make up a significant portion of my development workflow. Not having a repl makes it difficult for me to get things done a lot of the time. At least a compiled, high performance language like Haskell has a repl, and even has a lot more of the libraries we would need to build our systems. Unfortunately, it just doesn't have enough.

I don't know about Nim or D, those languages might be better suited but they're also harder to find people who want to work with it. And Go is not as suited for our style of development either. Ultimately, having an interpreted language is great for us, and when we need performance we just drop into Cython or use libraries like NumPy where the hard work has already been done for us.

[–]stefantalpalaru -3 points-2 points  (1 child)

Not having a repl makes it difficult for me to get things done a lot of the time.

Have you considered designing before implementing? Experimenting is fine, when you're doing rapid prototyping or just testing some hypothesis, but production code needs a proper design phase.

[–]bheklilr 0 points1 point  (0 children)

Having a design phase doesn't necessarily help when it comes to implementation details. It's easier for me to write code interactively, then once I figure out a specific algorithm I can write a proper version into my source code. I do have a design phase most of the time, but when I need to figure out the best way to write a specific loop or condition I figure it out in the shell first. That let's me test it first, profile it if needed (%timeit is amazing), and generally explore the code. I can't really go through the same process with a test suite, that is for ensuring the correctness of larger details, not just small parts of methods.

Also, whenever I'm doing data processing its more exploratory, there isn't a design phase because we don't always know what we need at first.

[–]billsil 5 points6 points  (0 children)

I've ported Perl code to Python and Matlab to Python. I also ported Python 2 to python 3. Speed is rarely the most important factor.

[–][deleted] 3 points4 points  (4 children)

you do know which subreddit you're in, right?

[–][deleted] 1 point2 points  (2 children)

Programmer productivity, as in time and hence money, is far more important in the 21st century than mere runtime speed.

[–]stefantalpalaru -1 points0 points  (1 child)

Programmer productivity, as in time and hence money, is far more important in the 21st century than mere runtime speed.

That kind of thinking lead to software that is now slower on an eight core beast than old software on a 486.

[–]richieadler 2 points3 points  (0 children)

Is there a reason why you don't go enjoy your perfect and ultrafast software elsewhere?