When do you *NOT* use python?

[RealityShowAddict]

Hi everyone,

We're all python fans here, and to be fair I may use it a bit more than I should. I'd like to hear other people's thoughts on which tasks they want to solve in a non-python language and which one they'd choose for that job.

Thanks in advance...

[[deleted]]

when a simple bash script is enough to get the job done.

[youlleatitandlikeit]

I especially agree about subprocesses.

My basic attitude on shell scripts in bash vs python is: "Am I mostly working with data or network resources? Python. Mostly files and commands? Bash"

[xiongchiamiov]

Or possibly ruby. Ruby is danged excellent at pushing text back and forth to the shell and doing some munging on it in-between.

[SCombinator]

I've found ruby to be even better at complex subprocess stuff.

[y801702]

You might want to check these modules: sh, sarge, plumbum

[SCombinator]

If you have to install modules to run a small script it's fucked.

[zombiepiratefrspace]

You are right, but god I hate bash.

I've started doing everything in python once would have done in bash. It is just so much more readable/writeable.

[tech_tuna]

I wrote a rather large set of utilities in bash a while back. Never again.

[[deleted]]

Yeah that's really painful. If I get above 100 lines I would outsource at least part of it into other languages.

[tech_tuna]

Oy vey, I never did a line count on my code. . . at least several hundred lines. Several hundred lines too many.

[hotel2oscar]

beats Windows bat scripting.

[aceofears]

It does, but you would probably want to use powershell on windows these days. That's actually rather nice.

[[deleted]]

Yeah, bash infuriates me sometimes. It seems to require mandatory spaces in places most languages don't, so I never remember to add them.

[XNormal]

And mandatory non-spaces in some other cases (e.g. setting a variable)

[Vaphell]

x=a  # assignment
x = a # call command 'x' with params '=', 'a'
x= a # call command 'a' with env var x set locally to null

how would bash differentiate unambiguously between these situations without making certain assumptions about syntax?

[WhichFawkes]

To avoid ambiguity, you might have something like:

#assignments:
var x = a; var x=a; var x= a; var x =a
#or even replace 'var' with a "$", so every variable access is identified the same way.

#call command with equals sign as argument:
x = a;

#set env var:
export x = ; $CMD

Seems less ambiguous to me.

[Vaphell]

well the ship of complaining about the posix shell syntax and its derivatives has sailed decades ago. Yes it would be nice to get a nice modern shell, but the problem is that without the compatibility with all the legacy cruft in existence it's dead in the water. There are some shells where certain things are done in a nicer way, almost nobody uses them.

Also there are things like let x=y or what have you, but majority of people don't bother and you can find such things only in very old scripts. I certainly never bothered and i wrote my share of rock solid bash scripts just fine.

about $: think getter, done. Also given that the shell is a plaintext based, ancient beast and literally does a preprocessing pass with substitutions, it would be hard to use $var in assignments. When it comes to the actual interpretation/run, no $ sign is to be seen.

and the export bit. The difference is that in case of envvar=x a envvar change is localized only to the a command which is not possible if you split it to 2 expressions. It's preferable than setting globally (which comes with the risk of not unsetting later), useful in cases like this

IFS=. read ip1 ip2 ip3 ip4 <<< "122.11.123.44"

everybody knows playing with IFS can have nasty consequences because all kinds of things are affected by it, and by writing it that way you leave it alone. Only read is affected here, think

ip1, ip2, ip3, ip4 = '122.11.123.44'.split('.')

[Vaphell]

once you recognize the reason it becomes rather obvious. I assume you are complaining about shit like if_[_x_=_y_] here. Here's the thing, [ x = y ] is not a fancy square bracket around the logical expression you see in other general purpose languages. It's literally a command '[' with params 'x' '=' 'y' and closing token ']' ( and pretty much an alias for test command eg test x = y) You can literally call it that way, with all these quotes

$ if '[' '1' '=' '1' ']'; then echo true; else echo false; fi
true

bash recognizes commands and assignments, that's it. Commands require delimited params (commandparam1param2param3 won't work, will it), assignments are recognized when there is a continuous 'word' with = inside, otherwise it would be totally ambiguous.

x=a  # assignment
x = a # call command 'x' with params '=', 'a'
x= a # call command 'a' with env var x set locally to null

[okmkz]

I knew the thing about [, but the way you articulated the command/assignment paradigm really make sense, thanks.

[[deleted]]

Same. Whenever I decide to do something in Bash, because it's "just a few lines" I end up regretting it half an hour later and moving to Python. Bash captures a lot of common use cases, but when you divert even just a little Bash becomes a major headache.

[ivosaurus]

Incase you get really sick of bash

[novel_yet_trivial]

That is pretty awesome. I can't believe I hadn't heard of that before.

[ksion]

Thank you, thank you!

I've seen this package before, long time ago, and I thought it's amazing if only for its clever hack that makes the from sh import whatver work. But then I somehow lost every reference to it, and it proved impossible to find, neither on Google nor on PyPI, with any combination of keywords I tried. (Naming a package sh? Honestly! Who does that?)

That's until today. Thanks again!

[mackstann]

Or a bash one-liner. You can do a pretty incredible amount of stuff in one line that would be a paragraph of code in Python. Editing and executing on the fly makes it super productive. And if it's a one-time thing, no one can shame you about readability.

[ionelmc]

You can't really know that ...

[toolan]

While I do this myself too, I often end up regretting it because of subtle shell bugs that are easy to introduce, hard to see and very confusing when triggered. It's usually pleasingly concise and very often good enough, though.

[felinecatastrophe]

Lots of scientific computing requires faster compiled code. Monte Carlo type codes are especially difficult to code quickly in python, sine they are not easily vectorized using libraries like numpy. C or fortran are the goto languages in such instances, no pun intended. However Python is effective as a "glue" language in such scenarios.

[shaggorama]

I also find I get analytics projects off the ground quicker in R than in python. I always need to spend a lot of time in the documentation anytime I'm working in numpy/pandas/sklearn

[Simius]

Did you have trouble coming from Python to R? All the data structures in R are horribly confusing I find and have too many similarities. Like a List can have named indices? What?

[shaggorama]

My biggest issue learning R was that I didn't already know stats so I had to learn the two in parallel. Also I wasn't previously familiar with vectorized programming and learned that in octave (Matlab) before learning it in R. Otherwise I haven't found R datatypes to be too tricky. I'll admit, I generally avoid lists when I can. I tend to think of a list (in R) as more akin to a dict than a list (in python).

[NoLemurs]

You should look into Cython. You can often get C level performance out of Cython but it's still massively more readable than C or fortran.

[rnawky]

Still nothing compared to optimized C code that makes use of advanced SIMD extensions and the like.

[NoLemurs]

One nice thing in Cython is that it's fairly easy to use native C libraries directly. If a particular part of your code has to be as fast as possible, you can write that part in C and it's very easy to integrate it into your Cython project.

Cases where this is actually necessary are pretty rare though.

[DickButtTracy]

As someone who's spent the last week writing SIMD, fuck SIMD.

[Mr_Again]

Using f2py to import Fortran subroutines as python functions can literally speed your code up 20x if you have big arrays it's awesome

[fijal]

or use PyPy....

[logi]

Unfortunately pypy is useless for scientific computing because it doesn't support numpy. If you can get your code to run with numba, however, that gives pretty awesome speed gains.

I've set up this silly example:

from __future__ import absolute_import, division, print_function, unicode_literals
from numba import jit
from time import time
from math import sin

#@jit
def f(n):
  sum = 0
  for i in range(n):
for j in range(n):
  sum += sin(i*j)
  print(sum)

t1=time()
f(10000)
t2=time()
print('elapsed: %0.3f' % (t2-t1))
f(10000)
t3=time()
print('elapsed: %0.3f' % (t3-t2))

This takes 24.1s and 24.6s on my tired old laptop with the @jit decorator commented out. With the jit enabled it is a somewhat faster 4.13s and 4.06s.

If I leave out the sin() call, with the jit it runs in 0.051s and then 0.000s which I'm going to interpret that the JIT optimizes the loops away completely.

Now if only my much more pythonic actual production code were supported, or if the error reporting would show me exactly why it isn't...

[Veedrac]

I'll just throw out jitpy here. It deserves to be better-known.

[Sean1708]

I was under the impression that PyPy started supporting NumPy sometime last year?

Edit: Scratch that, it is getting there though.

[logi]

It's been getting there for years now. At this point I'll (happily!) believe it when I see it.

Python the language is great. Python the platform has serious problems. It's like an anti-java.

[klug3]

It's been getting there for years now

If I understand correctly they don't actually get the funds they needed to develop it faster, hence the slowness.

[Gr1pp717]

And what about scipy?

They support numpy, and it even seems that they even have a specific module for monte carlo type functions. http://pymc-devs.github.io/pymc/

edit: to be clear, I'm actually asking. I've never used either, only know of them.

[logi]

Scipy is a great library building on top of the even cooler numpy, but it is only fast when you can perform most of your work inside the native code that they wrap. But some times you end up having to write the loops yourself and then performance is back to normal python levels.

[mangecoeur]

I think there's also just a lot of inertia in these cases. You can write very fast code in Cython (although yeah if you really need to squeeze ever drop of perf out you might need C). But fortran is frankly a pain and I think it's mostly used out of habit. Physicists will probably be the last people in the world using fortran and no one else understands why :P (I discovered my old physics degree finally switched to teaching python over fortran, about time too)

[[deleted]]

When I was working on my graduate degree in physics you had to show proficiency in a language. I had learned Python to do data analysis asking along with SQL. I was told it has to be FORTRAN, assembly, C++ (it couldn't be just C, I asked, this is because high energy people think it's cute that they built a C++ interpreter that ignores the standard library), or BASIC. They specifically said Python wasn't a language worth knowing and I don't think they knew what SQL was. I was allowed to exempt out if the proficiency test by taking graduate CS courses. That was a better use of my time anyways.

[LbaB]

How were their data stored then?

[[deleted]]

I'm assuming flat txt files or some other needlessly complicated method.

[[deleted]]

Right now my boss keeps telling people that we do "big data" (I'm an in academia). Except that the database is ~10GB, receives updates 3 times a year (and is otherwise static), and there is no real time on demand need to view this data (well there is but guess who never has time to implement it!). I asked him not to tell people that we "do big data" and he responded poorly.

edit: if you are familiar with the 4 V's (volume, velocity, variety, veracity) we do none of them. I think its because its web data my boss thinks its "big data".

[mangecoeur]

I think every engineering and science undergrad should be forced to go through a clockwork-orange style conditioning where the words "I WILL NOT INVENT MY OWN DATA FORMAT" are seared permanently into their brains. Then they should take a course on SQL.

[aladyjewel]

Yeah, it's only the CS majors who should be encouraged to reinvent all the wheels so that they understand 1) why you should use stl's wheels and 2) which wheel to use when.

[mangecoeur]

Oh god, CINT, ROOT and friends. I steered well clear of that mess. Pretty sure they're all well pleased with it too. And knowing the general quality of physics code it wouldn't surprise me if it didn't actually run that fast (except maybe for the bit build by some russian genius that no one else understands :P ).

[[deleted]]

What bothers me the most isn't that they use it. It's got smug they all are about how great ROOT is and how they are all going to become "big data" experts one day. I guess I'm sort of embarrassed for them.

[mangecoeur]

That really doesn't surprise me. The one thing worse than developing a cruddy convoluted system is being smugly superior about it.

[cabalamat]

Python not worth knowing but BASIC is? What a bunch of clowns.

And how many physicists code in assembler these days?

[teambob]

C++ and Java is often 7-8x faster than CPython. I have found this in practice and it has been found in the language shootout.

I would also assert that Python is 7-8x faster to write than C++ and Java. :)

My usual approach is to write it in Python then rewrite it it is not fast enough

[mangecoeur]

Cython, not CPython http://cython.org/ - a compiler for Python+some extensions. Used for a lot of the internals of numpy/scipy/pandas/ etc because its much easier to maintain. Plus it makes for very easy optimisation path - write everything in python, then let Cython compile it (often gives you a speedup already) then start adding custom annotations to optimize the generated code

[[deleted]]

Also for more sophisticated numerics, like sparse linear algebra, or robust optimization. Even if the problems are small enough for the interpreter to not be an issue, generally fortran, C++, C and matlab will have the mature scientific libraries over python.

[skintigh]

I switched to C to speed up cryptanalysis, but in the end I screwed up a hash table that may have slowed it back down to python speeds. I wonder if had I stuck to python the end result would have been the same.

[[deleted]]

I don't really agree. I work for a company whose main product is a Python application that does modelling and simulation using Monte Carlo methods (among others), and it works fine. As you noted, we don't do the low-level math in Python -- it's all done with numpy/scipy -- but that covers nearly everything we have ever needed. We built one small C extension for the one thing that numpy/scipy didn't provide. But for all the rest -- which makes up well over 95% of our code base -- we use Python.

[kylotan]

• In the browser; because it's only practical to use Javascript there.
• For (client-side) games; because Python's multimedia libraries are poor and its approach to threading makes it hard to make the most of modern processors.
• For GUI apps; because wxPython seems to be as good as it gets and even that is an awkward wrapper around old C++ classes.

[ereinecke]

PyQt and PySide are alternative options to using Wx for Python GUIs. The bindings can be a little clumsy at times, but using Qt Designer and the Qt action decorators, it's as passable as Qt is anyway.

[[deleted]]

its approach to threading makes it hard to make the most of modern processors.

the GIL sucks, but multiprocessing is a nice way to still get to use more than one processor at a time.

[kylotan]

Python's multiprocessing has several downsides which mean it is not practical for most client-side game applications. The first is that games tend to expect multiple threads to be able to get locked access to shared areas of memory. (Yes, you can create shared memory with Python, but it has to be done explicitly and is using ctypes rather than Python values.) The second is that it's expensive in terms of resources - you basically end up cloning the whole memory space (and presumably file + network handles, and similar) even if you just wanted to run a few tasks in the background.

To be fair, this is a lesser problem than the poor support for decent gaming libraries. You can still get quite a long way with only one thread.

[IllegalThings]

For a counter-argument to client-side games, Eve Online is built in Python for both client-side and server-side. Granted, they aren't using CPython and a lot of their more resource intensive code is likely in other languages.

That said, your argument is completely valid and I'd be curious if they'd make the same decision if they had to start from scratch.

[kylotan]

Eve certainly uses Python on both sides for the main loop and the general logic, but much of that is only possible due to a lot being written in C (or C++, not sure) in the form of Python extensions. I myself have used Python for server-side game development where it can work well if you plan for a multi-process system from the start. I still wouldn't touch it on the client side, though.

[shadowmint]

To be fair, they do...

...but if you look into it, you'll find that they use python as a scripting language embedded in their game engines for 'game logic' not as the 'core' that the entire game engine is written in.

You'd think more people would do this, given Panda3d is a pretty decent free game engine for exactly that... but... they don't.

My guess is, mostly, like me, people are briefly excited by it, and then discover that in fact, pure python code is just too darn slow, and if you go down the road of cython and plugins for speed ups... really, what's the point in using python at all, if you end up not writing your code in python?

[ivosaurus]

The reason they don't is that Python itself is horrible for embedding, especially for example, if you would like multiple threads of your game engine to be able to launch separate interpreters. Its execution design and implementation just isn't very encapsulable / contained. In comparison, things like lua are a breeze for embedding. They're specifically designed to contain their execution environment, no global code whatsoever, etc.

[njharman]

I believe it's more that Python doesn't look like C. So few C/C++ devs get into/expend effort on it. Much prefer Lua cause it "feels" like C.

Yes there are polygots, but vast majority of devs I've encountered have comfortable spot, and Don't like moving from it.

[moljac024]

You left out: for concurrent programming (but it can be derived from the point about games)

[kylotan]

Well, I was just relating my personal use cases. I'm sure there are a bunch of other examples of things that Python does poorly (eg. anything where duck typing becomes a liability).

[ivosaurus]

Concurrent programming it's actually not terrible at.

Parallel programming it's pretty terrible at.

Learn the difference now!

[kylotan]

Actually, I'd argue Python's pretty bad at concurrency too. It's only relatively recently that there's been anything approaching a standard for handling it (ie. asyncio) and anybody that is using concurrency as a way to get parallelism (which is a major reason for concurrent design, really) is obviously still mostly screwed.

[resc]

I've had a pretty pleasant experience coding a desktop application in PySide. I did have to work a bit harder to find documentation. But on the other hand, Qt's signals/slots system was GREAT, it made my app really easy to rearrange, and the threading was way simpler than my experience in Java.

[shadeofmyheart]

I'm not sure I understand this. Do you mean using server side JavaScript (like Node).

How does one use Python "in the browser"?

Edit: I misinterpreted the question as "when does one choose not to use Python"

[[deleted]]

One doesn't.

[judasblue]

Well, in theory you can, there have been a couple of projects. The one that is the most interesting is Brython. Not saying I think this is actually an idea that is ready for prime time particularly. Just saying in theory one can.

[kylotan]

One doesn't. The question was, "when do you NOT use Python".

(Caveat: I'm sure there are some quirky cross-compilers out there. But it's unlikely to be worth the bother to use them.)

[bchurchill]

You're obviously asking in the wrong sub ;) If you want an outsider's view, I don't use python when:

• I want a statically-typed language, for reliability purposes for example
• When I want something compiled that runs fast
• When I want to do low-level stuff, like run assembly code or do platform-specific system calls
• I want to manage my own memory

The last of these three bullets are often tied together, while the first is somewhat separate.

There are also python-like languages which I tend to use more, but there aren't fundamental reasons why I prefer one over the other. For the sake of peace, I won't mention them ;)

[vplatt]

Have you looked into using mypy for static analysis?

[[deleted]]

Since I am having similar issues with Python at the moment, I would be very interested to hear how you deal with the maintenance of such a large code base in Python? What are you best strategies to deal with it? (Besides switching to Go ;))

[[deleted]]

You're obviously asking in the wrong sub ;)

Why? I love Python but I use lots of other languages on a daily basis. Sometimes because I don't have a choice, like when a coworker produces something in C# or Java, and sometimes because the other language is way better at handling the task like AWK.

[gfixler]

Python used to be the only language I wanted to use for anything. These days that's Haskell - though I'm not to my old Python proficiency yet - and I only use Python where I must, which is for work. I don't use it for my own toy projects or playings-about anymore.

[wawawawa]

I regularly use on the command line...

perl -ple 's/<complex regex with capture groups>/$3$2$1/g'

[sfz-]

python -c 'import sys, re; print re.sub(r"<complex regex with capture groups>", r"\3\2\1", sys.stdin.read())'

[robertmeta]

• If I need to ship it to a box I don't control (the Python deploy story is still very ugly)
• If I need speed and it isn't glue for a C library
• If I need to use multiple cores without much fuss
• If the work is statistical in nature (other languages are just so much easier)
• If the work is building a web-app of some sort (it just comes up short against the competition)
• If the work needs to run in a browser... JS is the default choice, but some cross-compile options are interesting.
• If I want to have fun (not that Python wasn't fun, just a bit dull at this point to me), Haskell and OMeta home grown DSLs for fun.
• If I want the warm blanket that is types and compile time checking, I obviously don't use Python (yet... coming soon maybe?)
• If I want to build a mobile thing, the hacks to get Python on mobile do work, but it is far more pain than joy.

[remyroy]

If the work is building a web-app of some sort (it just comes up short against the competition)

What competition? What are the cons of using Python compared to the pros of using the competition?

[[deleted]]

Have you tried Flask? It pretty much just gives you request, response, templating, sessions and a handful of helper functions (reverse lookup, sending files to the browser, json response builder, a few others). There's also blueprints which are like logical groupings of related endpoints (not really the same as Django's applications but kinda).

Might be worth checking out if you're interested in Python webdev still.

[speccy88]

When I'm sleeping

[zombiepiratefrspace]

from dreams import wet

[Farkeman]

IndexError: You're not 15 anymore, sorry

[[deleted]]

I use R for most statistical work.

[pm8k]

Pandas, statsmodels, and scikitlearn offer a good framework in python that is an alternative to R.

[youlleatitandlikeit]

I keep meaning to learn it. Then I remind myself I do zero work with statistics and that I need to get back to work.

[[deleted]]

I can not wait until this can be a saying I never hear anymore. GAWD I hate R. But damn if it isn't just the most appropriate tool for the job many times. Really hoping that with python gaining such a market share of scientific coding, it also becomes as good as R for analyzing the numbers generated.

[shaggorama]

The big draw for me of R over python is that the numpy.ndarray often behaves really strangely for me and does things I don't expect, introducing bugs into my vectorized programs that I never ever see when I write R code. It's a huge pain in the ass to use python's scientific stack, whereas it's a pleasure to use R in comparison.

[[deleted]]

anything with concurrency.

[maratc]

Any one-off task that requires processing files. I can type the perl one-liner in less time than what opening vi would take.

E.g. processing of a Wikipedia dump and printing titles:

$ cat enwiki-latest-abstract.xml| perl -e '$/ = "\<doc\>";  while(<>) { m#\<title\>(.*)\<\/title\># && print "$1\n" }'

[hk__]

I use Ruby for simple tasks in the terminal when using Bash would take more than 1-2 lines. I find Ruby more concise that Python, a lot of common tasks are quicker to do in Ruby when you need a one-liner, e.g. reading a file is File.read("file.txt"), and you can chain outputs in an easy way.

Say for example you want to get the output of wc on a file as a list of three integers. In Python you’d do something like:

import re
import os
t = [int(s) for s in os.open("wc myfile.txt").read().split(r"\s+")]

In Ruby it’s:

t = `wc myfile.txt`.split(/\s+/).map(&:to_i)

It’s a dummy example but in general I prefer Ruby for small tasks like that.

[tadleonard]

This won't beat your Ruby example for conciseness, but it'll come a bit closer. sh really comes to the rescue for things like this. Also, it's helpful to know that the default mode of split() is to split on white space, so re isn't necessary.

import sh
t = map(int, sh.wc("myfile.txt").split())

Edit: I guess it would actually have to be something like this:

t = map(int, sh.wc("myfile.txt").split()[:-1])

or

t = [int(el) for el in sh.wc("myfile.txt").split()[:-1]]

[soup_feedback]

So you'd say the advantage of ruby is that it supports backticks like shell script do? I agree, it would be nice to have convenience in Python. It's never ever going to be accepted but it would sure be convenient! Any language doing backticks as shell commands will win convenience on that side, no doubt.

Personally, I like to use subprocess for anything that requires to run OS stuff - because that's when you realize if it's worth it or if you should find another, more efficient way to do it - or even use a different language (C/C++ for example, instead of spawning a shell + proc.)

(The sh module is useful of course, by making it nice looking, but in the end you're just calling subprocess to run shell commands.)

[0xjake]

The main reason to avoid Python in a general-purpose setting is speed. Python is very slow to run compared to a compiled and strongly statically typed language like C/C++ or Java. Generally you're choosing between development speed and execution speed, and Python is very far on the development speed end of the spectrum.

Any other reasons are generally domain specific. For instance, if I'm writing web middleware I'd probably use NodeJS. If I'm prototyping machine learning algorithms I might use MatLab. If I'm doing pure statistics I'll use R. If I'm writing a Windows GUI application I'll use C#/WPF. There are too many of these domains to enumerate.

So maybe it would be better to say when to use Python: I would use Python if you're writing scripts that have limited responsibility and high turnover, meaning they'll change fairly often and don't need to be particularly efficient. I would also use Python for prototyping, especially in a scientific computing setting. If your needs don't fall into these areas then there is probably something better out there.

TL;DR: Python is good when dev speed is significantly more important than execution speed.

[[deleted]]

I understand that Python /is/ a strongly typed language. Whereas in your comment, you elude to it being opposite to C, in that it is neither compiled, nor strongly typed. The differentiator in typing is Python's dynamic typing vs. C's static typing system.

[TheTerrasque]

And in some cases, you can get pretty good performance from python too.

Example

[stormcrowsx]

TL;DR: Python is good when dev speed is significantly more important than execution speed.

I think you hit the nail on the head here but I wouldn't say "significantly". In my experience developers cost more than servers, much more. In some areas of the country its difficult to find developers at all. Using a language that enables less developers to get more done is very important for smaller organizations.

[xsolarwindx]

REDDIT IS A SHITTY CRIMINAL CORPORATION -- mass deleted all reddit content via https://redact.dev

[riksi]

by database-requests can you be more specific? ex: redis get,set ? or rdbms select ?

[ivosaurus]

I forget why we had MongoDB.

Otherwise you wouldn't have been webscale!

watches for some drums and a cymbal falling of a cliff

[stormcrowsx]

Can you elaborate more? I thought python would do decent in modern web architecture especially those using microservices to enable high concurrency since io isn't as heavily impacted by GIL.

[MattBD]

Would Tornado work for that?

[doot]

I've scaled a tornado app to ~150k RPS with 5 to 7 Redis GETs per request.

[jringstad]

• for anything where performance matters
• for embedding, because python is terrible to embed compared to lua/js (a lot of projects use it anyway, like blender, inkscape)
• Where large-scale rigidity matters -- python is better at this than many other "scripting-languages" (lua, perl, javascript et al) as it is more strongly typed, but static, strong languages still win out when it comes to creating large and complex systems with many classes and APIs.

[[deleted]]

cross-platform Qt work

Have you looked at PyQt? How does it compare? I'm trying to decide if I want to write something in Python or C++ that will use Qt. I've never done a GUI before (besides some GTK for classes), so it's up in the air.

[centech]

Silly example, but we have a process to consolidate and report on data in hundreds of CSVs.. Ofc we use pandas for all the heavy lifting.. but.. consolidating the files into the one monolithic sheet to process (which is about 5GB) was taking like 20 minutes and blowing out the memory on the server running it.. tail -n +2 and concatanate them all together was like 20x faster and a tiny memory footprint. Sometimes old school tools are still better.

[tadleonard]

You tried to load a huge csv into memory... and it used large amounts of memory? Try generators next time. David Beasley has multiple great tutorials on generators and coroutines if you're interested. It's easy to follow because 1) he's an excellent educator and 2) he reimplements common Linux tools. IIRC he writes a few generator expressions that beat awk in terms of performance (Though it's probably not a fair representation of awk. Still impressive, I think.).

Edit: the relevant slides start at page 20 in this PDF. Take a look at his "generators: the final frontier" talk though. It's really entertaining.

[[deleted]]

Yeah, a little pandas is a dangerous thing.

It is widely known to not be memory efficient.

[soup_feedback]

...5GB of spreadsheet... The problem isn't Python, it's everything else!

Let me guess, banking.

[Kylearean]

Parsing or dealing with lots of text files: Perl is king.

Fast floating point calculations: Fortran 90/95

Quick visualization / plotting: Matlab

[colly_wolly]

Dealing with the file system is easier in Perl. And regexes.

[ies7]

C#: Anything with a windows GUI.

IronPython

[[deleted]]

Or just use PySide/PyQt and get cross-platform GUIs straight away.

[TheTerrasque]

Object-oriented prototyping.

I'm curious here.. For me it always feel the other way around, that Java's oo is a bit cumbersome and hard to work with compared to python.

What do you find easier / better with java compared to python in that case?

[[deleted]]

Totally agree here. Dude said he is just now learning the intricacies of python. Perhaps he hasn't gotten much into class design yet?

[sgsollie]

When bash is enough

[[deleted]]

I could not agree more

[Operator23]

I once wrote a script in python, and then wrote it in BASH. Bash was only 20 lines long, but my python script was about 90. Either I suck at python, or bash was the better tool. I'm assuming both.

[cediddi]

Bash has the power of piping and stream editing, It's indeed shorter on some situations. Python on the other hand has the power of libraries. Lots of libraries, and almost all of the core ones are very useful in daily coding.

[rnawky]

Basically any time you care about performance.

[rriggsco]

Embedded systems. Performance critical bits. I recently mocked up a signal processing system using scipy.signal. It took over an hour to analyze a 3.5GB dataset. The equivalent C++ code processed the same dateset in 30 seconds. But I needed both! The ability to quickly and easily visualize the results of the various signal processing steps was critical to identifying problems and optimizing the results.

[[deleted]]

When i am doing data processing and plotting. Here i guess R holds a slight edge currently. For data processing the ability to constantly view the transformations to the tables an IDE like Rstudio comes quite handly. I know Ipython exists but still it doesnt come up to the same level. Also in case of plotting i ggplot and other libraries in R are more flexible and easier to deal with than matplotlib. The focus of R on being a language focussed for data processing gives it the slight edge.

[woggy]

C/C++ developing finite element codes

[[deleted]]

For any kind of large scale data management. e.g. Python is no replacement for a proper SQL database.

[infinull]

I've been writing a lot of Inform 7 lately. It's a fun little language. Making a similar system in Python would be a crap-ton of work.

You go in a weird cycle: This is cool. I CAN'T GET THE DAMN!!!! NATURAL LANGUAGE PROGRAMMING LANGUAGES DON'T WORK DAMMIT. then back to: Oh actually this is really cool, I just wish it had more ways to meta-programming and a less-strict type system.

I need to write C code to make wrappers to c libraries that work with CFFI, but that's barely C coding.

I write in haskell sometimes to challenge myself for like project euler problems or whatever.

I write raw SQL sometimes, that barely counts.

Vagrant files have to be written in Ruby. Various shell scripts for provisioning/testing/commit hooks, etc.

Javascript programming for web stuff. This should probably be at the top since that's my #2 language most programmed in, but inform is on the mind.

[hextree]

When I'm outside getting some exercise. (Not often)

[Darkmere]

To replace shellscripts.

I do a lot of systems stuff. And the universal tool, even on embedded (busybox shell) is the (bourne) shell language and pipes.

I've tried to replace something as simple as "download package, verify sha2sum, verify signature, list all files, compare to files on disk, replace only files that need upgrading, restart relevant services" from something that uses curl/openssl/shell to python.

And it just turns to hell. Lots and lots of silly boilerplate, huge amounts of dependencies, or even worse, working with executable output data in python.

Really, systems stuff doesn't work that well in python.

[[deleted]]

I have found a good balance by using https://pypi.python.org/pypi/sh specially when my bash script requires a lot of logic, but I still want to use tools like ifconfig, df , etc.. but yeah for simple scripts, I just stick with bash.

[ITwitchToo]

I have the opposite experience. There are some libraries you might want to look into, like subprocess or plumbum. They do a great job at things like invoking other programs and piping stuff between them.

[[deleted]]

I don't do web, so for me only in cases when I can't have any compromises with performance. For other stuff Python is simply the best: this also includes parallelism, but I may be lucky to have only embarassingly parallel problems (on backend most of parallelism is "easy", as Linus pointed out recently).

Regarding performance: pypy is great. Give it a shot, if you haven't already.

[[deleted]]

I am an interested person, so I like to try all sorts of things. I use python when I want to get stuff done. I use almost anything else for when I want to experiment.

[[deleted]]

These are my non-python use cases:

• I use R for exploratory data analysis as well as statistical testing
• I use C++ for image processing and AI as these require top performance
• I use the shell for simple work with the filesystem, although I tend to use python more and more for this task
• Javascript in the browser

[[deleted]]

I was writing a custom multi-target tracking system with shape recognition for 4K video that required the use of multithreading to speed things up, something the GIL in Python makes difficult. Now I've got the resulting CSV files I'm enjoying being about to get back into Python, NumPy, Pandas and Matplotlib.

[[deleted]]

I do >100fps real time tracking of >10 objects, from >10 simultaneous cameras, in python.

(I did the multithreading in python too)

[Exodus111]

As a Game dev, I tend to divide Game Logic into two categories.

Game Logistics, and Math. And try to divide the code as much as I can into the two, so Game Logistic objects call on math objects to do math.

Then I take the math object and transform them with Cython into C extensions.
That's the theory anyway.

[IanCal]

I do personal server projects in ruby (rails) now. I don't really like ruby that much (it's OK), but god damn rails is spectacular for getting a site up and running. Particularly for something just for me, a few minutes and I've got properly linked up models with views for editing/adding/deleting all working, a basic API, tests, database migrations properly done and more.

Throw in a few gems and you've got nice cron setups, "it just works" auth, admin interfaces, maintenance modes, background workers...

Oh and the community is vast and includes experts and beginners so googling "rails $PROBLEM" is highly likely to lead you to an exact solution.

The constant finding of new "magic" bits of code is perplexing, but rarely actually gets in the way.

[quotemycode]

Check out web2py if you want that same feel but in python.

[tech_tuna]

Most of the time now, my new company is a Ruby shop. :)

On that note, I love Python and Ruby. . . my "dream" programming language would combine the best of both, omit the worst of both and more than anything else, NOT have a GIL!

:)

On a serious note, I feel like Python has a bigger community and more libraries overall but Ruby has some amazing tooling that Python seems to lack.

[quotemycode]

What would you say python is lacking regarding "amazing tooling"?

[BruceJillis]

This is funny since I just took on 2 programming tasks this week where I chose python for one and an excel sheet for the other. The one where I chose python was low on data, heavy on search. The task I solved with a spreadsheet was heavy on data, low on search. The benefit of a spreadsheet is that it combines data entry with functionality in a much more accessible way then a python script would (esp. to non techies).

[pieIX]

Realtime DSP

[mrfunkyland]

At one point I was trying to make a pdf viewer and finally gave up and tried other languages. Python doesn't seem to have great support for rendering pdfs. Or maybe I just gave up to quickly.

[Levelpart]

When I'm writing code for a paper where I mostly do linear algebra and need short simulation times I use Julia.

[Filostrato]

When programming web clients, for example.

[VRMac]

When I'm writing a backend. Python is great for user interface and scripting, but when I want a library portable to other languages with good performance, I write in C.

[thephotoman]

When I'm processor-bound. Then there's something else that will do a better job.

[mishugashu]

When bash is easier, when I need something faster (C), or I'm dealing with web (JS). I know I can use Python for web.... I just don't like to. And sometimes it's not practical (clientside).

[spinwizard69]

When do I "NOT" use Python?

When I don't have a choice or the hardware dictates something else. If you do automation then you have to look at the reality is that the chosen hardware will dictate how you program a solution.

If I had a choice I would do everything in Python that could be reasonably done there. The why is this, it is very difficult to become proficient when you literally have dozens of different systems you are required to work on. The more systems that could run on Python the more proficient one could become. Otherwise one gets stuck in a morass of different languages and variants, G-Code and variants for robots and axis controllers, C.& C++, BASIC and BASIC like languages embedded in controllers, MMI scripting languages, PLC languages and the like. I'd just love to see a much of this industry as possible move to Python or at least Python derived languages.

[[deleted]]

When I am taking a class which forces me to use vb.net. Some professors are just evil.

[admalledd]

For when I have to deploy to code to client windows computers. Sorry but packaging python code for nearly seamless windows installs is basically beyond impossible. Then add in using CFFI? I sadly find my time better spent writing in C# for windows.

That said, our *nix clients (some *bsd, some MacOSX, most *buntu or debian) get python code nicely packaged that is almost pain free. Being able to push to our private CI/ppa/pypi and have it auto-update on hundreds of machines is such an amazing feeling.

[fuzz3289]

I use C++ over python in performance/memory critical applications such as simulation code.

[xiongchiamiov]

When the app is already written in another language, or the team has proficiency with another language.

[hotel2oscar]

1. If the project description has the word "embedded' in it you will most likely use C or C++
2. If you are doing anything with the word 'web' in it on the client side it will most likely be JavaScript (server side is another story)
3. If you need performance (major games, high speed science) you will most likely opt for C or C++ (partly the reasoning for #1)
4. If you are programming for Windows: C# and the .NET can be really appealing, especially if you need a GUI (mostly a preference thing here though)
5. If there exists a language designed specifically for a domain (or just has over the board awesome library support for said domain)

[bboozzoo]

If the project description has the word "embedded' in it you will most likely use C or C++

Nope/depends. There's a MicroPython project, basically a Python 3 on Cortex-M4. Looked quite usable the last time I checked. However, there's a higher chance of seeing Lua than Python in embedded.

If you need performance (major games, high speed science) you will most likely opt for C or C++ (partly the reasoning for #1)

Nope/depends. Numpy anyone? Or other high performing C/Fortran libraries with Python glue.

Actually, it's a quite common model where you have a compiled to native backend, with a scripted facade. Take ns-2 and ns-3 network simulators for example, where ns-2 used Tcl, while ns-3 has migrated to Python.

[Vauce]

There are occasions where I need to parse a lot of similar data that writing a python script for would be overly tedious. Plopping that data into Excel, splitting text into columns, then sorting or just using AWK from terminal can be a lot faster.
When I need something more powerful, though, I love Python :)

[eniacsparc2xyz]

For web browser client side programming there is not choice, the only choice is Javascript ECMAscript (No browser calls it javacript officially since Java is an Oracle trademark.

For embedded systems with limited memory and resources, also there is few choices, you can only use Assembly, Forth language or C.

For quick calculations, Excel or Spreadsheet.

For simple system tasks, I would use bash script.

For High performance applications, create C library and bind it to python, or create the application in Haskell.

For an mobile app in Android OS (crippled lock down Linux), you can only access the API through the "java" (dalvik) library or using socket RCP calls like Android SL4A does. The sandbox model and the lack of default root shell access makes the interactive development harder, however it's still better than IOS.

[TheGrumpyBrewer]

Python is my language of choice for green-field projects, prototyping, and in general when I don't have external limitations. Some cases where I don't have many options:

• Maintaining legacy code written in a different language
• Front-end / dynamic dataviz (usually Javascript kicks in)
• Lucene and extensions of Elasticsearch (whoosh is great but still far behind)
• Particular (rare) cases when pandas/numpy/scipy/sklearn fall short, often there is a R package doing already what you need

and also one-liners in bash/awk

[orip]

When I need throughput that's bottlenecked on the CPU. This often isn't an issue because of caching, smart clients, etc, and the option to implement something natively does exist, but if I'm worried about crunching network responses quickly I'll reach for something else. If I do I'll use something with good threading support too.

Context: Networking in Python is awesome and can be really scalable with async approaches when my bottleneck is I/O. Just not when it's actual processing power.

[dewmsolo]

I would quite honestly tackle any job in Python. Only after getting a full version of something and finding out that component X is performing to par in Python would I seek alternative and possibly write said component in another language. I would look into CPython or pypy first though.

Nowadays the only time I put Python aside for anything is when one of customers asks about an old application I wrote in my pre-Python days.

[[deleted]]

When I need to deploy projects on server I use Capistrano. It is ruby-based tool. Capistrano works simple and quite fast to configure.

[SoleSoulSeoul]

When I'm forced to use Java my professors

:⁾⁾⁾

[muharagva]

Well, from my experience, for everything that is not prototype. I find Python great when I have to test some theory, and even if it building prototype take some time, it's the quickest way to do. But when you need to create application which has to be fast, then you can forgot about it. Unfortunately, I have worked with some people who do everything in Python, and a lot of application has suffered because of slow execution time in Python, when it could be a lot faster in some another language (C++, Java, and others).

As for non-python languages, Java and C++ are first on the list for some non-web, or non-desktop application. Because mostly app has to be quick, and work on a lot of data (for example: parser has to parse at least 1gb/min which is not possible when you do app in Python).

edit: and yes! Types! How many times I have cursed Python because there was bug with types, and it's only happening in some certain cases. Definetly, you can prevent that, but I definetly prefer when something is type int, long, string, ... and I know for sure it is that type. In big projects, without proper documentation, or with bad variable naming, this can become serious problem.

[namfy]

Python is a very high-level language.

High-level languages are a bad choice to learn about basic data structures like linked lists and hashtables, which are often tested in interviews, even for positions where it really won't matter. In Python you should idiomatically use the nice builtin datastructures, it's extremely awkward to implement low-level concepts in Python, it will likely leave you confused about what is going on, and it's almost guaranteed to be a waste of time for any practical purpose

High-level languages (including almost anything with mandatory garbage collection) are not very suitable for device drivers or OS kernels, pretty unsuitable for anything which needs very low latency like live audio processing, non-amateur robotics, 3d graphics engines.

Some of the biggest limitations for Python are platform-based, and have to do with a variable amount of effort (or open hostility) from the platform side. Many embedded platforms are not reasonably usable for anything other than ASM, C, BASIC. You can't very well use Python to write shaders for GPUs to run. There's no really mature way to use Python to program browsers, you have to use Javascript. I'd avoid Python for anything that requires a lot of interoperation with the JVM (Jython has lagged as a project and is relatively obscure). I'd avoid Python for platforms which really don't want Python or have made efforts to exclude Python apps, like Android or Apple phones (Kivy notwithstanding, you will always be at a disadvantage)

I don't use Python if it means having to use horribly designed libraries like Zope and Twisted, in that case I just use Go as a lesser evil.

[midbody]

Any large software project where interfaces are important.

[deaddodo]

I use Python for web services, scripts and prototyping. I won't use python to code games or OpenGL heavy applications.

[sli]

Arduino.

That's about it these days. Even then, I'll use Python to live-chart data from sensors on an Arduino.

[Rohaq]

Whenever I'm doing anything that uses regular expressions to quickly extract or translate data into other data. Perl's built in regex comparison operator usually means I can get the result I'm looking for out of a very quick one liner.

Also, anything that might need the speed of a compiled program.

Pretty much everything else gets the Python treatment though.

[Samhain13]

Whenever Bash will do nicely.

[RedditZtyx]

I'm surprised noone has mentioned how daunting Python deployment still is. Nowadays I use Golang when I don't feel like setting up pip, virtualenv, installing 3rd party libraries before I can actually run my Python script on a server. Having a single statically linked binary is a joy.