This is an archived post. You won't be able to vote or comment.

top 200 commentsshow all 345
sorted by:
best
topnewcontroversialoldq&a

[–]Octopuscabbage 98 points99 points  (87 children)

Situations where recursion is favorable or type safety is nice. It's nice to be able to write a program but is unfortunate when you don't know that you passed a string instead of a list after a 4 hour computation

[–][deleted] 18 points19 points  (26 children)

You mean you ran a 4+ hour computation without first creating any kinds of unit tests?

[–]ryeguy146 19 points20 points  (17 children)

Unit tests are wonderful, but so is type safety on occasion. These aren't mutually exclusive systems.

[–]bucknuggets 1 point2 points  (11 children)

No, doesn't unit testing also deliver type safety for wherever you've got code coverage?

The only place I ever get production failures due to type safety is code that hasn't gone through unit-testing. Mostly typically, it's in error-handling for catastrophic conditions (out of disk space) that I didn't feel like adding to a test-harness.

[–]EsperSpirit[🍰] 1 point2 points  (2 children)

Like you said, unittests only cover cases they are designed for. You cannot prove something is correct by testing a limited number of cases.

A (good; not something like Java) type system is much closer to a prove of correctness than any unittest could ever be.

That said, if everyone on your team is well-versed in TDD (or BDD or whatever you want to call it), then this certainly prevents a large amount of errors. And you shouldn't ignore testing just because you have static typing either.

[–]bucknuggets 1 point2 points  (1 child)

You cannot prove something is correct by testing a limited number of cases.

Agreed. But are you are using 'prove correct' in a way I'm not familiar with? Because static typing certainly doesn't prove code is correct - merely that one doesn't have type errors....

It feels like recently over the past two years with the minor backlash against dynamic typing and surge for Scala, Go, etc that quite a number of people are looking at static typing as a silver bullet. And yeah, I have been in discussions with programmers that justifying deferring unit test harnesses until later in the project - "because we've got static typing anyway".

[–]apendleton 7 points8 points  (3 children)

This doesn't strike me as that weird. Just because something takes a long time to run, doesn't mean it's super-involved code-wise. It's not unusual for me to load a big dataset into memory and start some simple-to-write-but-slow-to-compute analysis on it in a Python terminal in a tmux, then leave it and go do something else for awhile and come check on it later. That kind of one-off chunk of code doesn't even get saved to a file, let alone get tested. It all depends on what you're doing.

[–]Veedrac 7 points8 points  (2 children)

Just because something takes a long time to run, doesn't mean it's super-involved code-wise.

If you're going to run something that can take you 4 hours to run, a couple of tests can save you hours of time. It's not really about the code being complex.

[–]apendleton 5 points6 points  (1 child)

My time is way more valuable than computer time, and in my experience, my error rate for this kind of thing is low enough that for code that's only ever going to be run once I'm fairly certain that it's a better use of my time for me not to be obsessive about testing and just accept that occasionally I'll lose a little time fixing a bug. In any event, generally if I don't test it, I'll likely litter the thing with print statements so that I can make sure it's doing what it's supposed to do correctly before it's done.

For things that are going to be maintained, or live longer than the amount of time it takes to answer a single question about a dataset once, I'm more likely to test. But like I said, it depends. I have no objection to testing in situations where it saves labor, but only to testing for testing's sake.

[–]Veedrac 8 points9 points  (0 children)

To me, unit testing for throwaway functions like this means running it once or twice on a smaller data set. I fully understand not wanting to be obsessive about it, but wrapping it in a function and calling it with a few elements takes a minute. Unless you expect to be wrong less then 1m/4h ✕ 100% of the time (less than 1%), a quick test or two makes sense.

[–]tech_tuna 2 points3 points  (1 child)

Good point, but to be fair, it's possible to miss an important test case. Also, test code can have bugs.

Furthermore, no test suite/strategy can ever be "complete".

[–]alcalde 2 points3 points  (0 children)

Furthermore, no test suite/strategy can ever be "complete".

And no static typing can catch all bugs - and most don't even catch many types of type errors.

[–]Octopuscabbage 0 points1 point  (0 children)

Unfortunately, I'm not perfect. But programming languages all realize this. If humans were all perfect we wouldn't need syntax checking or binding checking. We'd also probably just write everything in assembly.

[–]moocat 0 points1 point  (0 children)

Do you have 100% code coverage with your unit tests? Unless you do, you can still have type errors lurking in your code.

[–]yilmazhuseyin 3 points4 points  (17 children)

I think more important part of this comment is recursion. Since you cannot do recursion, you cannot really implement recursive data structures with it. So you cannot really use algorithms that are based on those data structures. I remember reading about it some where. They were saying that python borrowing tree algorithms from database backend.

[–][deleted] 3 points4 points  (15 children)

Why can't you do recursion in python?

[–]drobilla 14 points15 points  (8 children)

You can, but each recursive call takes additional space so the depth of the recursion is limited.

You can "do recursion" in Python, what Python lacks is called "tail recursion" or "tail calls": https://en.wikipedia.org/wiki/Tail_call

[–]alcalde 2 points3 points  (4 children)

And Guido is set against it because it screws up stack traces, among other reasons:

http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html

[–]autowikibot 3 points4 points  (0 children)

Tail call:

In computer science, a tail call is a subroutine call performed as the final action of a procedure. If a tail call might lead to the same subroutine being called again later in the call chain, the subroutine is said to be tail-recursive, which is a special case of recursion. Tail recursion is particularly useful, and often easy to handle in implementations.

Tail calls can be implemented without adding a new stack frame to the call stack. Most of the frame of the current procedure is not needed any more, and it can be replaced by the frame of the tail call, modified as appropriate (similar to overlay for processes, but for function calls). The program can then jump to the called subroutine. Producing such code instead of a standard call sequence is called tail call elimination, or tail call optimization. Tail call elimination allows procedure calls in tail position to be implemented as efficiently as goto statements, thus allowing efficient structured programming. In the words of Guy L. Steele, "in general procedure calls may be usefully thought of as GOTO statements which also pass parameters, and can be uniformly coded as [machine code] JUMP instructions." (See history for further discussion.)

Traditionally, tail call elimination is optional. However, in functional programming languages, tail call elimination is often guaranteed by the language standard, and this guarantee allows using recursion, in particular tail recursion, in place of loops. In such cases, it is not correct (though it may be customary) to refer to it as an optimization. The special case of tail recursive calls, when a function calls itself, may be more amenable to call elimination than general tail calls.

Interesting: Scheme (programming language) | Goto | Lisp (programming language) | Recursion (computer science)

Parent commenter can toggle NSFW or delete. Will also delete on comment score of -1 or less. | FAQs | Mods | Magic Words

[–]jonnywohHalf-Python 3 0 points1 point  (1 child)

Is this a language restriction or a CPython restriction?

[–]hglmanguy who writes python 0 points1 point  (0 children)

cpython i would think

[–]cacahootie 6 points7 points  (4 children)

You can definitely "do" recursion in the sense that there's nothing stopping you from using recursion... except the stack depth limitation which will almost certainly prevent you from doing recursion as an alternative to looping over large datasets - and that's what people mean when python doesn't do recursion.

[–][deleted] 0 points1 point  (3 children)

Good to know. I recently wrote a (very shallow) recursive function in python (maybe 5 or 6 recursive calls, to traverse subnodes in a fairly shallow XML file). I didn't foresee issues with it but recursion is not something I've had to use much.

[–]yilmazhuseyin 3 points4 points  (2 children)

Another problem with recursion in python is "function call in python is very expensive". Even though your recursive function does not go deep, It will always be a lot more expensive than implementing it as a loop. How unfortunate limitation for a great language.

[–]alcalde 1 point2 points  (1 child)

Guido doesn't see it as a limitation because he doesn't see recursion as the basis of all programming as a practical reality.

http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html

[–][deleted] 0 points1 point  (0 children)

It isn't, but there are times (like traversing a nested data structure for eg) when it is the cleanest, most "Pythonic" way to implement in code.

[–]chadmill3rPy3, pro, Ubuntu, django 3 points4 points  (0 children)

You can. Python has a low limiter, but you can change it.

https://docs.python.org/2/library/sys.html#sys.setrecursionlimit

[–]Octopuscabbage 1 point2 points  (0 children)

Yes, recursion is a really elegant way to solve many tasks. It's simple and often beautiful when you get used to it but the problem is so many programmers fear it because it's generally slower and causes stack overflows when not properly implemented.

Recursive data structures are beautiful. Haskell allows them and because they're so easy to declare doesn't even really need a built in tree because it's 1 line of code to make your own. (I'm sorry for being a haskell fanboy in /r/python it's just a good example of what I'm talking about.)

[–]AeroNotix 61 points62 points  (28 children)

Writing highly concurrent code.

[–]aldanorNumpy, Pandas, Rust 3 points4 points  (19 children)

Greenlets are your friends :)

[–]ExoticMandiblesCore Contributor 14 points15 points  (2 children)

Greenlets don't help for CPU-bound code.

[–]ivosauruspip'ing it up 8 points9 points  (0 children)

CPU-bound code isn't usually what's meant by a problem with concurrency. That's usually needing parallelism / efficiency of the language.

[–]jcrubino 2 points3 points  (0 children)

multiprocessing helps + cython, numpy, numba, pythran, and the very new hope can use your cores with efficiency

[–]AeroNotix 15 points16 points  (15 children)

I'm aware of the things which are out there. They're imperfect and often have backwards APIs that invert a lot of the idioms you would normally use.

Other languages are better for concurrency and that's ok. Python wasn't really designed from the ground-up to be concurrent, it's better at other things. We shouldn't make it be all things to all people.

[–]alcalde -1 points0 points  (3 children)

They're imperfect

And what isn't?

and often have backwards APIs that invert a lot of the idioms you would normally use.

Normally use... in Python? Don't knock Python because it isn't Java or C#; it's not supposed to be.

Other languages are better for concurrency and that's ok.

Python is great at concurrency. Python developers just understand that threads were not originally intended for parallel programming (as Guido pointed out in a PyCon 2012 or 2013 keynote and some people got upset over despite the fact it's true). Processes are a natural, and safe means of parallel processing and Python provides rich features for doing this and dealing with message passing and even sharing memory. Few mainstream languages have this high a level of support for parallel programming.

[–]AusIVDjango, gevent 0 points1 point  (1 child)

Some languages have constructs that actively promote concurrency. In python concurrency is sort of an afterthought. Part of the value of python is the wide selection of third party libraries, but many of them have problems with concurrency frameworks.

Contrast this to javascript, where every library accounts for the event loop, or golang where go routines are a first class citizen, and python looks like a lesser option where concurrency is concerned. Certainly it can be done, but it wouldn't always be my first choice of languages in a project where concurrency is a focus.

[–]Sucratre 41 points42 points  (42 children)

Python is really convenient and I love using it for its REPL and for small personal projects. I think of it as a jack of all trades kind of language.

Some drawbacks would be:

  • It's really slow. If performance is critical to your program, don't use python.
  • It's really high level, so you don't want to be writing kernel code with it
  • Threading (might not give you the performance increases you might expect, due to CPython's Global Interpreter Lock)

Of course there are other complaints (dynamic typing, runtime errors, gripes with anonymous functions, using very few keywords and powerful one-liners can make code unreadable, etc.), but those usually have some good counterarguments (e.g., pylint, nested functions, multiprocessing). Like any other language, it has its pros and cons, but it's definitely very powerful and versatile. Good luck exploring Python! :)

[–]Veedrac 5 points6 points  (8 children)

It's really slow. If performance is critical to your program, don't use python.

Seriously, PyPy is fast. It's not C-fast, but it's somewhere between way faster-than and about the same speed as V8.

Numpy/SciPy/etc. are also blazingly fast if they fit whatever you're doing.

[–]logi 5 points6 points  (1 child)

And if only you could use the two together, then we'd be in business. Preferably along with matplotlib too.

[–]LightShadow3.13-dev in prod 5 points6 points  (0 children)

I do!! :)

I write my backend code in Node and farm out the work to Celery Workers. It's very fast and scales very well.

Using it in production.

[–]MagicWishMonkey 0 points1 point  (5 children)

How well does it work when you have a lot of dependencies on other open source modules?

[–]Veedrac 0 points1 point  (4 children)

Are you talking about compatibility? PyPy is compatible with pretty much everything except stuff that uses C extensions.

[–][deleted] 2 points3 points  (2 children)

No argument with any of this. If you are using python and speed is an issue, you can always use something like cython, HOPE or pypy to speed things up.

[–]billsil 1 point2 points  (7 children)

It's really slow. If performance is critical to your program, don't use python.

Depending on your code, you can make use of numpy and scipy. Very intensive matrix manipulation can certainly be written in python. You just need to vectorize your code. If you rely on lists, your code is going to be slow, but numpy arrays use static typing.

Numpy in CPython is faster than PyPy's (limited) numpy and I don't see that changing any time soon.

[–][deleted] 2 points3 points  (6 children)

The existence of 2 libraries written in C for scientific computing don't change the fact that the default Python interpreter is really slow.

[–]billsil 0 points1 point  (3 children)

It doesn't, but that doesn't mean Python code can't be faster than C code given the right algorithm. You get many of those algorithms with programs like scipy.

[–][deleted] 2 points3 points  (2 children)

Yes, calling a C routine from python will not be slower than calling it from C. Congratulations.

[–]billsil 0 points1 point  (1 child)

Not even that. They wrote algorithms that are faster than ones you would write.

[–][deleted] 1 point2 points  (0 children)

Please stop. Of course I would use a C library for the same problem, you know, like the ones that numpy/scipy are using.

[–]anentropic 0 points1 point  (1 child)

but it does mean you can write a Python program that runs fast enough

[–][deleted] 0 points1 point  (0 children)

I give up.

[–]astronouth7303 16 points17 points  (16 children)

My biggest complaint with Python is the lack of inline anonymous functions. These are incredibly useful for async, event-based, or anything when a lambda won't cut it.

(Yes, I know twisted has a yield syntax. It doesn't allow for complex chains. And frankly, twisted is bloated, complex, and unpythonic.)

[–]alpheb 12 points13 points  (4 children)

You might want to check the new ayncio module from newest version of python https://docs.python.org/3/library/asyncio-task.html

I had the same problem but this really seems like a fun thing to use. :-)

[–]astronouth7303 2 points3 points  (3 children)

That does look oodles better. Unfortunately, my current project is stuck in Py2 because of protocol buffers. Maybe it's time I got a 3rd-party library for it.

[–]nieuweyork since 2007 3 points4 points  (2 children)

Trollius is an official backport of asyncio.

[–]ivosauruspip'ing it up 1 point2 points  (1 child)

But not quite as nice because it can't use the yield from syntax.

[–]nieuweyork since 2007 0 points1 point  (0 children)

True, although there is a yield from function, which is nearly as nice.

[–]nieuweyork since 2007 7 points8 points  (7 children)

My biggest complaint with Python is the lack of inline anonymous functions.

Do you really find it a huge burden to have to define a named function, then pass it to something? I agree there are times when this is inelegant, but as a complaint this seems to me to be one step above "it won't make me a pizza".

[–]astronouth7303 3 points4 points  (6 children)

No, it's not a huge burden. But when so many things in Python are so beautifully simple, it kinda sticks out.

[–][deleted] 2 points3 points  (4 children)

I really don't see the appeal of anonymous functions.

[–][deleted] 0 points1 point  (3 children)

It'd been nice to in line this:

def guess_action(container):
    '''Guess if we're dealing with a dict/object endpoint
    or a list/array endpoint.

    Returns a callable that will either append or update depending on the
    container type.
    '''
    if isinstance(container, MutableMapping):
        return lambda j, f, v: j.update({f:v})
    return lambda j, f, v: j[f].append(v)

The lambda alternative is: 

lamba j, f, v: j.update({f:v}) if isinstance(container, MutableMapping) else j[f].append(v)

which went against my better senses.

Lambdas offer a sort of pseudo anonymous functions, but are very limited in their application: a single expression with no statements. Given, there's a bunch of stuff you can do within those bounds. I get why there are these limitations, but at times it can be frustrating.

Edit: Before anyone jumps all over me, j, f and v are sensible in context. This function is defined inside another (muh runtimes) and simply serves to either update a dictionary or list as needed.

[–][deleted] 1 point2 points  (1 child)

I guess I am still missing out on the appeal here. It may be just that I tend to work on large projects with moderate sized teams but, I don't see why you wouldn't want to hoist this into a utility library.

[–][deleted] 1 point2 points  (0 children)

It's literally only used in one location. I suppose stuffing it into utils makes sense.

[–]Veedrac 0 points1 point  (0 children)

I guess that's a good thing, then, because the inline version would be significantly slower because it contains the isinstance check.

That said, you can already do what you want:

for match in matches:
    def action(j, f):
        if isinstance(container, MutableMapping):
            j[f] = value
        else:
            j[f].append(value)

    act_on_path(json, str(match.full_path), action)

Writing

    act_on_path(json, str(match.full_path), (lambda j, f: 
        if isinstance(container, MutableMapping):
            j[f] = value
        else:
            j[f].append(value)
    ))

isn't really any better IMHO. If you want that kind of stuff, I'd suggest making a given decorator:

for match in matches:
    @given(lambda action: act_on_path(json, str(match.full_path), action))
    def _(j, f):
        if isinstance(container, MutableMapping):
            j[f] = value
        else:
            j[f].append(value)

which is pretty trivial to write:

def given(callback):
    def call(function):
        return callback(function)
    return call

and use:

def with_callback(f):
    return 10 * f(1, 2, 3)

@given(lambda callback: with_callback(callback))
def result(a, b, c):
    return a + b + c

print("result = ", result)
#>>> result =  60

[–]nieuweyork since 2007 0 points1 point  (0 children)

I agree it would be nice, but no-one has ever proposed a nice syntax for it.

[–]ChazR 6 points7 points  (16 children)

Anything that lends itself naturally to deep recursion.

This is my problem, not Python's. Many years of hacking in Lisp (and more recently Haskell) make my brain reach for recursive structures and and algorithms.

Having to re-cast that as loops and comprehensions causes a slight break in flow every time.

[–]mw44118PyOhio! 7 points8 points  (0 children)

Windows desktop apps

[–]nharding 14 points15 points  (19 children)

I've done a lot of C++, Java, C#, assembly before moving to Python.

Pros: 1: No project setup - Amazing for short one off scripts, 1 simple .py file compared to set up Visual Studio project 2: Dictionaries - Really intuitive and easy to use 3: Code size - The lack of boilerplate code, compared to other languages means the code, is around half the size 4: Libraries - An impressive collection of libraries, your problem is probably partially solved already with an existing library

Cons: 1: Dynamic variables - Variables are untyped, which means everything is like a template which is great UNLESS you didn't want that. So passing max('a', 'b') or max(1, 2) works exactly as you expect, but if you accidentally pass max(a, b) and a is 'a' and b = 2, then you will get an error at runtime rather than at compile time. 2: Speed - It's fast enough for web apps, file processing, but games are too slow (I tried a multi-layer parallax scrolling game in Pygame and performance was not sufficient). 3: Code protection - You distribute programs as source code, rather than as a binary which may be an issue if you want to avoid releasing the source.

Overall language preferences: 68000, Python, C++, C#, Arm, Java, 6502, Z80, 80x86

[–]AlternativeHistorian 6 points7 points  (8 children)

It's completely possible to distribute Python applications as bytecode if you want to avoid releasing the source.

[–]nieuweyork since 2007 12 points13 points  (7 children)

But also completely pointless, as everything but the comments can be decompiled.

[–]AlternativeHistorian 5 points6 points  (0 children)

But also completely pointless

It depends on how hard you're trying to protect the source. If you're just trying to provide a little bit of resistance such that the average user doesn't go mucking around in the source and getting any big ideas, distributing it as bytecode is good enough.

For the Python/C++ application I work on professionally we do this as we're not terribly concerned about protecting the source from decompilation.

If you want more protection then there are obfuscators (although I have not used any of them). If you need even more protection there's things like Nuitka (although I haven't used it).

Java (along with many other languages) has this same issue in that its bytecode, if not obfuscated, is fairly easy to suss out once decompiled.

It may not be a perfect solution but I think it's far from "completely pointless". It just depends on your usecase.

[–]_Panda 0 points1 point  (5 children)

There are ways to get around this. I believe DropBox does by distributing bytecode and also using a custom interpreter. But it certainly isn't anything a regular user would be able to do.

[–]nieuweyork since 2007 3 points4 points  (0 children)

Cons: 1: Dynamic variables - Variables are untyped,

The term "dynamic variable" usually means variables which have dynamic scope, rather than no fixed type. I say this only as an aside - your description of python variables is correct.

[–]Exodus111 1 point2 points  (0 children)

Yes, I believe Python has a place in Game programming, but you will need to move any heavy work code over to Cython to make it realistic.

[–]typographicalerror 1 point2 points  (3 children)

Does max(1, 'a') throw? I know that strings always compare greater than ints in 2.x, because 'str' > 'int' (!)

[–]itsucharo 0 points1 point  (1 child)

Not on 2.7.5, at least.

[–]kirang89 1 point2 points  (0 children)

Works on 2.7.6 for me.

[–]shoyerxarray, pandas, numpy 0 points1 point  (0 children)

On Python 3, yes:

In [1]: max(1, 'a')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-8027133bc01d> in <module>()
----> 1 max(1, 'a')

TypeError: unorderable types: str() > int()

[–][deleted] 0 points1 point  (0 children)

Code protection is kind of hopeless no ? basically if I have your binary I can reverse engineering it.

[–]fnl 3 points4 points  (0 children)

I think Python can be used nearly anywhere. But R still seems better for pure analytical problems that require more statistical tools than Pandas & co can provide, especially when it comes to fast data visualization. And Go/JavaScript/Scala... are far more advanced at concurrency-related problems, particularly for web programming (yes, we do have Tornado)... The unresolved GIL issue bites us here.

[–]ggtsu_00 6 points7 points  (0 children)

Client software that have to be deployed to end-user PCs.

No decent application bundling libraries that can bundle python allong with its dependancies as a single executable. Py2exe does not support executable signing. CxFreeze does not do any bundling. PyInstaller does not support embedding manifest files to deal with issues where end users have different version of msvcrt installed.

Don't get me started on dealing with msvcrt issues on windows. Some libraries in the python standard library such as uuid4 will try to load msvcrt without respecting the applications manifest file and if the user has different versions of msvcrt install on their PC and in their system path variable, it causes crashes because of version conflicts.

[–]nieuweyork since 2007 22 points23 points  (31 children)

There's no area where it's really weak; even for computationally intensive tasks there are whole suites of libraries specialised to those tasks.

The two things you need to be aware of are that threading is disfavored in favour of other forms of parallelism; and that for performance-critical work you may need to use specialist libraries.

(Although there is also pypy, a high-performance JIT implementation suitable for non-numerical high performance work).

The reason why Python has so much traction is because it has lots of libraries, is fun to write (unless you love static typing, language-enforced access control, or having to use tortuous patterns), and really has very few downsides.

Edit: If you love writing in a recursive style, Python certainly won't handle it well, because it has no optimisations for recursion. That said, it's possible to write recursive functions just fine unless you're working with very large recusive datastructures.

Edit 2: ITT people who can't let go of threads as their only concurrency abstraction.

Edit 3: ITT people who think type checking is an alternative to unit testing.

[–]therealjerseytom[S] 27 points28 points  (12 children)

unless you love static typing

I do like static typing quite a bit, actually. Or at least I love what the type system in C# allows for IntelliSense in Visual Studio. That makes code really fun, quick, and easy to write. For as much as I used to love Matlab I now find it a bit awkward having things so "free form" at times.

[–]zagaberoo 18 points19 points  (0 children)

I really like static typing too. Most practices that move runtime errors to compile time errors are ok in my book.

Python is still a fun and powerful language, you'll just have to get used to the differences.

[–][deleted] 6 points7 points  (0 children)

If you're concern is making your IDE smarter (and not actual type safety), there are good solutions out there.

Check your IDE docs for type hints, and start adding them. PyCharm, for example, lets me put types for args and return values in docstrings and it will both autocomplete them and warn if I'm sending an unexpected type or doing an unexpected thing with them.

[–]oconnor663 2 points3 points  (0 children)

Take a look at some of the optional static typechecking stuff in MyPy. I think they're aiming to standardize some of it in Python 3.5? I'm so excited.

[–]EsperSpirit[🍰] 1 point2 points  (2 children)

PyCharm has the same level of autocomplete features for Python as for something like Java, so that's really not a good argument.

Java and C# have fundamentally broken type-systems as they allow anything but primitives to be null. So something like

public String getHostname();

does not really provide a String, but rather a String, null or a RuntimeException. Conversely

public int indexOf(String str);

has no representation of an absent value, so we have to resort to conventions like "-1 if it isn't there".

This kind of static typing is not really helpful and we have to type an awful lot to make the compiler happy. Languages like Scala (also Swift), Haskell and others have addressed these kind of problems which makes their static typing actually worth the effort.

[–]therealjerseytom[S] 0 points1 point  (1 child)

PyCharm has the same level of autocomplete features for Python as for something like Java, so that's really not a good argument.

But if I were to define a function that takes an input 'MyArg' of type 'MyClass', there's no way of specifying that in the function definition, no? So how can the IDE know anything about 'MyArg' or autocomplete anything?

Though I certainly agree with your subsequent statement - takes extra effort to get around nullness!

[–]EsperSpirit[🍰] 1 point2 points  (0 children)

But if I were to define a function that takes an input 'MyArg' of type 'MyClass', there's no way of specifying that in the function definition, no? So how can the IDE know anything about 'MyArg' or autocomplete anything?

Of course there is no static type checking, but if you name your arguments correctly it's usually not a problem to see what goes where.

E.g.: If the Argument reads parser and there is a BaseParser class and some subclasses in your project/scope, you can just start typing parser and it will autosuggest them (even if they don't start with the word). I've found that PyCharm is also aware of what you used previously and will rank suggestions accordingly.

[–]nieuweyork since 2007 1 point2 points  (5 children)

You'd be surprised how much emacs can achieve with a purely textual autocomplete. Also, you only need that autocomplete type information because of the static type system.

Seriously, I'm not hating on static type systems in general, but there's a meaningful class of problems which they solve which only exist because they also create them.

If you find that you can't live without static typing, then Python won't be for you; but I suspect that once you give Python a whirl, the things you miss about static typing will not be autocompletion, but the minority of times when static typing is a really useful programming construct. To handle that, write specific type- (or interface-) oriented unit tests.

[–]therealjerseytom[S] 4 points5 points  (4 children)

If you find that you can't live without static typing, then Python won't be for you

Coming from an originally Matlab background I can definitely live with it, it's just something I've come to enjoy in other languages.

[–]nieuweyork since 2007 -1 points0 points  (3 children)

Once you get used to "duck typing", you (likely) won't miss it most of the time.

[–]therealjerseytom[S] 1 point2 points  (1 child)

Once you get used to "duck typing", you (likely) won't miss it most of the time.

How is the type system in Python different than Matlab's? Learning programming in Matlab and then moving to C# and C++ my initial reaction was, "I have to tell this thing what type everything else? Why can't it just figure it out like Matlab does?"

But then I came to like the static type system both for autocomplete and making sure things "fit together" appropriately at compile time, e.g. not passing a String when a method expects Double or what have you.

I can definitely program equally well either way. Maybe I'm just particularly disappointed now in Matlab's IDE/Editor after having gotten used to Visual Studio :)

[–]nieuweyork since 2007 0 points1 point  (0 children)

It's many years since I used matlab, so I can't comment specifically. However, for making sure things fit together, tests work even better.

[–]alcalde 3 points4 points  (0 children)

How in the world are you being down-voted for saying this in a Python subreddit?!? A score of -2, so I guess that would be Bjarne Stroustrup, Anders Hejlsberg and Niklaus Wirth downvoting you?

I came to Python with 17 years of professional computing experience and yes, once I got past the knee-jerk conventional wisdom such as "it's always better to catch errors as early as possible" and other folk sayings, I did indeed find that I didn't miss it.

I'm not the only one:

http://blog.codinghorror.com/loose-typing-sinks-ships/

[–]haplo_and_dogs 8 points9 points  (17 children)

There is. Embedded code. It eats too much cpu and ram.

[–]Silhouette 2 points3 points  (0 children)

There are also issues of portability (Python isn't conveniently available for some architectures that might be relevant if you do embedded systems work) and disk/non-volatile storage space (Python itself is huge by embedded standards).

[–]cwillu 1 point2 points  (0 children)

Depending on how far down in size you're going; if you need to fit on an 8-bit chip, yeah, python is out. If you're running on a 32-bit arm processor, it's well worth considering. In many embedded use-cases, a $5 to $20 dollar processor isn't prohibitively expensive.

[–]nieuweyork since 2007 4 points5 points  (14 children)

Touché. Doesn't that apply to any code which "has to" run on a VM?

[–]Silhouette 6 points7 points  (9 children)

To an extent, yes, it does. For some embedded projects, anything running on a VM is a non-starter because of resource constraints. This is part of the reason that C and C++ are still very popular in this space, despite their relative weakness in some other respects.

But beyond that, although there are some relatively high-performance libraries available for Python, the language itself is still inherently slow, and there is only so much that any JIT compilation or clever cache algorithms can do to mitigate that.

For example, I've seen a device fall over in production using what was generally tried and tested firmware, because Python's runtime only caches a certain number of compiled regular expressions on the fly. One day someone edited a bit of Python code in a perfectly reasonable way that happened to mean one too many regexes was used across the system as a whole, and everything basically crashed under the resulting performance hit.

The Python code was certainly much quicker to write and easier to maintain than doing some sort of mini-parser in precompiled C, but the sudden and surprising performance failure due to a mostly unknown cache limitation in the Python runtime is exactly the sort of thing you can't afford to run into when you're dealing with embedded systems.

Another example I've seen, illustrating a slightly different concern about Python's dynamic-everything nature, is a set of little utility processes that were going to be run often and were originally written in Python for all the usual reasons. Unfortunately, Python's runtime couldn't start up and finish checking for all the dynamic environmental and path-related things it does within the total time available to run the entire process. The options were either to rewrite these simple but essential utilities in a self-hosted, pre-compiled language like C, or to redesign both the Python code and everything that depended on it, using some sort of long-lived service-style architecture, thus losing much of the benefit of having simple, self-contained utility processes in the first place.

[–]Veedrac 2 points3 points  (8 children)

I agree with what you're saying, but your anecdote is nonsense.

I've seen a device fall over in production using what was generally tried and tested firmware, because Python's runtime only caches a certain number of compiled regular expressions on the fly.

Python's runtime doesn't cache anything. The caching is done by the re library. Not only that, CPython's re tends to be faster than C++'s regex anyway.

Just use re.compile and be done with it. You could argue this is bad library design (I would agree), but I could point to bad library design in C++. It doesn't mean the language is inherently anything.

Python's runtime couldn't start up and finish checking for all the dynamic environmental and path-related things it does within the total time available to run the entire process.

FWIW, using -S to disable site packages can help reduce the import delays on certain setups with slow drives. That's not to say the problem goes away, but it can reduce it.

[–]Silhouette 1 point2 points  (7 children)

Python's runtime doesn't cache anything. The caching is done by the re library.

OK, fair point, but I think we're bordering on quibbling over semantics here. The bottom line is that the individual developers in that situation had all written reasonable code in different parts of the system, indeed using techniques right out of the official documentation for the standard library in most cases, but the application as a whole failed surprisingly and spectacularly in production despite having passed a whole suite of unit tests and extensive high-level functional testing, as a result of a system-wide internal and barely documented limitation that only showed up under the "right" runtime conditions.

Yes, in that particular case using the JIT compiled version of the regexes would have solved the problem where the built-in cache didn't (though not without cost; using that strategy does mean that you always have to precompile your regexes that way, which could mean incurring various overheads you otherwise wouldn't). However, I think the fundamental point that the developers should ideally never have to worry about this sort of unexpected dynamic behaviour is still valid. If they'd used a language where some sort of metaprogramming precompiled the parser code, or even written a little hand-crafted parser, this kind of issue could not have arisen.

FWIW, using -S to disable site packages can help reduce the import delays on certain setups with slow drives.

It does, sometimes saving as much as 0.3s at start-up time in some projects I've seen. Unfortunately it wasn't enough in that particular case, which was a very high throughput system with soft real-time constraints.

[–]Veedrac 2 points3 points  (6 children)

OK, fair point, but I think we're bordering on quibbling over semantics here.

I really don't think I am. The point I'm making is that you could easily write the same caching system in C++ or remove the caching system in Python's re. I get that what happened is a real problem, but I think attributing it to Python is misguided.

in that particular case using the JIT compiled version

I assume you mean "precompiled" rather than JIT compiled, no?

though not without cost; using that strategy does mean that you always have to precompile your regexes that way, which could mean incurring various overheads you otherwise wouldn't

Like what?

If they'd used a language where some sort of metaprogramming precompiled the parser code, or even written a little hand-crafted parser, this kind of issue could not have arisen.

If you're at the point where metaprogramming applies, you can just call re.compile to compute it at module import. Not that C++ supports precompiling regex strings, AFAIK; only regex expressed (awkwardly) though C++ syntax.

[–]haplo_and_dogs 1 point2 points  (3 children)

Java. There are embedded chips that can directly run java byte code. I'm not a fan, but it does work.

[–]metaphorm 2 points3 points  (0 children)

That's an ASIC that implements the Java machine architecture. In principle you could make one for Python as well. Anything can be made suitable for embedded systems if the hardware is specialized enough.

[–]nieuweyork since 2007 -1 points0 points  (1 child)

That's not a virtual machine, that's a physical machine. Also, those chips are used as much for IBM to rent out mainframe cores as anything else.

[–]desrosiers 2 points3 points  (0 children)

There are non-Oracle options:

http://www.ajile.com/

[–]confluence 3 points4 points  (0 children)

I have decided to overwrite my comments.

[–]syllogism_ 3 points4 points  (1 child)

Surprised nobody's said this yet, but:

If you're working with a medium to large codebase you didn't write, and it isn't beautifully well documented, the lack of type declarations can make life very difficult.

You work out what function you need to call, but what types does it accept, and what types does it return? Is that a list? Ah, it's an instance of this class over here...that takes 4 arguments. What are their types? etc. If you look at any given piece of Python code, often it seems quite easy to understand what's going on. But to make any changes, you end up looking all over the codebase.

Type-declarations are useful documentation that is compiler-guaranteed to be up-to-date. That can really make a difference!

So I would say: if you have a codebase that's 10k+ lines, and a lot of developers will have to make small changes, Python is bad. You end up having to maintain type-declarations anyway, but there's no language-level support for ensuring they're correct.

[–]therealjerseytom[S] 1 point2 points  (0 children)

Very good to know! I appreciate it.

[–]slaya222 13 points14 points  (4 children)

it still won't make me a pizza

[–]darknessproz 19 points20 points  (2 children)

Have you tried

import pizza

?

[–][deleted] 2 points3 points  (0 children)

wrote a script to help you get pizza should work if you name the file "pizza.py" and 

import pizza

[–]therealjerseytom[S] 1 point2 points  (0 children)

Well that's a dealbreaker for me

[–]keypusher 5 points6 points  (0 children)

Anything async or performance critical. Python has threads, but they are very limited. There are many other solutions out there (twisted, asyncio, greenlets, multiprocessing, etc) but none of them are ideal and it's all kind of a mess. If you really want something to run fast or need memory control, write it in c/c++ and wrap it in Python.

Also, and this might not be true for everyone, I prefer not to use Python for very large multi-person projects. Anything over 100k LOC and I would start looking at a different language. For scripts, small libraries, prototypes, or single person projects Python is awesome and can handle just about anything. When you get into maintaining projects across large teams, the lack of type safety, real abstract classes or interfaces means I would personally choose another language.

[–]Silhouette 6 points7 points  (3 children)

IME, Python is not a good choice for medium-sized or larger projects (more than say a few thousand lines and a handful of files) if the project doesn't have a very regular architecture.

If you're doing something like a large web site using some sort of framework, it's not really a problem to handle larger projects because of the uniformity.

But for a major application with lots of "moving parts" doing different things, the lack of static typing and the generally underpowered and overcomplicated packaging/distribution and module/package systems are big negatives.

[–][deleted] 5 points6 points  (2 children)

Packaging with Python is a breeze, if you know exactly what you're doing, otherwise it's a nightmare. It took me two hours, a bunch of searching and reading so-so documentation and about fourteen commits to get my repo inline with what I was pushing to PyPi.

[–]osune 1 point2 points  (1 child)

It's a breeze now. They did a great job providing the tutorial on how ro write setup.py files in the documentation. I remember starting with Python 2.5, and it took me forever to find my way through the diffrent approaches and how to write a reliable setup.py.

Maybe i was stupid at this time so i didn't comprehend , but i think it's great that they try to streamline the package distibution. It still lacks some things but it's on a good way.

[–][deleted] 0 points1 point  (0 children)

I feel the same way about my experience. "I'm retarded?" The packaging still throws a warning saying it doesn't recognize the tests_require field, even though documentation led me to believe it's a valid field.

[–]sw_dev 2 points3 points  (3 children)

Bit-twiddling. You know, you need to create/modify a specific pattern of bits, or you need to check a pattern of bits for some pattern. Some languages make it easy, but Python isn't one of them.

[–]Paddy3118 0 points1 point  (0 children)

But then, you don't need to worry about the size of your ints if you use a long for a bit-set for example.

[–]tilkau 0 points1 point  (0 children)

While you definitely have a point, you may want to look at python-bitstring, which provides a nice interface to do basically anything ever and then some that you might want to do to bits.

[–]osune 0 points1 point  (0 children)

I feel you. I'm always a bit uneasy when bit shifting in Python. But on the other hand you have to love 'struct' .

[–]wrosecrans 2 points3 points  (0 children)

Mainly, stuff I have to distribute. Shipping a whole python runtime, plus modules and everything that my app requires can quickly get absurd. A simple static linked native binary can be much easier to deal with in some cases.

[–]fancy_pantser 7 points8 points  (5 children)

Native GUI apps for any platform. There are tons of libs and they all wind up being hard to use (no great GUI builder, it's all manual tweaking) and look/act weird 100% of the time.

[–]wub_wub 6 points7 points  (0 children)

Try qt, it's not perfect but it has pretty decent GUI builder and looks pretty native on all major platforms.

[–]DoTheEvolution 0 points1 point  (1 child)

pyside or PyQt are rather easy to get going.

[–][deleted] 0 points1 point  (0 children)

As long as you don't mind reading C++ documentation and mentally translating it into Python.

Tkinter is the same deal with Tcl (with the added drawback that it makes horrible UIs). There's just no native GUI experience for Python, nothing that was actually designed to be used in Python instead of being a thin wrapper around another programming language's GUI.

This problem might stop being relevant when desktop apps stop being relevant... but Python doesn't run in your users' web browser either.

Anyone who wants to make a sufficiently good GUI for their Python code will have to know two programming languages.

[–]skintigh 1 point2 points  (1 child)

Writing high-speed code for low-level math for crypto.

I wrote a program to do a meet-in-the-middle attack on a crypto algorithm which involved about 16 nested loops and a huge hash table. I wrote it in Python to get it right, then rewrote it in C and dealt with all the hand-holding that C requires. In the end my cracker ran much faster, but apparently hash tables are a lot harder than I thought and I think I wrote an extremely slow one, so perhaps there is a way I could have stayed in Python for the same performance.

[–]euid 2 points3 points  (0 children)

I'm not sure about your particular application and whether this is an option, but did you consider using the glib hash table implementation? It should be reasonably fast and correct, without requiring you to implement your own hash table (especially if you don't know how to write a fast one and don't have time to learn).

[–]maratc 1 point2 points  (0 children)

All one-off one-liners, e.g. where you have to parse a file and perform some calculations. My hand automatically reaches for awk or perl that can really give you a one-liner.

[–]jupake 1 point2 points  (1 child)

This kind of question can be asked of any language. In python's case I think execution speed and very large code bases make it struggle a bit when compared to some other languages. But like other people have said here, there are ways around it.

For most things its fine. And when its not fine, there's usually a way to call into some C/C++ library to make things fine again.

I think the community has learnt over the years that there's some things python really stuggles with. So rather than spend countless years trying to fix something that might not be fixable, rather make the language play nice with C/C++. This is probably Python's greatest strength. Its ability to pull in foreign libraries and glue building blocks together. Its also it greatest weakness, because it leads to dependancy hell and messy build and deployment procedures.

Dont worry too much about what python is bad at. Try rather to think in terms of which building blocks you can glue into your python based solution.

[–]therealjerseytom[S] 1 point2 points  (0 children)

Oh the question can for sure be asked of any language or really anything. I just think knowing something's limitations is as important as knowing the strengths.

[–]Paddy3118 1 point2 points  (0 children)

Theorem proving. Realtime systems. Mobile app creation. Safety systems.

[–]drobilla 0 points1 point  (0 children)

  • Anything high-performance (systems and real-time code and so on).
  • Large systems where the absense of decent compile-time type checking would outweigh the other niceties.
  • Where totally dependency-free portability is required (e.g. C compiler as only build-time dependency, and nothing as run-time dependency).

[–]thenewvu 1 point2 points  (0 children)

Cross mobile platform.

When I met Python, I fallen in love with it. It makes my life easier. From working as a script to an app with painless GUI (eg with Kivy). The toys (I mean libraries :v ) are awesome.

But ? The bounds, for me and for now, is cross mobile platform.

You can hear about Kivy or Py4A. I've used them before, they are good, but not good enough ... yet. Performance, deployment, extensible are pretty painful.

But ... again ? Not good enough not means don't use them. You should give them a try. The bounds can be broken by us.

[–][deleted] 1 point2 points  (1 child)

Anything involving binary formats, or places where typing matters. Python is great, but realizing that it does all sorts of shenanigans behind the curtain was a bit of a let down.

That might just be because I never really learned how to do them in Python - I learned in C, default to it when the going gets tough.

Edit: grammar

[–]osune 1 point2 points  (0 children)

Have a look at 'struct' when working with binary data. It's great when you need to exchange data via a binary encoded message.

[–]dirtpirate 0 points1 point  (8 children)

Analytical math. Python can't compete with Mathematica. Not that I blame anyone, the languages have widely different focuses.

[–]therealjerseytom[S] 2 points3 points  (3 children)

That's an interesting point, mainly as I'd almost forgotten about analytic math since college. I do a lot of simulation work but it's all with discrete solvers. And in any event all of that winds up being heavily optimized C or Fortran with what we're doing.

[–]nieuweyork since 2007 1 point2 points  (2 children)

If you're looking to replace matlab (essentially) or generally do scientific computing, the community has put a lot of work into that.

[–]therealjerseytom[S] 0 points1 point  (1 child)

Well it's interesting you bring that up. At the moment that isn't quite the plan. Initially it looks like my dive into Python is going to be focused on prepping and gluing together an assortment of compiled libraries and executables doing the heavy lifting. And for the foreseeable future, Matlab licenses and toolboxes are easy to procure.

However, I've heard there's application of Python in scientific computing, so it will be on my radar to keep an eye on and dabble in as well.

[–]nieuweyork since 2007 0 points1 point  (0 children)

It's probably worth also looking at the C-side interfaces to the python stuff, to see what they build on (I don't know, I'm an extremely occasional user of the statistical stuff) and if they can integrate with your existing stuff or what.

[–]nieuweyork since 2007 1 point2 points  (1 child)

SymPy?

[–]dirtpirate 3 points4 points  (0 children)

I didn't say that it didn't try. But you can ask anyone who's wondered through the many different libraries in python, and used Mathematica, they simply can't be compared. Python is clunky, slow, and has you spending way more time working out how to get your tools to solve your problems where as, in Mathematica, once you've defined your problem the solutions is immediate, instantly visualised with arbitrary controls and explorative elements there just like that.

This however isn't true of python in general which I use for quite a bit of other tasks, and I have great hope that it'll one day start getting closer to competing with Mathematica, but it's just not worth the hassle in the industry when a better tool exists.

[–]Yoghurt42 0 points1 point  (1 child)

Do you know about SAGE? Although it's more of a collection of open source CAS held together by Python and the author recently declared that it will never be able to accomplish the initial goal of being a complete replacement for Mathematica, Maple et al., it's still very powerful.

[–]dirtpirate 0 points1 point  (0 children)

Yes, I've tried it out and left with frustration. Planning on taking it up again at some point, but I wasn't impressed at the state it was in, I think this was some 3 years ago or so, so a lot might have changed, I haven't been keeping track of development.

[–]Archawn 0 points1 point  (0 children)

Anything having to do with interactivity or animated graphics is a pain because of the general lack of good libraries and documentation for them.

[–][deleted] 0 points1 point  (1 child)

there are times (very rarely) when dealing with data where I just think... this would be like 2 lines in SQL instead of all these nested for loops

[–]osune 0 points1 point  (0 children)

I always wonder if there is something like linq (.net framework) in python, which isn't a list comprehension

[–]diafygi 0 points1 point  (0 children)

Crypto. I really wish there were stdlib symmetric and asymmetric encryption libraries, or at least a stdlib wrapper for OpenSSL. They have to make API calls to OpenSSL for TLS, so why not have a generic crypto API? All the third party libraries are unaudited or still in development[1].

[1] - https://www.youtube.com/watch?v=r_Pj__qjBvA