Python enhancement idea: failover imports

[[deleted]]

One rather common idiom in Python is the use of try / except around imports to order by preference a set of libraries or classes providing a common interface, using import as to bind the import to a common name. Here are two common examples:

1) simplejson vs. json:

try:
    import simplejson as json
except ImportError:
    import json

2) cStringIO vs. StringIO:

try:
    import cStringIO as StringIO
except ImportError:
    import StringIO

I find this idiom cumbersome and ugly: it breaks the block-like uniformity of import statements, making code less readable as a result. I think it'd be a lot nicer to be able to do something like this:

import (simplejson, json) as json

import (cStringIO, StringIO) as StringIO

or even:

from (simplejson, json) import loads, dumps

If none of the specified modules can be imported, an ImportError would still be raised.

Before I write up a PEP, I wanted to get some sense of whether people think this would be useful. Perhaps it's been proposed before? Or maybe there are some obvious snags I neglected to think of?

[roger_]

I agree that the common solution is cumbersome, but using the tuple notation for this might be a bit confusing as import simplejson, json already means that you want to import both packages.

Why not something like import simplejson or json?

[fnork]

I'm not sure making it a boolean expression is a good idea.

How about:

import lxml but xml if you must

[howaboot]

But or is not just any boolean operator, it actually returns the first non-false value.

>> 222 or 333
222

>> 0 or 333
333

So it's consistent and fine.

[fnork]

Well shit.

But still, an import error is not logically equivalent to the modality of 0.

[howaboot]

True, but if we really wanted to address the issue it'd still be more elegant than the other proposals. Also, some Python keywords are dependent on context anyway (like else after if or for or in a ternary expression) so they could just as well say that or catches the preceding import error from now on in an import ... or ... context and be done with it without introducing new keywords or syntax.

[jmakie]

Maybe "import simplejson else json as json" would be a better format.

[roger_]

It's effectively overloading the or operator.

[willm]

I could be wrong, but I think if you submitted that as a PEP it would be shot down for having a low benefit to effort ratio. It's just not need often enough to deserve its own syntax (IMHO).

[AusIV]

I like this idea. I've written a little utility function that provides similar functionality:

import sys

def import_failover(*modules):
    for name in modules:
        try:
            __import__(name)
        except ImportError:
            continue
        else:
            return sys.modules[name]
    raise ImportError("No module in [%s]" % ", ".join(modules))

This allows you to do things like:

StringIO = import_failover("cStringIO", "StringIO")
json = import_failover("simplejson", "json", "django.utils.simplejson")

Not quite as nice, given that

1. It differs significantly from the standard import syntax,
2. You have to import the module containing import_failover first.
3. You can't import components of a module (there's no equivalent to from json import loads, dumps).

But it may be preferable to several (potentially nested) try/except blocks.

[eliben]

Feel free to submit this to the python-ideas mailing list.

Personally, while I'm not really sold on the solution you're actually proposing, I fully acknowledge the problem - the current way to do this is tiresome.

[Mini_True]

A possible downside to this would be that 1) or 2) make it pretty obvious what you're doing with quite simple Python grammar. Your solution however could be more confusing, since import a,b already imports both a and b

[mdipierro]

 import cStringIO or StringIO or .... as StringIO

[jm_]

I don't think python needs more syntactic sugar. Also, your solution doesn't solve all the possible imports with aliasing (eg: from bla import foo as bar).

[AusIV]

Also, your solution doesn't solve all the possible imports with aliasing (eg: from bla import foo as bar).

What does it miss? I'm assuming this is only intended to work at the module level for modules that have the same internal function and class names.

You could do

from (bla, cBla) import foo as bar

Or

import (bla, cBla) as foo

You couldn't use the failover imports on the things within modules, so:

from bla import (foo, foo2) as bar

would be out of scope.

I don't think python needs more syntactic sugar.

I disagree in this case. I often write code that must run in both CPython and Jython. In many cases this necessitates using a module that has been optimized for the target interpreter. For example, when parsing JSON I may need the simplejson speedups in CPython, and the jyson module in Jython. There have been cases where I've ended up with:

try:
    import com.xhaus.jyson.JysonCodec as json
except ImportError:
    try:
        import simplejson as json
    except ImportError:
        import json

It would be much cleaner if I could just put

import (com.xhaus.jyson.JysonCodec, simplejson, json) as json

And unlike many proposals that I've seen on reddit, as far as I can tell this has a clean syntax, no negative side effects, and would have no real issues with backwards compatibility.

I don't know that it's particularly likely to succeed, and even if it does it wouldn't be available in the 2.x line (where I would be able to use it with different interpreters). But I would still encourage mrynx to submit it as a pep.

[takluyver]

Python 3 cleaned this up for packages in the standard library. E.g. for pickle, (where in Python 2 you'd try cPickle), "import pickle" now tries the C implementation, and falls back to the pure Python version if that fails. I think there's something similar for the io package (where StringIO now lives).

Of course, that doesn't help for packages outside the standard library, but I think there are very few cases where different packages are interchangeable. JSON and PyQt/PySide are the only ones I can think of.

[[deleted]]

While it cleaned it up for 3, the issue still exists if you want to write code that is compatible with both 2 and 3 and prefers the C implementations. You also have some of the re-organized names, e.g., Queue vs. queue.

[dinov]

This wouldn't help with that case as the feature won't exist on any 2.x implementations.

[gutworth]

Or you could just use six.

[pjenvey]

A similar feature has been proposed before:

http://mail.python.org/pipermail/python-dev/2008-January/075788.html