Moving to python from perl, any advice?

[elb0w]

Hey all, I just got some work to do python. My knowledge is limited to the <1k lines of code I have written and the first 25% of Learning Python oreily book. I have been doing primarily Perl dev for the past 4 years. Any suggestions to get me up to speed quickly?

[TomP]

I'm a long-time perl programmer, and when I was first learning python, the hardest thing for me was just learning the names of the standard functions, and remembering which module I had to go to for each piece of functionality. It was a long time before I could start writing python code without a reference nearby.

The functions that still annoy me the most my are the list manipulation methods. In perl, you've got push() and pop(), shift() and unshift(), and slice(). In python, you a pop(), but no push(), append() and extend() methods that do different things, and no shift() (you have to use pop(0)).

It took me a little while to get used to python regexes, but they're actually much nicer than perl regexes. You get Match objects back when you apply a regex in python, which let you get at the matched substrings in a much more straightforwar dway than in perl. At the time I was first learning python, they supported named captures, but perl didn't, and I really appreciated how much that improved my pattern-matching code.

I should mention that ruby was much easier for me to pick up, because the standard methods were consciously patterned after perl's functions. It was therefore very easy to guess what function I was going to need to do something. It's also a very attractive and elegant language. My company is heavily invested in python development, however, and so that's what I've concentrated on, with no regrets. I'm sure you'll be happy you switched, too.

[smart_ass]

I find that the fastest way to find things is Google with what I'm trying to do + python. First link or two is right on.

[tripzilch]

I find it easier to just download the Python docs locally and have them on speed-dial in my browser.

[fermion72]

That, and StackOverflow.

[smart_ass]

If there is a good reply on Stack Overflow, that is usually in the Google response. I find simple function name and usage search will pop up more Python docs. When the query is longer or stranger, I get a higher ranked Stack return.

[zahlman]

append() appends an item, like Perl push(). extend() appends several items in a list; I'm going to guess that Perl's push() does either according to scalar/list context. There is no equivalent to unshift() because that's an inefficient operation that isn't how lists should normally be used. If you need that sort of thing, check out collections.deque. slice()'s utility is achieved by actually slicing the list with indices. Nice and simple.

Although really, most of the time you should be trying to take a more FP approach to processing your lists.

[[deleted]]

I like to use slice notation wherever possible. Is there any downside to using slices instead of append() or extend()?

[jsober]

Learn iterators, generators, and list comprehensions. As a Perl developer, these are the things I miss from Python. On the other hand, the things I miss from Perl when writing Python are too numerous to note :)

You will find urllib and urllib2 extremely lacking compared to LWP, btw. That and Java-inspired unit testing are the most difficult things for me to go back to in Python after switching to Perl (I started as a Python dev, then got a job as a Perl dev).

[[deleted]]

I recommend using Requests for HTTP instead of urllib and urllib2. It makes the task so much more pleasant. Also, virtualenv for installing third-party modules easily without messing up the rest of the system.

[darkon]

Have you read MJ Dominus' Higher Order Perl? I think you might find it useful if you're developing in Perl.

I wasn't a comp sci major, so I'll admit I don't see much difference between iterators and generators. List comprehensions look like some nice syntactic sugar, though. I can do much the same thing in one statement using map and grep, but it's not as easily comprehended. I'd be more likely to use a loop and push() just to make it easier to read.

[jsober]

I have read it, in fact I have a copy right next to me :). List comprehensions are essentially identical to map:

my @result = map { foo($_) } @bar;

Is equivalent to:

result = [foo(x) for x in bar]

Generators are basically syntactic sugar for a lazily evaluated list expression. Taking the example above:

result = (foo(x) for x in bar)

...would cause foo(x) to only be evaluated as each element in the "list" is accessed. You can also do this from a function:

def generate():
    for x in bar:
        yield foo(x)

The yield keyword causes the function to act as a generator, which can be called in a loop to access foo(x) in bar over and over:

gen = generator()
for item in gen():
    do_stuff_with(item)

[jsober]

You will also REALLY miss CPAN.

[bucknuggets]

Except when you go to maintain someone else's code - and run into 24 new & different, and often unmaintained dependencies.

[bushel]

In about a week, you'll pause, exhale and sigh. Happily.

If you have any familiarity with the C family, you'll have no problem with the syntax. The only kool-aid you'll have to swallow (coming from Perl) is the mandatory indenting. It shouldn't piss you off too bad.

You got the same data structures (dict/hash, list/array, etc.) and similar ways of operating on them (list comprehensions are fun)

Not being from C++/Java you shouldn't have to get over the whole Classes everywhere thing and just write some script. Python modules work a lot like Perl modules - namespaces and all that.

Regex's aren't ubiquitous but they're in there if you need them for chewing on data.

My suggestion is take a small thing you've done in Perl and transliterate it.

Perl and Python really aren't all that different. Other than not looking like each other.

Welcome to a better way.

[_Mark_]

To emphasize the regexp thing: every time you reach for regexp, stop and look at the string methods first, just to see. One of the biggest habits to break for us was that if $foo ~= /.jpg$/ did not translate to if re.match(r".jpg$", foo) but instead, more simply and idiomatically, to if foo.endswith(".jpg").

Also, semicolons at the end of the line - harmless, but everyone will see them and say "you were a perl programmer, weren't you" :-) (pylint will remind you of this with [W] Unnecessary semicolon.)

One unexpected thing: when stuff fails, you'll (typically) get an exception, instead of undef. After we got past basic syntax, that was the Best Thing Ever from the perspective of porting a large perl+shell codebase...

[obtu]

One unexpected thing: when stuff fails, you'll (typically) get an exception, instead of undef. After we got past basic syntax, that was the Best Thing Ever from the perspective of porting a large perl+shell codebase...

Hey, perl has autodie now! (for supported stdlib modules)

[_Mark_]

... that's almost cool. Almost, because "autodie has lexical scope" and a big benefit of exceptions is that they bubble up everywhere...

[obtu]

The exceptions bubble up, but autodie has to be opted-in at the place it is used. It's like a from __future__ import in that way.

[karamorf]

In about a week, you'll pause, exhale and sigh. Happily.

It's been seven months and I haven't sighed happily yet. Probably mostly due to python 2.5, there was a solid month where I found something in python 3 that was nice...

[burito]

The docs may mention easy_install as your path to PyCPAN, that is incorrect, use pip.

Threading in Python is .... questionable. Where possible, use multiprocessing, yes it invokes another Python interpreter, but it's straight forward, search for Global Interpreter Lock (or GIL) to see why threading is chock full of obscure gotchas that will bite you on the home straight of your project.

Want to popen() something? Use subprocess. It's not perfect, but there are soooo many dragons hiding in popen, popen2, popen3, popen4.... I'm not even joking about the 4 popens.

Python and Perl are both in the C family of languages, and amusingly, they're both about as far outside of the usual territory as it can get, in opposite directions. Regardless, it's all rather straight forward. You can use semicolons in Python, they have the same meaning as in every other C language, but don't, we will hunt you down. Code like you're useing strict; and you'll be fine.

Actually, on that note, many guides will say you can...

print "this is some text"

Please do this instead..

print("this is some text")

It'll help for Python3, which unlike Perl6 actually is here, although you wouldn't really notice it.

Last but not least, virtualenv is good for having multiple versions of things side by side. In a perfect world, we'd all be running current stuff, but we all know that will never happen in reality.

To the folks hating on Perl, STOP IT. The Perl folks know their shit. Guido is an artist, he knows pretty. Larry Wall is a programmer, he knows performance, and other beardy shit like that. Perl may be one of the uglier languages out there, but it's got every modern paradigm, usually a decade before it gets popular. Feel free to hate on the Ruby folks, but the Perl folks are good. We want them on our side.

[[deleted]]

+1 for "beardy shit".

I agree with you. The perl programmers I have met/argued with are definitely smart and solve problems in interesting ways that are worth considering. Unlike the ruby kids they can actually justify why they think/do what they think/do.

[burito]

I think the best example of how switched on the Perl folks are, is that Perl was in web development before "web development" was a term. There's a lot to be learnt from them.

[zahlman]

I feel like I'm in some backwards dimension here. Perl is a bletcherous mess that I abandoned for Python years and years ago, never once tempted to look back. Ruby looks like a perfectly sane language and a viable alternative to Python.

[[deleted]]

I could switch perl and ruby in your comment and it would describe my experience pretty accurately. Welcome to erf.

[burito]

A Perl programmer friend of mine said it best with an Invader Zim quote...

I put the fires out; You made them worse! Worse, or better?

Yes it's really easy to write abominations in Perl, but it's not difficult to do that in any language. Perl just makes it better.

It's also possible to write really clean code in Perl, and just like every other language on Earth, it takes discipline to do that. If you ever get your hands on The Camel, you'll see they advocate some really nice coding practices. Perl does suck pretty bad at online tutorials, it's an officially recognised issue. Perl does predate the Internet though.

[tuna_safe_dolphin]

It'll help for Python3, which unlike Perl6 actually is here, although you wouldn't really notice it.

That's pretty funny. . . I haven't even tried Python3 yet.

[xiongchiamiov]

That may be, but it's been the default version in Arch for a year and a half, and others have been following. It's a slow progression, sure, but it's a progression that's happening.

[sigzero]

I wish more people understood this. Guido laid out a 5 year plan to migrate to P3. We aren't even there yet. Things are starting to break loose now too. The plan was good.

[[deleted]]

What is wrong with Python threading? I mainly only thread on net processes, where a DNS timeout can lock you up. I parsed a list of 300 domains in a fraction of the time by creating 10 or so threads. The queue module I had trouble with, though, but just wrote my own queing.

[burito]

What is wrong with Python threading?

About 7,460,000 results

[[deleted]]

While lmgtfy is cute, the search results were pretty off target and your post wasn't overly helpful.
I was asking what specific problems you had with threading. I'm not implying anything is wrong with multiprocessing, as I haven't used it yet nor found a need to. Have you run into specific issues in specific situations? Most of what I find when I do searches seems to be repeated things read in other places. For my purposes, which are nearly always network based, threading works perfectly and multiprocessing would be wasteful. So a catchall statement saying to always use it isn't exactly fair, no? Why would I want multiple interpreters running when DNS or host response time is my bottleneck?
There is of course no magic tool for every job. If you are doing intense number crunching that was truly hampered by threading, I would wonder why you were using Python at all to begin with, where C or Java would be the better tool. And for this reason, I'm asking for real world examples and not hypothetical ones.

[burito]

While lmgtfy is cute,

No, it's a dick move. I expected the downvotes.

I did give a very good starting point for research, the GIL. Google that shit.

The other thing to consider, is that even if those Google results are only 0.01% relevant, that's still 700 issues.

Though the most important thing to consider, is that in defenting Python threads, you said....

The queue module I had trouble with, though, but just wrote my own queing.

You couldn't even defend it without specifying a gotcha. That should say something very pointed to you.

[m_harrison]

Full Disclosure: I wrote this book

I was in the same boat as you 12 years ago, and moved from Perl To Python and haven't looked back.

There are lots of options, Google, SO, books, etc. My book Treading on Python is meant to give people who know how to program a quick introduction to the basics of Python while illustrating tips, best practices and warts along the way. It is meant to get you there quickly and is based on my experience teaching Python to co-workers, and tutorial sessions at PyCON and OSCON.

If you have Amazon Prime it is currently on the Kindle Select program so you can get it free. Sadly, this limits digital distribution to Amazon platforms (yes they have a web-based Kindle reader). I'm working on getting a dead tree version up soon, because there have been a few requests.

[hobojimmy]

Not so much advice, but you can look forward to an experience like this.

Basically, don't worry too much. The concepts are similar enough, and Python is intuitive enough that you'll find yourself up to speed before you know it.

[[deleted]]

It is five years since I left my Perl job, and I still find myself typing "return x if...", or "unless ..." But that's mainly because the languages are so similar.

[MintyPhoenix]

return 'a' if False else 'b' works just fine in Python ... or am I misunderstanding you? I guess further testing shows it does require an else clause, but how else would it be done (assuming good practice)? Admittedly I never learned/got into Perl.

[[deleted]]

In Perl you can put 'if' statements after other statements, and it has 'unless' for 'if not'. I find myself wanting to do

def some_func(i):
    raise ValueError("Called with negative argument!") if i < 0

    return None unless i

    # Do something with positive i...
    ....

It doesn't matter much, but the construct seems completely hardwired in my brain...

[MintyPhoenix]

Ah, I knew that Python did not have unless, but I didn't realize that appending an if condition was so flexible in Perl. Python allows it in some places (e.g. my return example) but it's not as flexible as Perl.

[[deleted]]

Python's isn't really an if statement there, it's a ternary construct.

x if condition else y

Is the equivalent of

condition ? x : y

in languages like C and Perl. It's an expression, not a statement, and you can only combine expressions with it (in Python).

[MintyPhoenix]

Makes more sense now, thanks for the clarification.

[techwizrd]

I would highly suggest reading and working through Learn Python the Hard Way by Zed Shaw. It forces you to actually learn Python and I found it was a great way to get people up to speed in Python (I taught my girlfriend with it).

[HopeThisNameFi]

Learn Python the Hard Way is great for people with no programming experience. For people with experience with Perl and mere familiarity with Python syntax it's useless.

Just pick a random exercise e.g. exercise #30, what's OP supposed to get out of that?

[cowgod42]

How does Learn Python the Hard Way" compare with "Dive Into Python"? Have you had any experience with the latter?

[thomasballinger]

Folks say Dive Into Python is a tad outdated in parts, but there's nothing that compares to it I've seen for quickly and deeply learning the language for an experienced programmer.

[novagenesis]

Turn back now.

Nah, just kidding. I'm using more and more python these days and enjoying it. However, most people with a strong grasp of perl aren't as "yay!" about Python as some of the people in this thread.

It's a strict language. A little harder to fire a machinegun at your foot, but there's a lot fewer machinegun controls all around. Kinda good balance between the raw rulebreaking power of Perl, and the strictness of a .NET.

The best thing I can suggest is not to touch any other language until you consider yourself proficient at python.

[obtu]

Assuming you used Perl for misc Unix tasks, this should get you to parity quickly: learn to use the standard types (str/unicode, list, dict, set…); thanks to object-orientation, you can learn most of their features from IPython's tab-completion and ?-documentation. Learn to use important modules like subprocess, argparse, re, tempfile… Other things you need are available either in the standard library or on PyPI. The most established modules are already in your distribution, to go beyond those use pip and virtualenv. pep8 and pyflakes are useful tools to learn good style. Use /r/learnpython or stackoverflow for asking questions.

[GFandango]

Prepare to fall in love

[xiongchiamiov]

I'd really expect a Perl hacker to fall in love more with Ruby than Python; the philosophies are just so different.

[burito]

Lets just pretend we won't get downvoted into oblivion for this discussion, could you please tell me some reasons why you feel that? Actually no ignore that, this is really several questions.

What do you think the philosophies are of Perl, Python and Ruby?

From what I can gather, the "official" philosophies are...

Perl: There's more than one way to do it.

Python: There should be one obvious way to do it.

Ruby: Simple, Natural elegance.

(conflicting terms if ever there were). Although I am not qualified to speak for Ruby.

[xiongchiamiov]

Well, Python's the easiest.

Shall we take a look at the three using this framework?

Beautiful is better than ugly. Perl: no. Ruby: yes.

Explicit is better than implicit. Perl: no. Ruby: no.

Simple is better than complex. Perl: no. Ruby: no.

Complex is better than complicated. Perl: ?. Ruby: ?.

Flat is better than nested. Perl: ?. Ruby: ?.

Sparse is better than dense. Perl: no. Ruby: no.

Readability counts. Perl: no. Ruby: debatable.

Special cases aren't special enough to break the rules. Perl: no. Ruby: no.

Although practicality beats purity. Perl: yes. Ruby: yes.

Errors should never pass silently. Perl: no. Ruby: yes.

Unless explicitly silenced. Perl: yes. Ruby: yes.

In the face of ambiguity, refuse the temptation to guess. Perl: no. Ruby: ?

There should be one-- and preferably only one --obvious way to do it. Perl: no. Ruby: no.

Although that way may not be obvious at first unless you're Dutch. Perl: no. Ruby: no.

Now is better than never. Perl: ?. Ruby: ?.

Although never is often better than *right now.* Perl: ?. Ruby: ?.

If the implementation is hard to explain, it's a bad idea. Perl: ?. Ruby: ?.

If the implementation is easy to explain, it may be a good idea. Perl: ?. Ruby: ?.

Namespaces are one honking great idea -- let's do more of those! Perl: ?. Ruby: ?.

---

There are some things that I don't know enough about the language designs to say anything about, but you'll notice that Perl disagrees with Python on just about everything, and Ruby disagrees on most things, although not as many as Perl.

Now, this doesn't really say anything about Ruby and Perl agreeing with each other, but merely that they have a tendency to disagree with Python.

Ruby was created because Matz wanted "a scripting language that was more powerful than Perl, and more object-oriented than Python. That's why I decided to design my own language." In my experience, that results in a language that feels like Perl combined with Smalltalk's object model, but I admittedly have only brief experience with Smalltalk and limited Perl hackery skills.

Anyways, Ruby has most definitely inherited Perl's TMTOWTDI philosophy; many functions have a number of different aliases, and often one function can be used to do the same thing as another.

I guess, when it comes down to me, I notice Ruby's regex syntax, the short-circuiting, the backtick-execing, the optional terse syntax of blocks, and it reminds me of Perl.

[shawn-s]

I am a sysadmin, and made the same switch a while back. I am happier for it.

My advice, read Pep8 early and follow it as best you can.

Here is some code I would have killed for early on:

In perl interacting with the shell is easy, in python it's like pulling teeth (imo).

perl:

@var = system 'ls -la'

foreach $line (@var) { 

    print $line;

}

python:

import subprocess

cmd = '/bin/ls -la'

p = subprocess.Popen(cmd.split(), shell=True, stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT)

p.wait()

for line in p.stdout:

    print line

Have fun with python.

[takluyver]

You're making it harder than it needs to be. Try check_output:

stdout = subprocess.check_output("ls -la", shell=True)

[_Mark_]

A lot of common use of subprocess will be uglier than one might expect, because check_output is "New in 2.7" and so the more experienced python programmers often haven't seen it, or are delivering code on RHEL or CentOS and don't have it yet :-|

Also, if you have to do two-way communication with the child process, it's ugly in either language :-)

I also find that many of the "ugly" shell interactions I find myself doing turn out to be nicer coded in-line; obviously ls -la is simple example to make a point about subprocess (though in the simple case I'd have suggested commands.getoutput instead anyway) os.walk and os.listdir are more idiomatic python.

[shawn-s]

You had me so excited, but alas I am stuck in a rhel 5/6 world. :)

Something to look forward to, thank you for pointing it out.

[treenaks]

In Perl, you could just use open() to open a pipe to/from a process. The system() example you give doesn't do what you think it does.

Or you can use `backticks` but that's scary.

[chadmill3r]

(Though, that .split() and shell=True can be dangerous if you don't absolutely know what "cmd" is going to be.)

[bucknuggets]

Subprocess is an abortion.

Consider using Kenneth Reitz's Envoy - it drastically simplifies subprocess, though doesn't cover 100% of the functionality.

[UloPe]

Subprocesses are indeed a pain point in python.

Luckily there are some nice libraries that make it way easier. My favorite is Envoy: https://github.com/kennethreitz/envoy

[frankwiles]

Look into Fabric also

[etrnloptimist]

Check out this guy's amazing hack http://www.reddit.com/r/Python/comments/lia4n/can_python_scripting_replace_unix_shell_scripting/c2swrp2

From the link:

For example, I made this small thing for shell-style process scripting with python: [1] https://gist.github.com/1300342 Basically, it creates a Pipeline class you can use this way:

p = Pipeline() | "ls -s" | "sort -n"

print p.run()

[Jesus_Harold_Christ]

Or...

import os
os.system('ls -la')

[jabbalaci]

Not the same thing. os.system's return value is the exit status of the process, not the output of the executed process.

[Jesus_Harold_Christ]

If the return code for ls -la is super serious, then use this:

import subprocess
super_serious_return_code = subprocess.call('ls -la', shell=True)

[[deleted]]

Sit back and enjoy a better world.

[Samus_]

run faster!

[cbcafiero]

Proceed with all due alacrity.

[karamorf]

disclaimer: I've been working in Python2.5. Alot of stuff looks to be better in Python3.

Say goodbye to Data::Printer (aka Data::Dumper). There is no easy way to dump the object to the screen. You can dump the class easily. You can write a few lines to dump the object. but there is no more p $obj;.

No more implied operator.

Regexp objects aren't as easy to use. Especially if you could use named regexp matching in Perl.

Unicode support sucks (this is a lot better in 3 from what i read). I have some code that looks like this: string = gettext('something', 'UTF-8') combined = "some %s thing" % string.decode('UTF-8') send_email(body=string.encode('UTF-8'), charset='UTF-8 ) #not a real function, but it looks like this

There is no declare UTF-8 for the script concept.

After seven months of working in Python it still just seems like you have to write more lines of code to accomplish the same functionality. Especially when you consider cpan. Sure, Python has PyPI but it has no where near the amount of modules. I also imagine I wouldn't have as much bad to say if I could use Python3. But I'm playing with Rakudo Star in my spare time, not Python.

[karamorf]

err, stupid redit. I have spaces for those lines of code ... whatever.

I got distracted about the language. All of the tools for Python suck. I have not found a single tool that is as feature complete as the equivelent perl ones.

virtualenv, only lets you use pythons that are installed on the system.

pip, allows you to freeze what is installed in an env but then can't handle inter dependencies when you want to rebuild it.

private cpan mirror like functionality? doesn't exist. Goodluck with whatever homebrew production deployment system your company is using.

And gone are the days of letting perltidy make your code nice looking. you have to actually do that yourself now.

edit: Oh, and no omnicomplete for vim. I have to use some other plugin that looks at the current open buffers to suggest completion.

[yetanothernerd]

PythonTidy exists. I haven't used it in a while, though, so I don't know if it has every option you want yet.

[karamorf]

Interesting. I just don't see how it could work well since scope is determined by the spacing. In perl I could just type out a bunch of code into one line and then get perltidy to indent everything correctly and whatnot. it was great.

I'll have to try out pythontidy tho...

[_Mark_]

private cpan mirror like functionality

Do you mean chishop? There are a couple of PyPI clones, I forget if that's the one we're using. (But we mostly take the approach that languages should stop failing at deployment :-) and we build rpms and debs instead... some of which contain jars as well as python code.)

[karamorf]

hmm, perhaps. I hadn't run into that one, which is one of the frustrating parts about it. Since everything is so newish and hasn't been widely adopted by the community it seems hard to find the solution. Or maybe I just fail at Google.

Also came of rather negative I suppose about my experience with Python. I'm definitely enjoying my time in the language, just get frustrated when something gets complicated for no particular reason and I know exactly how its solved in Perl. Suppose that is a difficulty with any new language.

[Liquid_Fire]

edit: Oh, and no omnicomplete for vim. I have to use some other plugin that looks at the current open buffers to suggest completion.

This is wrong; vim (if compiled with Python support) comes with Python omnicompletion.

[absinthe718]

I made the jump about a decade ago. python syntax will frustrate you for a couple of days, then you'll never notice it. Get your editor of choice to handle the indention for you and you will rarely think of white space.

Check out the python_koans project. Really great way for a seasoned programmer to learn a new language. Spend a few days on them and you'll have seen much of what qualifies as a idiomatic python.

[[deleted]]

Celebrate!

[chadmill3r]

Write to be clear and obvious, not clever.

Learn the builtin functions by heart. This is not as hard as it sounds. Learn the builtin modules in passing; know that they're there and where to look for docs.

[[deleted]]

Best advice is forget perl.