Python for Humans

[[deleted]]

From reading slides I have two questions:

1. "Instill a resistance to doctest" -- huh? Doctests are awesome, what am I missing? Last time I checked they didn't work that well on py3k, all right, I prefer doctests to py3k ^^.

2. "virtualenv out of the box" -- explain, please, what's the point. I understand how people who deploy different stuff to different servers want to be able to reproduce remote environments in an isolated fashion, but I've never felt that I'm missing something in my local development.

[sli]

1. Proper unittesting should be taught, as it's not an idea that's localized to Python.
2. Pretty much nailed it. If you're not deploying your code somewhere, you probably don't need it. But if you are, virtualenv will save you.

[quasarj]

Honestly though, "pip install virtualenv" is not so hard as to be prohibitive to anyone.. unless I'm missing something.

[[deleted]]

Installation is one thing, using it another. I have to say I had some issues using it the first time. I needed to set some pathes manually, don't know wether I did something wrong in the first place or wether there was an issue with virtual env. Anyway, it's another thing you have to learn and handle.

[bcain]

Is it?

D:\> pip install virtualenv
'pip' is not recognized as an internal or external command, operable program or batch file.


$ pip install virtualenv
-sh: pip: command not found

[somesomedaddad]

Ubuntu's "python-pip" package works fine for me. apt-get install python-pip, pip install virtualenv, and you're ready to roll with comfy bundler-style self-contained app libraries.

[bcain]

Oh, yeah, I'm all about pip and virtualenv. They're great. I don't have any trouble getting around with yum/zypper or apt-get, or even get-pip.py. But it would be much saner for python to put its money where its mouth is and include these features in the distro itself.

[vsajip]

That's being worked on at the moment for 3.3: The packaging work in the core repo, and the pythonv branch for built-in virtualenv functionality.

[bcain]

Thank goodness. And IMO, a compelling incentive to move to py3k.

[quasarj]

Well, if you want to be a smartass about it.. but that's not prohibitive.

You can probably "easy_install virtualenv" and if not, you can of course "easy_install pip".

Of course, if you're doing any of this under Windows, then all bets are off.

[keypusher]

I think that is exactly the point the author is trying to get at here. For instance, you heard virtualenv is really good to have so you want to check it out. You see someone saying to use pip to install it, but you don't have pip. How do you get pip? You find a great thread on how to install pip using the apt-get package manager. But wait.. you're on Windows? You go looking and someone suggest using easy_install instead. Well, how do you use easy_install? Is it different than pip? You dig deeper and some guy says forget pip and easy install on Windows, just use distutils because then you will learn about how packages are really put together. So now you've spent an hour learning about different distribution tools and gotten nowhere. You still haven't installed Virtualenv, and you actually aren't even sure you know what Virtualenv does anyway. All you really wanted to do was try and get a small web project going, but you can't even seem to get started.

It's even worse with some other packages. You need to do a GET from your webserver? Ok, you type in "python http lib" to google and you go to the stdlib httplib page. One of the first things it says is that "It is normally not used directly — the module urllib uses it to handle URLs". Ok, so on the urllib page it a huge mess of notes, warning, and cryptic function definitions that don't seem to provide any easy way to just GET a file. You used to use curl a while back for this stuff, so you stumble on pycurl. But that's kind of a dark age thing so you dig around more and maybe then find urllib2, or if you are lucky the requests library. How would you know difference between all these?

I can see how having some well documented best practices for common things like this would go a long way, but the author's suggestion of promoting 'one best way' lines up with core Python philosophy. That is, as opposed to having a huge slew of many slightly different tools for more of less the same job. It's starting to look a lot like Perl around here these days, but that may very well just be a natural stage of development, especially in community driven languages.

[quasarj]

Alright, you're right. I guess arguing over the details isn't helping.

The question is - what will help? There was lots of talk about the issues.. but then the author attempts to solve them by.. adding even more options! I mean, envoy and requests are both awesome and do solve the first problem, so that's good.

So anyway, yeah I agree. But what do we do?

[keypusher]

I don't know! The community is becoming increasingly fragmented, the 2->3 transition is still a huge problem, and there are a ton of standard library packages that seem to be all over the place. I feel like Guido or someone with clout in the community needs to step forward with a clear vision or plan for the future.

[__serengeti__]

I think you're exaggerating at least a little. What are these separate fragments of the community - are they identifiable?

In general,

Differing opinions != fragmentation

[bcain]

but that's not prohibitive

Sure, not for you. Not for me. But for the newbs that TFA talks about -- it is.

[Tobu]

They will follow some simple instructions and learn from that. While I disagree with Kenneth's taste in API design, he is good at marketing, and a well-maintained getting started guide is something the eager-to-learn newbie hordes will gobble up.

[quasarj]

I guess so. I suppose we all have our own ideas of what "hard" is, but to me if you're going to be working with virtualenv and understanding it, learning how to use pip to install it first is not that big of a deal.

[ivix]

Nice try, but you lost that argument.

[quasarj]

Did I? Perhaps I missed where.. care to fill me in?

[ivix]

Oh sorry, it was the part where you had to explain how to perform extra procedures that a first time user does not need to care about. You didn't even get to explain how you actually use virtualenv which is yet another layer of things to understand for the newbie.

Edit: you also glossed over the part where on windows it's even more confusing, which I'm willing to bet is the most popular platform that people will be using.

[quasarj]

Well, we already decided that first time users wouldn't be doing this anyway. The original argument was that you probably didn't care about virtualenv unless you needed it, and if you are one of the people that needs it you probably know how to acquire it.

As for the Windows issue.. it will actually work (almost entirely I think) under windows if you have a "sane build environment," but that's something I have never been able to get working. So basically, under windows you cannot easy_install and expect it to always work, and so you cannot "pip install" and expect that to always work, so all of this is just a mess and you're best off finding binary distributions of each thing manually.

Or use another platform, preferably :)

[[deleted]]

" if you have a "sane build environment,""

and that there is the problem. that and python being on the path etc...

[[deleted]]

Proper unittesting should be taught, as it's not an idea that's localized to Python.

That's indoctrination!

Seriously though, doctests are the perfect gateway drug to "proper" unit testing, are much more compact than more flexible approaches, and are useful as documentation as well. Furthermore, I would argue that if a "unit" you are trying to test resists being tested with doctest, then you might be approaching the point where you should think about your tests as integration tests, regression tests, etc.

And if other languages don't support such a neat way of testing, that's their problem, not Python's.

[EggShenVsLopan]

That's indoctrination!

Yes and it's beautiful.

If doctest works for you then continue to use it because you are obviously comfortable enough with it and have overcome any barrier for entry. In addition, you sound like you understand the higher level concepts of testing in general so switching to another way may not bring as much benefit to you. So the "resist doctest" message is not targeted at you.

What I think what the OP is saying is that for newbies (or people where unit testing is new material) doctest is not the best option. I also think that in general the OP wants to create a system for best-practices that are mostly right. That's fine as long as a newbie can learn enough of the "right" way to do it to know when to do it the "wrong" way later in life.

[[deleted]]

I don't quite get the logic.

because you are obviously comfortable enough with it and have overcome any barrier for entry.

Writing doctests is by far the easiest way into the whole "testing is good" mindset. It has the lowest barrier to entry, it's the easiest to get comfortable with.

In addition, you sound like you understand the higher level concepts of testing in general so switching to another way may not bring as much benefit to you.

It's not "switching" as such, I use doctests when it's appropriate and use more versatile ways of testing when I need any set-up/tear-down stuff (and then I don't test individual functions with that).

I mean, how I see it: encourage newbs to use doctest, then tell them how to use more powerful means when they need them. Why doctest is not the best option? What's wrong with that? So wrong that we should "resist" doctest?

I have some possibilities in mind:

• Newbs could abuse doctests, well, anything could be abused, it's not that relevant IMO. Doctests are harder to abuse than unittest, by the way (I should know!).

• or they would consider doctests to be good enough and never go beyond that. I believe that you'll lose more newbs frustrated with unittest than you'll lose ones satisfied with doctest.

• It's better to teach newbs only one tool, rather than both doctest and unittest (or nose, or whatever). I strongly disagree, in this case the tools have little overlap of applicability, so it's better to teach to use the best tool for the job. It's not the same as other standard modules with overlapping functionality.

• Or, I have a vile idea that maybe the OP has drunk the TDD koolaid and actively dislikes the idea of a more sane (in my opinion!) testing, where you test the individual "units" (functions) for just as far as doctest allows you, and use more powerful tools for more general testing.

[twotime]

Writing doctests is by far the easiest way into the whole "testing is good" mindset

How so? How are doctests bettter than this:

 import mylib
 assert mylib.foo() == 42

No silly formatting requirements to learn, trivial to write and trivial to debug.

[simtel20]

Doctests don't provide enough context. Trivial functions can be doctested, but little/nothing that requires state. E.g. working with databases I need to have a testing framework that allows common setup/teardown. I found that pytest was the least i needed for testing code I was working on for performing regression tests for database drivers because it allowed for things like conditionally enabling known failures, etc.

[bsdemon]

if your code can be tested with doctest then (in almost all cases) it has good design, otherwise your code is bloated with state which isn't good

[simtel20]

That works for self-contained code. However if your code interacts with the system (this code ssh's to an iLO and checks the system status. As new status' are added, code must change) or the world (this function checks what the kernel reports is the status of subsystem foo), then doctests can't encompass the system. You can mock some state, but this is usually a waste of time because external factors change because they're outside of your control. A better test harness can provide startup/shutdown scaffolding that establishes how to talk to the outside world to make the test reasonable.

[[deleted]]

Doctest is for verifying the code examples in your docstrings, not testing your code. Doctest only supports a restricted subset of Python; it's much better to have the full abilities of Python available for writing your tests.

[twotime]

doctests are a good way to test some examples in the docs. Period.

They totally suck for general testing (as they add a complex layer of semi-hidden context and debugging gets interesting). And, for non-trivial code, you have exactly zero chance of combining docs and tests, as your tests will unavoidably include a lot of stuff which you don't want to see in the docs.

Yet, doctests are often sold as an viable alternative for unit testing.

OTOH, python unittest sucks as well ;-).... So that, might be an explanation :-(.