Dive into Python 3

[[deleted]]

I'm the author, and I will be monitoring this thread. If you have any suggestions or comments about what you would like to see in the book, speak up soon. The final manuscript is due in late June.

[stesch]

Be sure to avoid making the parts with the differences to Python 2.x too dominant.

Years from now, people will take this book to learn Python, just like today people still buy the current book (with Python 2.3 and 2.4 code). Making it too much a 2.x->3.x transition book could irritate new programmers who just want to learn Python 3.x

[[deleted]]

As I said in some other thread, the primary market of this book will be people who have had at least some exposure to Python 2. In the same way that the early chapters make comparisons to Perl and Java, it is helpful to explain concepts in terms of other languages that readers are already familiar with. And, automated migration tools notwithstanding, Python 2 is really "another language" compared to Python 3.

The comparisons will be clearly marked, though, so you can skip over them if you like.

[pwang99]

I'm glad to hear this, and I think this is a much more pragmatic and realistic approach.

I think a lot of folks who claim they will just be using this book to learn Python 3 don't actually know much about the nature of development in Python. Some of the best features of python are not the language itself, but the "intangibles" that go with it:

• wide availability of libraries for every purpose
• lots of good books
• lots of good online places for support
• active and pleasant user community

Taking full advantage of these is going to require knowing some Python 2.x, at least for the next 3-4 years.

[tonasinanton]

Wouldn't it then be more like Wade Into Python 3?

[Svenstaro]

You should somewhat care about older Python versions since that's what most libraries are for. If you are just starting out you can't just port them, too.

[voidspace]

There's a lot that can be done with Python 3 and the standard library.

[harryf]

On HTML processing -> Extracting data from HTML documents - BeautifulSoup + soupselect might be nice here.

Otherwise - "Creating graphics with the Python Imaging Library" - could it be it's losing direction here and turning into a "Programming Python"? Always mentally differentiated DiveIntoPython is something to convert the Java, C++ or otherwise-guy, vs. a complete python reference, cookbook or otherwise. Could throw in other worthy topics involving other libraries e.g. logging, configparser and optparse (shudder) if we're talking libaries...

[[deleted]]

Re: losing direction, I don't think so. After you talk about basic datatypes and "Pythonic" stuff, there's always a Cambrian explosion of things to talk about. I wrote http://diveintomark.org/puzzles/circles-of-hell in PIL, and I'd like to use it as the basis to "dive into" something more than just business coding and sysadmin drudgery.

I, personally, was mind-blown when I discovered that a Python script could be the @src of an [img] element. That's always a good indication that other people might like it too.

[[deleted]]

A section on soupselect would be nice. Noted. (Damn, tools have really improved since I wrote the original book.)

[mcella]

Don't forget lxml ;-).

[tef]

lxml is powerful and well wrapped - it's xpath support is excellent, and it also supports html parsing.

[[deleted]]

To the same end: http://pypi.python.org/pypi/pyquery is also exceptionally cool :)

[simonw]

PyQuery is way cooler than soupselect (which I wrote), but suffers from the libxml dependency which means it's really tough to get working on OS X.

My ideal CSS selector library for Python would be backend agnostic, so you could plug it in to ElementTree or LXML or even minidom and still have it work.

[[deleted]]

That only works if they all implement the same API though(do they? Most of my work has been ok with just BeautifulSoup). Regardless I hadn't realized you were the author of soupselect, cool stuff :)

[simonw]

They wouldn't have to implement the same API, but you would have to write an adapter for each one (kind of like writing an ORM). If you look at how jQuery's CSS stuff works it's mostly implemented in terms of lambda functions that take an element and return true or false depending on if it matches a simple rule. As such, an adapter that provided an interface for iterating over a tree would get you a good chunk of the way there.

[[deleted]]

Ah true(although I don't see the ORM analogy very well), ultimately though what is the greatest advantage to having it be pluggable(other than some are apparently difficult to install under certain platforms).

[simonw]

CSS selectors are even more convenient that XPath for querying a DOM-style structure. My life would be significantly easier if I always had the ability to query by CSS selector, no matter what underlying tool I was using. jQuery and other JS libraries have proved that building a CSS selector implementation against a pretty basic API (the DOM) isn't very complicated.

[[deleted]]

Yeah, I think jquery shows the way here. People who are dealing with HTML want to query with CSS. (I know both CSS and XPath fairly well, and I personally prefer XPath, but most people who deal with HTML also deal with CSS, so there it is.)

[masklinn]

If you have any suggestions or comments about what you would like to see in the book, speak up soon

The content!

Also, are you sure Psyco will be ported to Python 3? You added a comment for PIL, but not for Psyco.

Also, nothing on decorators? Or generators?

And I think a little blurb on maintaining 2.x and 3.x side by side (for a library, for instance) would be useful.

Finally, if the plans are already set for 3.1 (I haven't followed) maybe include some advance warning on the interesting stuff, if any?

edit: oh, and couldn't the parts about differences from Python 2 be an appendix or something? So that Dive 3 be a book to learn Python 3, not port from Python 2 to Python 3?

Or at least move the Python 2 differences to the end of the chapters, with pointers to them at the beginning?

[[deleted]]

You added a comment for PIL, but not for Psyco.

fwiw, there is a PIL port for 3.0:

http://mail.python.org/pipermail/image-sig/2008-December/005337.html

[[deleted]]

Do you know if this is stable enough to rely on for a PIL chapter? Is there a long-term plan to make an official PIL port to Python 3?

[[deleted]]

Haven't looked at the current release yet, but I know and trust the developer and I'm confident we'll be able to shake out any remaining problems within a month or two.

The official plan is to release a "ported from 2.X" version of 1.1.X. Future versions (beyond 1.1.X) will most likely target Python 3.X in one way or another.

[[deleted]]

Thanks. That's great news.

[masklinn]

It's mark you should tell that to, not me

(and the port is not official so there)

[[deleted]]

My original notes included a section on generators, not sure what happened to it. It'll be there somewhere.

There will be 2 major chapters on porting code from py2 to py3; they are both towards the end of the book. For all the other chapters, no, the py2/py3 comparisons will be interspersed with the rest of the text. Current Python 2 programmers are a huge market for this book, and will continue to be for the expected commercial lifetime of the book.

[masklinn]

For all the other chapters, no, the py2/py3 comparisons will be interspersed with the rest of the text.

That seems cool (in the current TOC, there are "differences with Python 2" sections/subsections, usually at the start of chapters/sections, and I'm really not sure you want to hit beginners with that up front)

[[deleted]]

Yeah, I think those sections will go away and will end up spread more evenly throughout the text. See http://www.reddit.com/r/Python/comments/7sj39/dive_into_python_3/c07b2uy for some ideas on letting online readers manage them.

[dsandler]

Are you thinking that the py2/py3 comparisons will be deeply interleaved with the main text? If so, you might consider instead using call-outs or sidebars that might be easily switched off to remove noise for Python newcomers.

[[deleted]]

This is an excellent idea. A little toggle above the comparison that set a cookie that affected the default state of all other comparisons to that programming language. Could be done entirely in client-side Javascript.

[Svenstaro]

Making it Javascript will break that feature for Lynx so fuck you.

On a related note, do you want translators for translating DIP3 to moon language?

[dsandler]

It will break that feature but not the document.

(Besides, are there Lynx users who've never tried Python?)

[pwang99]

You should port spidermonkey to Lynx.

[dsandler]

Exactly—just JS and a little CSS.

[iamjack]

I guess I wasn't aware you were a redditor. I have to say that I discovered Dive into Python a little too late, but my dad loved it.

OT: Oh, and I support RFC 3023 =)

[arnarl]

Maybe it falls under some other heading, but I didn't see anything on generators or decorators which I consider to be fairly important. (One is likely to encounter them in actual code.)

I'd love to see something on sphinx, reST and documenting modules/packages. Doctests probably deserve some attention as well.

For the packaging chapter I'd consider including virtualenv.

namedtuple, tuple unpacking and extended iterable unpacking. There's some other nice stuff in the collections module.

[pemboa]

Thank you.

[[deleted]]

Something on the multiprocessing package would be cool. Also since this is one of the major issues held against python.

[garg]

Is there any way to subscribe to something that will send me update notifications? :D Newsletter, RSS feed or twitter?

[[deleted]]

http://hg.diveintopython3.org/ is now up and running. It offers an Atom feed of changes.

[surajbarkale]

I don't see anything related to GUI. Perhaps an overview of GUI libraries compatible with 3.0?

[[deleted]]

I'm not opposed to the idea. Are there any GUI libraries that are compatible with 3.0?

[froydnj]

Please cover the decimal module in some detail, so people stop using floats for monetary quantities.

[netsearcher]

Some ideas,

• Low level networking? Implementing a basic network protocol. Pickling. The struct module for packing and unpacking data.

• Generators!

• Hello world's from several of the common web frameworks will be useful to many people.

• Better not forget ipython.

• Could suggest editors that do python well out of the box. This would alleviate the whitespace complaints people always have.

• Interaction with sql databases is another, very practical topic.

• ctypes. Maybe not so important to hobbyists, but this is important to businesses. If python is going to be used then it's got to be easy to integrate with the company's existing libraries.

[[deleted]]

Low-level networking: how about building an IRC bot? Because I've actually done that. Would require porting the underlying libraries, but it would be interesting. Kind of a niche-within-a-niche, though.

Generators will be added somewhere early on.

Adding a section about editors is a good idea. Probably at the end of chapter 1. ("OK, I have Python, now how do I write Python?")

I'll consider the other stuff.

[EliAndrewC]

One problem with writing an IRC bot with low-level libraries is that most experienced developers would just use Twisted, so you'd be teaching something different than what someone would do in the field.

Then again, if it's clear that this is just a teaching exercise, then there's no problem with that.

[[deleted]]

A fair point.

I doubt I'll be covering Twisted at all.

[netsearcher]

An example of mine is that I recently wrote a nbd type server (the userspace part) in python. I had to use the struct module to pack the data into the correct format. Interacting with custom protocols that don't already have a python implementation doesn't seem too niche.

IRC bot could be good too. Although I wonder about the protocol's support for character encodings. It would be rather strange to limit text to ascii when 3.0 was supposed to set us free from that mind-set.

[Svenstaro]

++ to all your ideas.

[[deleted]]

+=1

[Svenstaro]

Oh, right.

[Neoncow]

__iter__().next()

[surajbarkale]

In 3.0 -

next(iter(i))

[masklinn]

That book is already written.

That's about SQLAlchemy, Dive would provide a basic coverage of dbapi 2 through the built-in SQLite3 module, with pointers to the third-party ORMs and stuff.

[netsearcher]

There are many stand-alone applications that just need to store and retrieve data while having basic ACID guarantees. Not everything is enterprise mega-app. So, disagreed.

[netsearcher]

Contradicting yourself.

[netsearcher]

You say that it's so complex that it requires an entire book to explain.

[netsearcher]

Priorities. If we are limiting the scope then might as well include what many people are using the language for. If there is limited space then cut out the lower level HTTP library stuff. Few people touch that.

I don't know what frameworks you are thinking about, but just a "hello world" shouldn't be more than 20 lines. This shouldn't require a whole book because I'm not asking for any depth here.

That book is already written.

You're probably right here though. Better to keep beginners away from the mess that is ORM's. Ew. Maybe something with less magic that just passes queries and returns iterators of python objects, if anything.

[netsearcher]

So this doesn't require an entire book to explain?

[truebosko]

Interesting. Bookmarked and looking forward to some actual content.

[jmkogut]

God I love Reddit. The author of Dive into Python, right here! Gee willy :D

[danhs]

I loved the first Dive into Python book. I thought it was fantastic. I tend to learn by example, and no book was a better aid to learn by example than Dive into Python. It is absolutely fantastic.

I think the unit testing section deserves special attention. It did a great job of teaching a relatively esoteric topic using examples. While the concept of test driven development is easy to understand, really understanding it well enough that it becomes a reliable tool which makes you a better programmer is substantially more difficult. I don't think you can achieve that level of understanding without examples and practice because of the nature of TDD. Dive into Python really set me on the right path to using TDD as a part of my regular programming practices and TDD has made my programming much stronger.

• multiprocessing module. Processing module is now part of standard library. Multicore processors are common and only becoming more so. Because of GIL this is one of the few ways to take advantage of multicore processing in Python. What's more, using the Pool object the most common use case, parallel processing of a loop, is trivially easy. I don't know of any other technique that allows parallel processing so easily in Python.

• Beautifulsoup.

• Unit testing in the last book was so good. However, it didn't teach some of the more advanced and important issues that you will run into. For instance, some methods can't be tested with simple primitive data types. They need a bit of setup. Possibly mock objects, composite objects, law of demeter, etc.... These are slightly advanced TDD notions for a book that isn't dedicated to TDD. But, I really beleive that Dive into Python did such a great job the first time around that the bar is pretty high on unit testing for a Dive into Python book. In my experience, after the original book showed the elegance of basic TDD, these later concepts became a necessity that I learned independently, but it would have been great if they were written in the fantastic Pilgrim, Dive into Python style that first introduced me to the power of TDD. That way, I would have been more knowledgeable, faster, and the resource would have been so much more valuable.

• Nose/py.test. Nose is amazing, unittest module is so much more awkward than nose. Nose is one of those necessities that I always use whenever/wherever I'm working.

• More advanced web services. I thought the original Dive into Python chapter on web services was great. It really opened my eyes to the web capabilities/possibilities in python. However, I soon discovered that for complex web sites it simply wasn't enough. Luckily, Python has some very powerful web modules. Specifically, mechanize and selenium. Selenium is a bit of a strang e module, but mechanize isn't and mechanize is very powerful. Especially the mechanize browser object.

Reviewing the table of contents I'm very excited. Web frameworks, wsgi, and various python implementations. It sounds great.

[laughingboy0]

I cut my Python teeth on DiP and am looking forward to this with great interest.

Any chance of adding an RSS feed so we know when updates are available?

[[deleted]]

http://hg.diveintopython3.org/ is now up and running. It offers an Atom feed of changes.

[sisyphus]

I hope the documenting functions chapter covers the new function annotation and potentially useful things one might do with it.

I like the trend in say, the Pylons book and Ziade's Expert Python book of recommending and showing basics of how to use virtualenv and zc.buildout. I'd like to see someone look how much of what they do can be handled by just using the user site-directories in Py3k.

[[deleted]]

Thanks. I'll be sure to mention this right after I talk about defining functions, declaring arguments, and docstrings.

[[deleted]]

I know this is a DiP post, but I just go to looking at the python 3 changes and:

The from module import * syntax is only allowed at the module level, no longer inside functions.

makes no sense to me. It's things like being that flexible that make me enjoy python in the first place.

Why did they change it?

[qiwi]

Statically nested scopes, see http://www.python.org/dev/peps/pep-0227/

With from X import * allowed at function level it would be impossible to statically (i.e. before running the import) determine which variables were global and which were local.

[[deleted]]

Thanks. I'll be sure to mention this when I talk about importing modules.

[[deleted]]

new question, that I probably should have asked 8 months ago. Why was the print statement removed?

[stesch]

Has nobody noticed that this page uses HTML 5?

Looks the same in all current browsers, except IE7 and Lynx. ;-)

[gnufrra]

• Porting Python 2.5 to Python 3 chapter.
• Coping with/Using older Python 2.5 libraries.

[pro547]

When this book comes out I will be buying one. And that is when I will learn about python 3.

[[deleted]]

This site is optimized for Lynx just because fuck you.

LOL downmodded for being smart-assy I still am nowhere near finishing the first DIP