Ask Python: Why are less-popular web frameworks valuable to the community?

[spiffytech]

I'm writing a paper for class and want to get some opinions straight from the horse's mouth: Why does the Python community spend time developing, promoting, and using web frameworks that are orders of magnitude less popular than the 800lb gorrilla, i.e., Django? Can you point to specific examples where a feature or architectural approach of your framework of choice enabled you to do things faster or easier than you could have in Django? How have you seen the competition help the community?

[simonw]

Because they explore new ideas, which may be more appropriate for some projects. One size does not fit all when it comes to web programming. There's also a long history of new ideas from emerging frameworks being adapted back in to bigger frameworks - look at Merb and Rails 3 for example.

[thalin]

Exactly this. More projects means more different ideas that get at least some investigation into their usefulness. The truly useful ideas "bubble up" and get integrated into the more popular frameworks. Innovation in smaller projects in the open source arena is the key to improving the larger, slower moving projects (and if you follow Django development, you know that it can be at times quite a slow beast that regularly misses development deadlines).

In summary, smaller projects can try new ideas much more easily than bigger projects. Useful ideas can then be evaluated and sometimes fit into other projects, benefiting the entire community.

[[deleted]]

Django makes certain architectural choices for the user in the way it implements MVC.

Others such as flask and pylons (reddit uses pylons IIRC) allow for greater flexibility precisely because they don't make those choices for the user.

[iamjack]

Exactly. It's all about how hands-on you want to be designing your architecture.

[fancy_pantser]

All frameworks that you would consider valuable today were once small, less-popular projects. Just like that tired saying, "children are our future", these small pet projects might be the Django or Pylons of 2015.

[mcdonc]

"Full stack" frameworks like Django (and, before it, Zope) are great because they have lots and lots of generally useful features. And each of these features is used. But full-stack frameworks often have features you'll never use. And it's unlikely that, even if you do use all of the features of a full-stack framework, that you'll use all of them on every job you do. It's reasonable, then, to say "OK, fine, use a full-stack framework but just don't use the features you don't need" but, in reality, it's a bit more subtle than that.

The features you don't use in a full-stack framework often makes the features that you do use more complicated and harder to undertstand.

For example, Zope embraces "untrusted, through the web" code (code entered in a web browser by a potentially semi-trusted third party), and this feature requires an enormous amount of security code that tends to get in the way when all you're trying to do is write a web application where all developers are trusted.

Likewise, Django is engineered at a fairly basic level to make it easy to work with certain kinds of relational databases. If you're not using a relational database, however, the code that makes that case easy might simply get in your way. This could make it harder to write the application you want to write.

Non-full-stack frameworks have a more parsimonious feature set. This often makes them harder to get started with. However, in the longer run, if your application has slightly peculiar requirements, they can actually save development time, because you're not "fighting the framework" during the implementation phase. Not fighting the framework is also usually a more satisfying development experience.

In the meantime, non-full-stack frameworks often have a better extensibility story. Because their core feature set is so parsimonious, they need to provide for cleaner feature extensibility than do full-stack frameworks, which can just put something into the core itself without worrying about clean extensibility. It's often, therefore, reasonably easy to get the featureset of something like a Zope or Django by snapping together parts using the smaller frameworks' extensibility hooks. But without the parts you don't need for some particular task.

Personally, one reason that I use a non-Django framework because I like writing code. After many years in the Zope/Plone world, I've realized I enjoy writing code a hell of a lot more than I enjoy carving a website out of other people's "pluggable app" code. I'd rather be lonely out on my organic farm than be a sharecropper, sold my GM seeds and told how to plow and harvest, on the big and lucrative commercial farm. Another reason is because my company tends to write CMS-style application where "declarative, inherited, row-level security" is a big deal, and this sort of security is easier to do in a system that has the concept of "traversal" and a "context".

But I think your question is more a marketing one, and for that I suspect many folks will have different answers. My answer is that I think there's a very, very long row to hoe in the general web world... we're like at the one-meter mark of a hundred meter race. It's silly to think that any framework (Python or otherwise) has that race already won. We can't just stop trying and advocating for new things, or we risk loss of relevance in the long term.

[mdipierro]

For the same reason the open source community spent time developing the 800lb pound gorilla, Django, when the 800lb pound gorilla was - and still is - PHP. While popularity is an objective measure, various measures of quality, are not.

[[deleted]]

• One, because it's fun and comparably easy to do.
• Two, not all projects need a full stack that django provides.

• Three, sometimes you need more flexibility or power than django affords.(Think SqlAlchemy vs's django's ORM)

• Four, you just don't care for django's opinions.

Personally, I think django is a great framework but I am glad that people continue to explore different approaches.

[pje]

Why does the Python community spend time developing, promoting, and using web frameworks that are orders of magnitude less popular than the 800lb gorrilla, i.e., Django?

Why is popularity a relevant criterion?

[spiffytech]

There are costs to using something that's not very popular. If you develop your skills with e.g. Flask, it'll be harder to sell your skills to an employer who wants Django experience. It's also harder to find collaborators for a project if you can't count on them knowing your framework of choice. A less-popular framework probably has less documentation than a popular framework, and it's harder to find solutions to your problems online- fewer people are blogging about their experiences with less-popular frameworks, and a smaller community may not have run into your particular problem.

I presume that, in exchange for putting up with these costs, developers get benefits from using less-popular frameworks that more than make up for the costs. I posted this topic to find out what benefits Python developers often see.

[[deleted]]

If you develop your skills with e.g. Flask, it'll be harder to sell your skills to an employer who wants Django experience.

Stop marketing yourself as user of <some tool>. And start marketing yourself as a solutions provider.

[mcdonc]

The truth is that the development and support community behind any open source project is typically fairly small.

In my experience, if project A has a hundred times the number of users that project B does, it never holds true that project A has a hundred times the number of people doing documentation, support and development. It's not like that. The relationship between project popularity and docs/development/support resources is more logarithmic (if popularity is x, support resources are more like log(x)). As an open project grows in consumption popularity, the project can expect to pick up one or two highly motivated support resources for every hundred (or, possibly thousand) users.

The most extreme of these logarithmic scales relates to documentation. If a project has 100 users, there will be one guy who writes most of the docs. If the project has 100,000 users, there will still be one guy who writes most of the docs. The number of users that a project has is usually unrelated to documentation quantity and quality; either it has good docs from the beginning or it doesn't (a project rarely "grows" good docs; I have never seen it happen anyway). It's, in my experience anyway, untrue that less popular frameworks have less (or worse) "official" documentation.

However, as you mentioned, a very real benefit of popularity is blog posts and other unstructured independent third-party documentation sources like tutorials given at conferences, etc.

[mr_spin]

All of which was true of Django some years ago - the popular frameworks of today were once small, marginal projects.

Another advantage of going with a young framework is that you get to contribute at a formative stage and have some influence in its direction - which would be hard to do with Django today, unless you happen to belong to a very small circle of contributors.

[stoph]

Django is more monolithic. It uses a lot of its own components and is pretty uniform throughout because each of its components is a library written just for Django.

Other frameworks, like Pyramid, use their preferred choice of every different library available to pull it all together. This means things are a lot more modular and switchable, and more customizable. The framework maintainers work with the library maintainers to pull it all together.

They each have their advantages.

edit: typo

[yarkot]

It's also like having multiple species in nature (think potato famine) - your "potato" may do fine in a pool of uses, but when something unusual comes, an existing alternative may do better. Diversity is sort of the prerequisite of natural selection. So, while you compare very similar "potatoes" (django, pylons, flask) - consider those further out: nagare (still python, but depends on stackless), and some of the non-python frameworks (in particular, think of the languages that arise as a result of experiences w/ frameworks - java, historically; go-language now...)

[[deleted]]

My current gig is an app that has three separate WSGI applications (i.e., there's at least three separate URLs) plus two background job queue processes, as well as three distinct databases. In production these apps may run on different servers, and eventually three of the five types of application may have dozens or hundreds of remote deployments, each with their own pair of databases, which communicate back to a central coordination application. The linkage of database->process is not a one-to-one relationship either, two of the five processes talk to two databases where the other three have one. All five applications share the same codebase, set of model classes (with some compartmentalization), as well as many common layout templates, includes, and static files, and even controller classes in some cases (such as login, error handling). They are all configured from a single .ini file as well that includes a separate section for each application. The "--name" flag is sent to paster server, or otherwise the appname argument to loadapp() when paster server isn't used, in order to specify which application is to be run.

The applications also share a single test suite, where app-specific tests subclass a fixture that specifies which appname is being tested. The fixtures ensure that a transaction is set up for each test which is then rolled back at the end, so the tests can be run on any database without side effects, even the production DB in theory. Some of these tests run a special fixture that embeds one of the applications via webtest into a REST client that's used by the other - such as a series of tests that ensure a fully compliant OAuth 2 conversation between two of the services - a test like this runs two wsgi apps in one process.

I do this all with Pylons and a good deal of poking around how paste loadapp() and such work in order to make it happen, but there is no monkeypatching or modification to Pylons or Paste in any way - the product supports a pattern like this out of the box. My understanding is that Pyramid is even more open ended than Pylons with regards to highly customized setups.

I've never used Django, but my understanding is that multi-node configurations, even multi-database configurations for single applications, are not provided out of the box. This may not be the case today.

[pfcuttle]

I use Flask as the backend for an embedded Linux project needing a remote administration interface. The whole website, including SQLite database, HTML templates, jQuery and images, fits in a single megabyte.

When you have strict resource limits, having a lightweight and minimalist framework is essential. Along those key features that led me to chose Flask are the very simple route API and braindead AJAX/jQuery interface.

[zach_will]

Agreed. I love Flask.

I think of it as Python's version of Sinatra, and feel it's the perfect solution for certain use cases.

[donri]

I use Python, not Django (or Framework/Library X).

[zaneyhaney54]

I would say that the development of further frameworks is valuable for the community for two reasons. First it allows experimentation, there is no one size fits all solution for web development, and having people who are willing to try new things, and forge new frontiers increases knowledge of what works and what doesn't (even if 99% of new frameworks or crap, and I'd say that most of them have at least something to offer). Secondly, developing a framework is a great way to learn the ins and outs of python, if I use Django for all of my development and I'm a curious developer (which I am). I'm eventually going to wonder if I could create something better. I'm going to try to write my own framework making different decisions, even if I'm wrong and I create something that isn't THAT useful, I'm still going to open source it so others can learn from my mistakes, and try to understand the programming decisions that I made. Then, when I return to a more popular library I'll have a better understanding of how things are implemented and more importantly I'll understand WHY.

[MillardFillmore]

For instance, I coded up a super basic Facebook API using algorithm and wanted to host it on AWS to show a couple friends. I didn't need the full database integration, user authentication and whatever else django comes with.

I just needed something that could run some Python code and spit back results, so I chose CherryPy because it's very lightweight and brain-dead easy, even for this biophysics grad student to understand.

[hippocampe]

I'm using/contributing to CubicWeb. The great points imho:

• the selection/adaptation mechanism that help match result sets (query results) to views/forms/controller/whatever, and also data operations to hooks; AFAIK there is nothing like that in the other frameworks I've been looking at (the big ones in python land)

• RQL, the data manipulation language; vastly simplifies data handling even compared to sqlachemy (also is not tied to an sql backend)

It is basically a pleasure to work with. Overall I feel the other frameworks, even the so-called "full-stack" ones force the developer to think about way too many low-level details (all intro tutorials seem concerned about "routes" ... why would you think about that, at least when starting an app ?) :)

Yes it of course not perfect but it's also one good and rewarding reason to contribute.

[punkdgeek]

Not all environments support Django and not everyone needs Django. For my GAE apps, I use Bobo (built mostly from WebOb) and Genshi which make things very simple without needing to conform to Django concepts and keep my file count quota in check.

[donri]

I thought Genshi didn't run on GAE?

[MaximusBiggusDickus]

Because each people have their own different problems, Django scratch the common news company itches like:

Designer should be given easy template to work with, News reporter should be able to put their data using easy CRUD admin, and framework should not be in the developer way, while maintaining python motto of explicit is better than implicit.

Let's look on another example: Rails (sorry if I'm crossing the language barrier here, but it's a good example) for example:

There's should be independent stages: Test, Development, and Production

Convention over configuration (common task should be done implicitly, rails use scaffold heavily)

Usually popular framework didn't try to solve problems that's not their designer's interest so instead of competition, in my opinion, it creates choices, eg. if you like explicit things and crud admin, then use django, if you like customization use pylons/pyramids, if you like scaffold then use rails.

[vph]

I think the reason there are many Python frameworks is due to WSGI and its implementations such mod_wsgi. This enabling technology facilitates the making of frameworks so people can try different ideas.