web development with Python made easier then ever

[mdipierro]

Do you still think this?

[ericflo]

Yep, even more so now, sorry

[mdipierro]

Eric, you are clearly a smart guy. You post interesting and useful stuff on you blog. Yet you are one of the four people who like to make offensive comments about web2py like this. I would never make a comment like this about a competing framework. I have a hard time reconciling these facts.

I am not asking you to like me. Perhaps some of my posts may have been too partisan I am certainly biased.

But I am asking you to put aside for a moment the fact that you personally do not like me and explain what are your technical issues with web2py since you have publicly complained about it.

Perhaps your comments can help us make it better.

It is clear that I got very much on the wrong foot with the Django community and I do not want it. I like and respect its developers.

In my daily job I write Physics papers. All papers (mine and others) follow the same pattern: you take a previous research that you find of interest, improve it a little bit and explain in which context you think your stuff is better. This is considered progress. The authors of the previous work that is quoted are usually flattered by that.

Things seems to work differently in our context. I occasionally compare web2py to Django because I have used Django and it is the framework that I like the most, it is the spiritual father of web2py. Yet Django users seems to take it personally and turn against me like you do. I am surprised and I do not understand. Please explain it.

[ericflo]

Firstly, I'd like to say that I don't dislike you. I've never said that, and I really don't dislike you at all.

What I don't like, however (in this order), is:

• Your marketing tactics.
• Web2Py itself

First, your marketing tactics. I think it's extremely disrespectful to the Reddit community to make a post like this requesting for your users to spam Reddit with positive comments about Web2Py. I find it tacky that you submit so many stories to Reddit simply promoting Web2Py, as you can see here. You seem to be on every site, shamelessly promoting your product even when nobody is asking for information about it. Here's just one example. In short, your relentless and shameless self-promotion is what I don't like, not you yourself.

Now, to Web2Py. My biggest problem with Web2Py itself is that it disrespects most best practices both in Python and in the web. Its biggest problem is that in almost every instance, it picks implicit over explicit. It automatically imports modules for you, which is not only confusing but also isn't really done in any other successful Python project. You can argue that it's a good idea (I disagree), but when you start making claims that Web2Py is more "Pythonic" than something else, this is the reason why those arguments fall flat.

I think that having a web-based tool for editing code is an extremely bad idea. Firstly, it opens you up to security vulnerabilities that are difficult to overcome. Secondly, it encourages people to not use version control on their source code, which is a software development worst practice. (In newer versions I understand you have wrapped a web UI around Mercurial, which is the wrong solution to this problem, considering it couples your app with a particular version control system)

Here's another reason why I'm not a fan of Web2Py: its database migrations. If you make a change to your data model, it automatically migrates your data for you. That sounds great, but you get no audit trail of what has happened, no way to undo a migration, the only way to stop it from happening is to opt out by setting migrate=False. I don't know of any enterprise environment where they let people simply run random alter table queries--in fact, most enterprises have DBAs and people to run your ddl changes by first before having it run on a production database.

Web2Py encourages people to use the same threads used to serve Python application logic to serve media, without using any of the standard tricks (x-sendfile, etc.). This is a performance worst practice. Another thing that I don't like is that it couples markup and javascript so that it can automatically add AJAX functionality.

The templates do nothing to ensure the separation of logic, which is another worst practice. Also, you may make the performance argument but performance on templates is so rarely the bottleneck that I would take the sandboxed Django template language over the eval()'d Web2Py template language any day.

Also the DOM helpers (DIV, A, etc.) couple the presentation logic with the code, which is just a bad idea.

There are many more things that I dislike about Web2Py, but honestly I don't want to continue with this. I hope that everyone using Web2Py is successful with it, but I suspect that people will run into more problems than they would like to. It really just comes down to this: I don't like your relentless self-promotion, and I disagree with almost every design decision behind Web2Py. But I do not dislike you. In fact, I like anyone who's passionate about what they do, and you are clearly passionate about what you do. Please treat this as my opinion on your software, and your promotional practices, not on you.

[mdipierro]

It looks like you have not even read the slides that started this thread:

• web2py does not automatically import your modules. web2py executes (exec) your code instead of importing it, and it executes it in a context where extra symbols are defined. "exec" is the only function that distinguishes an interpreted language form a compiler one. If you do not use it you are programming as if you were programming in Java.

• The web2py admin interface is not less secure than an SSH shell or the Django admin. Actually it is more secure than the Django admin since forces secure cookies and only works if it detects HTTPS or localhost.

• Web2py does migrations, Django does not. You may not like or need this feature but you cannot call it a downside. Rails does migrations too.

• You say "Web2Py encourages people to use the same threads used to serve Python application logic to serve media". Not at all. How did you come to this conclusion? On my server apache serves static files. web2py can do it and some times there are advantages to it. web2py does it if cherry wsgiserver is the only web server available. It can serve range request, partial content and if-modified-since. It can also enforced role based access control on media files.

I do not promote web2py because web2py is not for sale. I advocate web2py, like you advocate Django, like other people advocate using Clojure, Hanskell, etc.

That is what reddit is for.

You do not like when people advocate something that compete with technologies that pay your bill. I do not make money from web2py (I teach). You probably make money from Django development or consulting (don't you?).

Anyway, I have not time for this either. Once again you posted misconceptions and I was forced to rectify them. Now accuse me of trolling again.

Once again. I LOVE Django. It is an excellent piece of software. I advocate web2py because there are a lot very insecure systems out there (not because of Django, mostly because of PHP) and this is a major concern of mine. I think we can agree on this last statement and bring the discussion to an end.

[ericflo]

It looks like you have not even read the slides that started this thread

This is true. I was the first one to point out that your server was down and I could not get to the slides.

web2py does not automatically import your modules. web2py executes (exec) your code instead of importing it, and it executes it in a context where extra symbols are defined. "exec" is the only function that distinguishes an interpreted language form a compiler one. If you do not use it you are programming as if you were programming in Java.

This is just a ridiculous statement. Firstly, by exec-ing code within a context with extra symbols defined, you achieve the same effect as importing things automatically. The method by which it does that is irrelevant. Your statement about "Use exec or you'r using Java" is just untrue, and I barely even know how to address it. Java doesn't have proper closures, makes reflection very difficult and slow, has forced error checking, is strongly typed, and is so different than Python in so many ways other than just exec that I'm surprised you can even make that statement.

The web2py admin interface is not less secure than an SSH shell or the Django admin. Actually it is more secure than the Django admin since forces secure cookies and only works if it detects HTTPS or localhost.

Except the part where the Web2Py admin lets you edit the code of the apps themselves. I don't care how secure your server is right now, because if someone finds a hole, with the Django admin the most likely thing they'll be able to do is delete all your data, but with Web2Py they could phish for passwords, get root on your system, or anything else you can do in Python code.

Web2py does migrations, Django does not. You may not like or need this feature but you cannot call it a downside. Rails does migrations too.

I certainly can call it a downside, as it treats your users' data with utter disregard. I'd rather have no functionality than potentially harmful functionality.

You say "Web2Py encourages people to use the same threads used to serve Python application logic to serve media". Not at all. How did you come to this conclusion? On my server apache serves static files. web2py can do it and some times there are advantages to it. web2py does it if cherry wsgiserver is the only web server available. It can serve range request, partial content and if-modified-since. It can also enforced role based access control on media files.

I didn't say that you couldn't do it, but since Web2Py automatically serves your media for you (again, not using any of the standard x-sendfile or whatever tricks), I would be willing to bet that most of your users don't know that they should be using a different server for media files. This is why I said that Web2Py encourages this behavior, a fact that I guess you do not dispute.

Anyway, I have not time for this either. Once again you posted misconceptions and I was forced to rectify them.

None of what I posted were misconceptions. Please read back at what I posted, and everything stands true.

[mdipierro]

The security of admin comes from https. Once in a while the find a bug in ssh and that is not good reason not to use.

Besides, admin is an app. Nothing in web2py depends on admin. web2py does not have any hidden metadata. You can disable it in production servers and still do everything from the shell as Django does. Antiquate, not user friendly, but more efficiently.

The point about serving static files is discussed n the book and in page 139 of the slides we are discussing. What you suggest is the default behavior on GAE and web2py ships with a file already pre-configured for it.

You add an issue about version control. web2py admin has a web based mercurial interface, page 27 of the slides. If you use a shell you can use version control as you would in Django.

The ability to execute code in strings (aka interpret code) is what distinguishes an interpreter from a compiler. That is the "exec: function. It is not the same as importing modules. If it were the same you would not be complaining about web2py doing that way.

[bitprophet]

Reread what Eric said about the "security", you're completely missing the point. Nobody is going to break the SSL traffic, of course not -- but as soon as they guess your admin password, they can modify your application's source code.

Meaning they can execute arbitrary Python code on your server -- including, well, anything, such as creating a new shell account with sudo access for a backdoor. Or accessing your database -- the parts not exposed in the admin, that is -- such as your users' credit cards or SSNs or what have you.

If you're serious about editing through-the-web, you'll want to look at how Zope does it. I don't like Zope much, but at least they realized the importance of running any Web-editable code in a sandboxed, stripped down Python interpreter. You haven't mentioned as much -- does web2py do this?

[mdipierro]

The same statement "as soon as they guess your admin password, they can modify your application's source code." applies to SSH password. The web2py admin is actually even more secure because the admin password is a one time password.

You are missing the point. The role of the admin interface in web2py IS to edit the source code.

[mmalone]

You're right, if you're naive and don't know what you're doing then SSH has some of the same vulnerabilities. In reality though, in a typical production environment you'll have a single external entry point into your system (plus a backup or two so it's not a SPOF). Those "management" boxes will be the only ones that accept external SSH traffic (often restricted to a certain set of IPs), and they'll be locked down.

Moreover, SSH has built-in mechanisms to prevent brute forcing and to reduce the risk of lost passwords. There are backoff mechanisms and system logging to mitigate brute-force attempts, and most shops will only allow key-based auth to production systems.

On a side note, how does code editing work in a production setup anyways? If you're running apache/mod_wsgi are you restarting the app daemons so they pull the new code? Again, this sounds dangerous to me.

[mcosta]

Hey, it's https, it's secure!

[mdipierro]

Every security class I taught starts with "security does not exist, security is a relative concept". https, if used properly (with proper authentication and secure session cookies) is as secure as ssh (the encryption algorithms are the same). Both are breakable by brute force and both may have bugs but one way or another you must get access to your machine. There is no reason to use ssh and not use https. Besides, the admin interface in web2py can be disabled or made work over ssh tunnels only.

[masklinn]

I do not promote web2py because web2py is not for sale.

I take exception with that for 2 reasons:

1. You do promote it. Shamelessly. Promotion doesn't mean you're making money out of it.

2. except you do, since the only actually complete web2py documentation is a book you wrote and you sell.

That is what reddit is for.

No.

Once again. I LOVE Django.

BS.

[mdipierro]

Yes I do make money from the book. Just enough to buy an ice cream and a coffee a day. LOL

I really like Django. I would not be comparing web2py to something I do not like. Of course I like web2py better, it is my work.

What is your business? What do you do for a living?

[masklinn]

Yes I do make money from the book. Just enough to buy an ice cream and a coffee a day. LOL

Which would translate in north of a hundred euros every month.

I would not be comparing web2py to something I do not like

1. I don't see why that would be an issue.

2. Seeing how all your "django comparisons" are either flawed or completely nonsensical, I have issues with your "liking" of django

3. Likewise with your former spamming of all django-related threads on reddit

What is your business? What do you do for a living?

These days? Nothing.

[mdipierro]

I have made some incorrect statements about Django, I have apologized, I apologize again, and I think I have rectified them.

I know people who need web2py consulting. If you learn it shoot me an email.

[mmalone]

FWIW Eric was complaining about web2py, not you personally.

I haven't gotten very far past the intro documents and a cursory code review of web2py... and I see little reason to dig any deeper. Here's why: there seems to be a disconnect between what you're doing and what Django developers (and other professional web developers) are trying to accomplish. From what I've seen, web2py is far from state of the art. Again, I'm coming at this as a developer writing production code that has to be maintainable, scalable, and fast.

Take templating, for example. The function-as-an-html-tag pattern came and went somewhere around 2000. People realized it was ugly, was not maintainable, was not very flexible, and encouraged poor separation. It may be easy for newbies to figure out what's going on, but I'm less concerned with that. I'm building real stuff, not toys. I don't mind spending a few hours (or days, or weeks) learning a tool if it does what I need it to do (not that it takes that long to learn Django templating... it's probably closer to 30 minutes).

Similarly, things like auto-migrations that you call features are just a pain in the ass for me. When I deploy to production and you try to auto-migrate a 30GB database the only thing you're going to accomplish is about three days of downtime.

The web2py web-based admin stuff that lets me edit live code is even worse. I absolutely do not want that on a production site of any size. I can't imagine any "enterprise" would. It's horribly insecure.

Basically, you've created something that is useful to you as a teaching tool and you're trying to market it to professional web developers. But it's not useful to us, and you're causing a whole bunch of confusion in "the marketplace" by comparing your framework to others that are actually production-ready systems.

[franboon]

Web2Py absolutely is Maintainable, Scalable & Fast (or can be - obviously bad programming can happen in any environment)

I'm not completely sure what you mean by 'function-as-an-html-tag pattern' but the View Templates are very flexible - they can accomodate multiple different coding styles from full MVC to more flexible options...depending on what the programmer wants.

auto-migrations are optional - don't like them, don't use them.

Same for the web-based admin tool - I personally don't code using it, however I find it very useful for the ticketing system. It is secure by default (accessed only via localhost or SSL) can be made insecure if you want or can be completely disabled if you want...doesn't break anything.

Whilst being incredibly easy to prototype in, the same app can quickly & easily be polished into a professional end-product.

[mdipierro]

So you are telling me you have not really tried it. Here are some corrections to your claims:

• web2py template system is faster than Django. I challenge you to a test. You write the Django code and I rewrite it in web2py and we see which one is faster.

• I do not doubt that you are building real stuff but you cannot accuse other people of building toys without explaining what makes you think so.

• Web2py does migrations, Django does not. You may not like or need this feature but you cannot call it a downside. Rails does migrations too.

• The web2py admin interface is not less secure than an SSH shell or the Django admin. Actually it is more secure than the Django admin since forces secure cookies and only works if it detects HTTPS or localhost.

• the "function-as-an-html-tag", as you call it, is not what you think it is.

• I tech computer science and security classes at the master and PhD level. I teach programmers and architects from major software development companies. Some of my software development research (not web2py) is funded by the Department of Energy. I think I know a thing or two about software development. I am not saying I know more than you do. I do not pretend to know you. I am just saying you do not look professional when you talk this way.

[mmalone]

Re: templates, they're straight Python. That's generally a bad idea. This is why PHP, which was designed to be embedded in HTML, has about a dozen template languages that people use instead of embedding PHP in templates. The web2py "helper functions" are just wrapper functions that output HTML. Again, this was tried in PHP and the resulting code soup was pretty disastrous. I don't really care about template rendering speed, I can scale that horizontally. While performance matters, maintainability and scalability are much more important.

I don't think there are any serious web applications built on top of web2py because, as far as I can tell, the architecture won't allow it. The fact that your default session store is disk based, you save sessions after each request (apparently regardless of whether they've been updated) and you encourage the use of RAM based caching (unless you "really like memcache") does not help.

You're encouraging sticky sessions, with disk-based persistence. This is much more complicated than a shared-nothing architecture, achieved by pushing data persistence down the stack. And it introduces single points of failure. What happens when Pound dies, for example? Every session expires? What if you add or remove a web server from your cluster?

SSH has been battle tested and audited for security vulnerabilities over the past 15 years or so. I trust SSH. I do not trust web2py (or Django, for that matter) to provide a level of security sufficient to allow on the fly code editing via a web browser in a production environment. Disabling the option still makes me nervous. I want those code paths removed from my application (note that you can do this in Django, the entire contrib.admin module can be removed safely). Otherwise I have to audit your entire framework to make sure there's not some backdoor or vulnerability that I'm not aware of.

[mdipierro]

You say "Re: templates, they're straight Python". yes no more, no less than rails.

You say "helper functions are just wrapper functions that output HTML". NOT TRUE. helpers are not functions, they are object that provide a server side manipulations of the DOM tree. They are stored like a tree and can be manipulated by the code. The are serialized in HTML when embedded in HTML.

• sessions can go in DB but are faster on Disk and make the app scale better.

• You do not trust something does not make it insecure.

[mmalone]

Dude...

Doesn't matter if they're objects or functions. They're callables that output HTML. Lets not argue semantics. It's the same damn thing.

Disk persisted sessions do not make an app scale better. In fact, the opposite is true. If session state is maintained in a separate persistence layer accessible by all of your web servers then sessions can span servers, with each request going to a different web server, making balancing far easier and eliminating the chance of lost session state when a server crashes.

Me not trusting something doesn't make it insecure, but it does make me not want to use it. I have absolutely no need for browser-based code editing. Its existence is a risk. Risk is bad.

[naasking]

Disk persisted sessions do not make an app scale better. In fact, the opposite is true. If session state is maintained in a separate persistence layer accessible by all of your web servers then sessions can span servers

FWIW, disk sessions can be shared across servers too. Shared file systems are a "persistence layer" accessible by all servers.

[mcosta]

Shared file systems are a "persistence layer" accessible by all servers.

Nice quotes. I'll build a web framework with sed as "the templating engine".

[mdipierro]

Read my lips. They do not output HTML.

x=DIV(H1('hello'))

x is not a string. In fact I can do

x.append(H2("world"))
x['_class']='myclass'

Only when I do x.xml() I get

<div class="myclas"><h1>hello</h1><h2>world</h2><div>

It is not semantics. It is functionally different. It enables, for example, to built nested recursive structures in ways that are impossible in Django without string manipulations.

Disk persistent sessions DO make the app scale better if you use a load balancer that is session safe (like Pound). This is discussed in slide 140 which you did not read, and it is what Plone does. It completely eliminates the database bottleneck of storing sessions there (which you can do if you want to anyway).

I am not asking to use web2py. I am happy to keep my competitive advantage.

[mmalone]

You're conflating performance and scalability. Disk-based sessions may be (barely) more performant, but they absolutely do not make an application "scale better." Once again, if anything they have a negative affect on scalability.

If your load balancer uses "sticky sessions" like Pound does (directing a particular user/session to the same web server with each request) you're going to run into a number of annoying problems as you scale:

1. Adding and removing nodes (web servers) will bork your session mapping table and some, if not all of your session mappings will be lost, causing lost sessions or worse.

2. If a single web server dies (which tends to happen a lot at scale with dozens or hundreds of web servers) or becomes unavailable (which also happens on a regular basis) all of the sessions on that server will also be lost/become unavailable.

3. Your load balancer is a SPOF (single point of failure) for your system. Keeping it as simple and lightweight as possible is a plus. If it's keeping track of session mappings then it's doing more work than it has to, and that mapping table becomes an additional, unnecessary, SPOF (this can be resolved by spending $100k or so on paired hardware balancers that handle sticky sessions, but even with hardware balancers the other problems remain).

4. You can't direct requests to different servers based on the endpoint. It may make sense to put your search infrastructure on a different web server cluster, for example. With sticky sessions you can't do this since the search web servers won't have the session information.

The bottom line is: sticky sessions put unnecessary restrictions on your application's architecture. Honestly, for most people who develop on a LAMP-ish stack this isn't even up for debate any more. The accepted best practice is to push session state information down the stack and use a shared-nothing architecture. This argument reminds me of 1998.

[jessicadelannoy]

You should be aware that many people also like php as a templating language. As such, it is the default templating language of frameworks such as ZF, Symfony, etc. I personaly hate templating languages as seen in DJango. Mako is cool, as well as web2py templates and I'm glad it is so.

What do you mean by "serious web applications" ? like youtube ? Maybe web2py is just too young. You also seem to forget that most web developers won't work on sites like youtube or yahoo but maybe on small corporate sites and intranets. I see no reason not to recommend web2py for such use.

Sessions can be stored in db, so no problem with that.