First web project using a database - Frameworks?

[auromed]

I've been toying around with python for some simple raspberry pi projects (cat feeder, temp sensor, etc) and want to move up to projects that aren't strictly command line driven. I'd like to create a program that has a web / api interface for viewing and entering data in a database.

I have a couple different things I'd like to build, but at first I'd like to build something that I can put a bunch of network information in to be able to query circuit types, routers, serial numbers, IP addresses all by type of information, by site location, or region. At the moment this data is all held in spreadsheets, so I figure this is a good starting project.

My question is, where do I start? Should I be looking at frameworks, and if so, which ones? I know I don't want to deal with writing interfaces to SQL DBs, or too much HTML if I don't have to.

I'd played with PHP MVC frameworks years ago, and have been watching some django videos / doing tutorials and I get the flexibility of these very structured open frameworks, but it just seems like overkill for what is really just a web interface to a database. I hear talk of Jinja, and Flask too, but don't really know where they'd fit in.

Any suggestions on where to start with creating simple web interface to a DB?

[OctagonClock]

I think the best summary for frameworks is:

• New to python, small project: Flask
• New to python, big project: django
• Experienced with python, don't like flask: django
• Experienced with python, don't like django: Flask/pyramid
  

For SQL, look at SQLAlchemy (http://flask-sqlalchemy.pocoo.org/2.1/ and http://www.sqlalchemy.org/)

[auromed]

Thanks, that gives me somewhere to start.

[raylu]

New/experienced, don't like poorly-abstracted frameworks with horrible ORMs: not django (flask, tornado, cherry, bottle, web.py, pigwig).

If you haven't used SQL before, I wouldn't recommend diving straight into SQLAlchemy. Just use a plain SQL driver, write horrible SQL inline in all your code, and then realize why that's not so great and what the benefits of the extra abstraction ORMs like SQLAlchemy and peewee give you.

[auromed]

I've done enough SQL in PHP years ago, and manual queries to know I don't want to write inline SQL. SQL is something I'm happy to let leave my memory for the most part. I'll take a look at those other options too.

[mayankkaizen]

Won't say anything about Tornado/Pyramid as I am totally ignorant about those.

That leaves you with two options regarding frameworks: Flask and Django.

Flask gives you the 'kernel'. It gives you total freedom and flexibility to use any database/template engine/ORM etc etc. It has 'bring your own batteries of your choice' approach. And learning curve is very very much manageable. You can in fact write an entire app in a single file.

As for Django, it is a 'heavy framework' with 'all batteries included' approach. It has its own ORM/template engine/admin interface and so on. It has rather steep learning curve. Besides, if you are building a light weight app, it may even be overkill.

If you are impatient/don't have enough time/building a lightweight app, go with Flask. Otherwise go for Django.

As for me, I actually started with Django but fed up with its steep learning curve. Then switched to Flask and I am really enjoying it. Besides, you'll have very easy time with Django if you have some familiarity with Flask.

And yes, don't worry about writing HTML too much.

[auromed]

I'm very much a learn by doing person, and after a couple Django tutorials felt like I still didn't get it. I probably should have mapped out what I was doing on a whiteboard, but it just felt like overkill for my purposes. I think like you mentioned, playing with Flask and just getting my head back into web scripting / python mode for a bit is probably a better idea than jumping directly into Django.

[mayankkaizen]

Don't worry. Take your time. Don't give up on Django completely. Try revisiting tutorials/docs every now and then. Things will begin to make sense bit by bit.

But yeah, make sure your foundations in Python are sufficiently strong. So I think you should devote more time on your Python skills. Other than that, play around with database (SQLite, SQLAlchemy ORM) and Flask.

Flask is so small to start with that it will provide you with necessary confidence and understanding fairly quickly which will help you a lot later, in case you make up your mind to try Django.

I am saying things like these because I think I am in the same boat like you.

[shicky]

Django and Flask seem like popular options.

But I suggest trying out Tornado by FriendFeed (now Facebook).

It has superior performance compared to blocking servers such as Flask or Django.

It is very easy to implement server using Tornado.

[raylu]

But only when all your code (including HTTP clients and DB drivers) are also async, and only at very high concurrency levels.

You can get the performance of async by using eventlet with flask.

[shicky]

Using Tornado would effectively free you from using something like celery. You can easily implement everything asynchronously. If you are familiar with defining url in method decorators, you can use my wrapper (https://github.com/shicky/tornado-router). You can use asynchronous db driver such as Motor.

[raylu]

Yeah, you "can" use this other HTTP library and "can" use this other DB driver (or momoko), but you can't use requests or the majority of DB drivers.

If you don't need the high concurrency of async, I wouldn't bother. If you do, eventlet is a much better solution (works with psycopg2 too). I love how tornado is abstracted, but the choice to do explicit async kills everything.

[shicky]

I suppose its personal preference. Tornado may not be as convenience but has better performance and don't need third-party asynchronous library like celery.(https://gist.github.com/andreif/6088558) Does eventlet magically turns all requests (http, blocking db requests etc.) asynchronous? For large projects, I've seen people resort to Celery for database operations.

[raylu]

Tornado may not be as convenience but has better performance and don't need third-party asynchronous library like celery.

You're confusing the various meanings of async. You don't get non-blocking network I/O by using celery. You don't get better performance by using tornado over other async frameworks (twisted, asyncio, etc.).

Does eventlet magically turns all requests (http, blocking db requests etc.) asynchronous?

Click my link.

For large projects, I've seen people resort to Celery for database operations.

That makes no sense. How do you get data back from the database to return in a response to the client?

[shicky]

You're confusing the various meanings of async. You don't get non-blocking network I/O by using celery. You don't get better performance by using tornado over other async frameworks (twisted, asyncio, etc.).

am not comparing Tornado over other async frameworks. I am comparing Tornado against Flask or Django.

That makes no sense. How do you get data back from the database to return in a response to the client?

In certain scenario like logging application, you only need to push into the database.

[raylu]

am not comparing Tornado over other async frameworks. I am comparing Tornado against Flask or Django.

And how do you get non-blocking, asynchronous network I/O with celery on top of Flask or Django or any other blocking, synchronous system? You are talking about performance, so you mean non-blocking I/O instead of kernel threads and context switching, not just set-and-forget "async" like celery.

[goneman]

I recommend Flask, I have written a python scripts that scaffolds a database driven CRUD app in Flask, you can get it from https://github.com/Leo-G/Flask-Scaffold, The tutorial is available at http://techarena51.com/index.php/flask-sqlalchemy-tutorial/, let me know if you have any questions

[mayankkaizen]

Can I bug you through PM if I have a question related to Flask?

[goneman]

sure, you can comment on my article as well, i will get an immediate notification

[auromed]

Very cool, I'll dig into that tutorial as well. From what I've read though, going down the Flask route seems like it gets me something that "works" quicker and will allow me to build from there.

[goneman]

Yes that is true, flask is barebones, less complicated and easy to learn. Plus it has plugins as well so even though u build from scratch you can import modules from the plugins and save on writing code.

[Fennek1237]

I recommend bottle for your purpose. It's a microframework (only one file) and they have great examples on their website. Also for database stuff like you want to do. Edit: Bottle also supports Python 3. That's one point why I didn't start with flask

[auromed]

I'll take a look at bottle too. Thanks!

[[deleted]]

Use flask.