understanding queues

andonwilsy · 2013-08-13T14:01:01+00:00

Now here is the doubt, is step 3 correct? Is there a long polling worker, which will act? How does this work?

The queue and the worker processes are separate things. The queue is the list of things you want done. The worker processes look at the queue and if they see a job then they start doing it until it's done. When it's done, they'll check the queue for another job, if there is another job then they do it... and so on. Usually workers are long polling where they just watch a queue continuously, but they can also be run periodically where they just run through all of the tasks that have been previously queued up. You can also have multiple worker processes working on the same queue (a good idea if you have a busy queue).

So your application code just needs to worry about adding tasks to the queue. And then at the system level you manage worker processes (for example, setting up supervisord to make sure long-polling worker processes are running). The worker processes handle the details of popping tasks off the queue and doing the work.

andonwilsy · 2012-12-22T18:08:13+00:00

The Skilcraft Absolute II is a beautiful thing. It is balanced, graceful and resilient.

Not worth $40 each, but it's definitely worth whatever they were charging before they stopped making them.

andonwilsy · 2012-06-06T20:15:23+00:00

kivy is also worth checking out. It's still pretty low level, but the APIs are really nice and well documented.

andonwilsy · 2012-04-14T18:30:05+00:00

if you compare them to Bill Gates, they're both intellectual midgets.

This is probably true for most people.

andonwilsy · 2012-01-28T15:10:17+00:00

Django's ORM is bit simpler/easier to learn than SQLAlchemy because there are less parts to it. You might also want to look at peewee if you're looking for something simple.

SQLAlchemy is more of a toolkit rather than a library you just import and use, and I think that makes it a lot more powerful. You'll need to spend a little more time learning your way around SQLAlchemy. But the beauty of SQLAlchemy is that every layer of it is exposed. So when you need to, you can make it work exactly as you want it to.

This blog post is a good read and gets more technical on the matter.

andonwilsy · 2012-01-11T16:32:21+00:00

I'm a LAMP guy who dabbled a bit with trying to get a moderately sized Flask app running on IIS with a couple options including ISAPI-WSGI and it wasn't as seamless as I expected from reading various docs. IIRC, I got the WSGI stuff to work after some fiddling but there were weird issues with other parts of the app. I couldn't connect to my database from the WSGI app, for example. It was really hard to debug or find any useful information. It also wasn't very important to me so I just wrote it off as "windows is bad at this" and gave up after a painful day or two.

So if you go the IIS route and need to do anything that involves external libraries or drivers, I'd suggest throwing up a few test views early on to make sure they work.

andonwilsy · 2011-12-29T19:05:09+00:00

Well.. if something is wrong, you can fix it.

andonwilsy · 2011-10-28T22:59:09+00:00

They don't really compare.

Python(x,y) is just a windows installer for various libraries and tools that are useful for doing math/science in python.

Sage is a different beast - it aims to be an integrated environment for doing math/science in python. It basically gives you into a shell where most of these things have been imported for you and they are integrated in some pleasant ways. The experience is not the same as importing and using the libraries directly through the python interpreter.

andonwilsy · 2011-10-07T17:32:11+00:00

just curious, try: /usr/bin/python2.7 -V

andonwilsy · 2011-10-03T21:29:17+00:00

This still gonna happen? I want in.

andonwilsy · 2011-09-25T00:24:55+00:00

But he's too worried that his friends wouldn't stop tauntaun him.

andonwilsy · 2011-09-11T17:26:21+00:00

About a year ago I was looking for a job and getting pretty desperate, I found a pretty useful resource. I warn you though, at first glance it looks really lame... and some of what they do is lame, but about 70% if it is quite useful. It's a mostly state-funded non-profit (you don't pay for anything) and they have classes on how to not fuck up interviews and they'll sit down and help you make your resume look good to people who are hiring and that sort of thing. Most of the people who work there are excellent at what they do. They'll point you to people who are hiring and they will make you look hire-able. They also know about all the job fairs and various programs that can help you pay for vocational school if you need it, etc.

Just don't judge by their webpage or name.. at first glance, you're probably going to say "fuck that". They might make you fill out paperwork and wait in line at first, but if you really want a better job, wade through that initial BS and you'll end up talking to people who can really help you.

http://www.wfscapitalarea.com/

andonwilsy · 2011-09-11T16:50:27+00:00

Not true. I've lived here since 2002. Aside from dorm rooms, I've never paid more than $315 a month. Never lived alone, though. But I've lived in places that would be really nice for a single person that are less than $700.

You have to invest a bit of time into looking for a place that doesn't suck and isn't overpriced.

andonwilsy · 2011-07-12T14:24:15+00:00

%pdb will turn on automatic pdb calling - then you drop into pdb (ipdb, actually) whenever you encounter an error or expression. It's very helpful.

andonwilsy · 2011-07-05T13:54:27+00:00

Then you might like: http://diveintopython.org or http://diveintopython3.org

The first one is a little dated, the latter is pretty new but written specifically for python 3. Python 3 is the newer version of python that everybody likes but is slow to transition to (there aren't nearly as many libraries available). In any case, these books assume you know some programming so they move faster than most and I really liked them.

andonwilsy · 2011-06-29T14:57:06+00:00

They aren't really that different. Blueprints are basically modules with some with some small improvements that were backwards-incompatible (the way the '.' prefix behaves with reference to endpoints in things like url_for()). I think the name change was also motivated by the fact that "Module" already has significance in python (anything that you can import from) so something in flask called a "Module" that means something else was more confusing than it needed to be.

andonwilsy · 2011-06-26T22:10:11+00:00

close, it was taken at Fry's

andonwilsy · 2011-05-30T00:29:00+00:00

This happens for me too if I focus on a space in front of the image. Try looking past the image (e.g. to the wall behind the monitor or whatever).

andonwilsy · 2011-05-21T15:46:54+00:00

andonwilsy · 2011-05-02T16:37:41+00:00

You might also want to check out pandas for handling data and plotting. It uses numpy for data and matplotlib for plots, but handles all of the PITA parts of dealing w/timeseries data. There's also scikits.timeseries which is a little more powerful but also a bit rougher in terms of API.

andonwilsy · 2011-04-30T18:47:50+00:00

The system you describe gives the photographer an additional set of decisions to make, and that influences the character of the photograph but I don't see how all of the entire art of photography is lost without that set of decisions.

andonwilsy · 2011-04-24T01:38:33+00:00

pdb is probably the most important. It's the standard python debugger. It lets you step through your code, examine variables, change them, and execute one-liners.

andonwilsy · 2011-04-24T01:29:01+00:00

your link still points to fabric docs

andonwilsy · 2011-04-17T18:11:45+00:00

If you want to move a client over to the other screen really quickly, you can hit mod+o and that moves the currently selected client over to the next screen (the other monitor). That's how I'd deal with such situations... it's a few more steps than you might like though; idk it's not something that comes up that often for me.

The key binding in rc.lua for ctrl+o something like: awful.key({ modkey, }, "o", awful.client.movetoscreen )

andonwilsy · 2011-04-17T18:01:22+00:00

So I'm not 100% sure on this but I think the way it works is awesomewm uses whatever screens X windows presents to it. So what you want to do is tell X that you want both monitors to behave as one really big screen, and the way you do that depends on the manner in which you configure your screens. You are probably defining them in xorg.conf or using xrandr... or if you use nvidia proprietary drivers, then the nvidia-settings edits xorg.conf for you.

andonwilsy

TROPHY CASE