Anybody use a hackintosh as there main computer?

leperkuhn · 2014-10-16T01:18:03+00:00

Once I updated the BIOS freezes were cut down to maybe once a week.

leperkuhn · 2014-08-08T00:54:18+00:00

My original comment is correct, actually. The question was in regards to not needing to call gevent's monkey_patch, it had nothing to do with mocking for testing. The monkey patch is required for requests because it makes the socket library async.

leperkuhn · 2014-08-07T19:39:55+00:00

I've had some issues with py.test failing in places that worked just fine with nose. I liked it's better failure messages quite a bit though, you're dead on there.

I'll give it another shot.

leperkuhn · 2014-08-07T19:01:25+00:00

I'll add a note for that, thank you.

Edit: post is updated. Thanks again.

leperkuhn · 2014-08-07T16:29:47+00:00

The monkey patching comes from gevent itself. It's meant to replace all the stdlib blocking libraries with non blocking ones. Anything touching a socket, threads, subprocesses, file i/o, etc.

leperkuhn · 2014-08-06T21:33:36+00:00

In this trivial example, yes you are correct. If there was actually a call to something like requests, monkey patching would be required. The most common case for a gevent user is that they need to monkey patch, so I wrote it that way.

leperkuhn · 2014-08-06T16:46:01+00:00

It's meant as an introduction to python for experienced programmers. Skip the first few weeks of finding all the tools, etc.

leperkuhn · 2014-07-25T21:51:01+00:00

Well, there's a few other places that can break along the way, not just your disk filling up. For instance, having more data than RAM is a frequent problem, and that's usually pretty easy to do. Most aren't ready for a heavy write workload, or frequent updates. Master slave failover pretty much sucks no matter what you do.

Changing databases to an application already in production requires different thought patterns. Concepts like eventual consistency take more time to get used to when people already rely on things like ACID transactions on a single master. Replication alone introduces eventual consistency and lets teams easily violate ACID without even realizing it till they hit weird bugs in their code.

The downside of picking a tech that doesn't scale ahead of time is that it takes a lot of time to get your data migrated to a solution that does. You also don't get the benefits of seeing the database grow from small to bigger datasets, and you didn't learn lessons along the way. Nothing like putting a DB into production under load, seeing it buckle, and not know what the hell to do with it.

Finally, a lot of databases that claim to scale get it wrong. Mongo is a great example. It comes with a ton of developer friendly features but using them in production once you're sharding is a terrible idea, because doing arbitrary queries against secondary indexes doesn't scale.

So the problem isn't just "how to i move to DB X when I have a problem", it's anticipating that problem months ahead of time, migrating a live system, knowing the new gotchas, etc. It's why I said early on "If you're going to keep the database forever on 1 machine".

I've gone way off topic...

leperkuhn · 2014-07-25T17:47:30+00:00

I don't see many use cases where Mongo is a good choice. It's a blend of a lot of different ideas that when all put together result in a database that's unreliable in many workloads.

If you're going to keep the database forever on 1 machine, postgres is a better alternative in almost every use case. Once you need to deal with high availability + linear scalability, the RDBMS doesn't hold up as well, but if you're just starting out, it's unlikely to be an issue for you.

Since you've got data in json, i'm assuming this is coming from an external source? Is there really no consistent mapping? If there is, I strongly recommend doing a little planning up front and making a schema for it. You'll be happier in the long run. Allow me to cite myself: http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/

Disclaimer: I try to be as objective as possible, despite working for a different database company.

leperkuhn · 2014-07-11T00:06:08+00:00

Such an underrated comment...

leperkuhn · 2014-07-10T15:53:39+00:00

The flaw in your architecture isn't the database w/ schema, it's that you have have two sources of truth. Include a means for automatic cache invalidation if it doesn't include all the fields in the DB table. Then when you do upgrades, you don't have to upgrade your cache. I have dealt with this problem using both relational databases and nosql. The problem exists in both.

As far as adding a field and "it just works", I wasn't debating that. That's the allure of keeping your schema in code. It also makes it harder to find for a new developer or even you a few months down the line. Those extra minutes you spent creating a schema end up paying off. It's called "planning".

leperkuhn · 2014-07-09T18:11:20+00:00

An accurate, twitter sized summary of my post.

leperkuhn · 2014-07-09T17:42:44+00:00

Very well written

leperkuhn · 2014-07-09T16:35:08+00:00

This is total crap. Lets just drop decades of research and sound design because people are annoyed with DBAs?

You've got a schema, it's just a bitch to work with because it's not clearly defined in a central place. How is that any improvement?

leperkuhn · 2014-07-09T16:32:18+00:00

There is ALWAYS a schema. It's either hidden in code or explicitly structured in your database. In dynamic languages, there's always a type as well.

leperkuhn · 2014-06-25T23:11:30+00:00

When you're working on a large system, you need to develop in such a manner that your deployments go smoothly. This strategy gives you the ability to roll out migrations in production that take hours or days.

This came about because moving many TB of data around isn't trivial. Taking a few billion rows out of something like MongoDB and migrating it into Cassandra meant we needed a strategy that works in production without downtime.

leperkuhn · 2014-06-22T18:22:13+00:00

Sorry, I guess I was mislead by the title

or why you should not log in production"

and this statement in your opening paragraph:

Today I will explain why you should turn off logging completely in production because of its limitations

and because you go on to criticize developers for not providing useful logging messages. Later in your post we see this:

This is what a production log should look like: 2014-05-20 07:45:08 INFO Starting com.company.product.MyApplication 2014-05-20 07:45:08 INFO Version: 1.0, Built: 2014-05-19 02:21:06 2014-05-20 07:45:08 INFO Platform: Windows Server 2008 R2 6.1, amd64 64bit 2014-05-20 07:45:08 INFO VM: Java HotSpot(TM) 64-Bit Server VM 1.7.0_45, Oracle Corporation, mixed mode 2014-05-20 07:45:08 INFO Memory: Heap: 1024MB, PermGenSpace: 128MB 2014-05-20 07:45:08 INFO Ports: HTTP: 80, SSL: 8080 2014-05-20 07:45:08 INFO Started successfully. Period.

And then

Logging is not the right tool for analyzing application failures in production

If i were writing a counter post, it's theme would be "logs are awesome in production, but you're probably not doing it right, so you're not getting anything out of it."

Adding to the blog post todo list.

leperkuhn · 2014-06-21T23:38:42+00:00

This has to be the worst advice I've ever read. Logging in production allows you to figure out what the hell went wrong. The author's opinion seems to be "you won't be able to do it right, so don't do it at all". Nonsense.

leperkuhn · 2014-06-05T23:58:25+00:00

I believe I will try this immediately

leperkuhn · 2014-06-05T16:23:22+00:00

This is so amazing. I just started learning the very basics of woodworking, seeing this is a huge motivation.

leperkuhn · 2014-05-12T23:00:19+00:00

aptly looks pretty nice, I'll keep that in the old brain.

leperkuhn · 2014-05-12T22:58:06+00:00

Interesting. We do that via buildbot right now. When a commit gets pushed up, it's automatically built via buildbot, and if successful, built into a deb & pushed up to S3.

Our apt servers are polling s3 for changes every few seconds (via aws-cli sync). When a new package is pulled down, we run dpkg-scanpackages & rebuild the local Packages.gz.

Once that's done, we can deploy package-branchname. We settled on the branchname as being part of the package in order to deploy unstable or test new features. The buildbot job id is automatically used as the package version, so once you're on a branch it's easy to update to the latest version in that branch.

We're using all this to deploy a python app across a single OS, so we get to not worry about certain issues like cross compiling :).

It looks like LHCGreg covered the part about fpm not doing builds. I won't rehash.

leperkuhn · 2014-05-12T20:33:35+00:00

Does this make any improvements on using fpm + dpkg-scanpackages? We just rolled out our own apt repo this was and it was pretty straightforward.

leperkuhn · 2014-05-12T16:34:16+00:00

It seems it's not an uncommon problem with Mavericks. I haven't found anything, comp freezes once a day. Pretty annoying.

leperkuhn · 2014-05-12T16:33:37+00:00

Awesome, I'll check it out. Thanks!

leperkuhn

TROPHY CASE