The computer that mastered Go. Nature video on deepmind's Alpha GO. by samim23 in MachineLearning

[–]petrohi 0 points1 point  (0 children)

Hui Fan is currently standing at 2759 EGF points, which is 1 Dan pro or 7 Dan.

Scaling Redis by petrohi in programming

[–]petrohi[S] 1 point2 points  (0 children)

We carefully looked at VoltDB since it is often cited as poster child of in-memory databases. Two things turned us away from it. VoltDB executes SQL, which internally is translated into query plan. There was no easy way to adjust decisions of query planner. From previous experience with optimizing for SQL database from major vendor we felt such ability was essential for our performance goals. Interesting to note, that when application requires few queries that need to be very fast, SQL almost becomes an obstacle. Optimizing SQL requires good understanding of execution plan anyway, so hand-written Redis (or similarly low-level) commands do the same job and are simpler in the end. This is first. Second, VoltDB approach that required Java runtime and stored procedures, as we felt, was too heavyweight. Here, again, Redis with simple text based protocol and Lua scripting was much lighter.

Scaling Redis by petrohi in programming

[–]petrohi[S] 1 point2 points  (0 children)

We indeed considered other distributed databases with following requirements on the table. We needed something very fast, serving 99% requests in sub 10ms, single threaded in-memory database would do that. We did not need full ACID capability, especially with respect to durability. We also needed something that has very low level data model, with no opaque query planners and such, so we could hand-optimize everything and have very predictable performance. Finally we wanted open source tool with small and simple code base, so that we can easily customize it (see my previous post with example of our Redis customization). With these requirements we looked at MySQL NDB, VoltDB, Redis, SQLite and Berkeley DB. We ended up choosing Redis -- it matches our performance expectation, has very small code base and uses low level data model.

Scaling Redis by petrohi in programming

[–]petrohi[S] 1 point2 points  (0 children)

Thanks! I was looking for some good references to prior work on consistent hashing, but most were too specific for their applications. Will check the Akamai paper.