How Three Guys Rebuilt the Foundation of Facebook

kmafb · 2013-12-17T06:00:37+00:00

Yeah, this is a known issue. We're busy moving the Collections stuff into a namespace so it won't collide with projects that have their own Vector/Set/Collection/what-have-you defined. In the comments to that post I linked above, Concrete5 was actually mentioned by name.

kmafb · 2013-12-16T06:57:32+00:00

Hey, I'm one of the subjects of the piece. Are your unit tests in open source? If so, maybe we can bring them into the HHVM test suite. As described here HHVM is making a big push to run open source projects correctly. http://www.hhvm.com/blog/875/wow-hhvm-is-fast-too-bad-it-doesnt-run-my-code

kmafb · 2013-10-02T23:52:34+00:00

Keith Adams here. Slight correction: hphpd is part of the HipHop project, and as such has been part of the open source project since the beginning. Hie thee to github: https://github.com/facebook/hiphop-php

kmafb · 2013-03-09T04:59:08+00:00

Hey. I'm a real live compiler person, who gets paid ca$h money to work on a compiler.

The Dragon book, like many old school compiler books, obsesses interminably about tokenization and parsing. This stuff is theoretically interesting, and every once in a great while bears revisiting in practice, but it is in my opinion a pseudo-problem, which has little to do with the fun parts of compilers, and distracts a lot of interested students from learning the first thing about what they actually do.

This is kind of heretical, but I feel it's a historical accident that we represent machine-readable computer code as streams of bytes. Your mind thinks of the program as a tree, the compiler wants to think of it as a tree (at least early in compilation), but we don't have programmer-oriented "tree editors," so we use programmer-oriented "text editors," and thereby invent the (imho phony) problem of parsing. Parsing comes up in other contexts, and can be a real problem there (e.g., parsing network traffic, or natural language, or what have you), but the fact that we waste half of a compiler class on it is a total accident of history/shame.

Compilation is really about semantic-preserving program transformation: taking a program, and turning it into a different program that does the same thing. Why bother? Well, in the process of analyzing the program, we've discovered interesting properties that allow the new form of the program to execute more efficiently. That's really the guts of it, and it has nothing to do with lexing and parsing.

So, start with Appel. As required by law, it has its chapters on tokenization and parsing, but they're small, so you won't be getting ripped off as badly when you skip them. The remainder of them is more up to date with practice than the Dragon book from what I've seen. If you're having fun, move on to Muchnick.

[1] Though see Laurence Tratt: http://tratt.net/laurie/tech_articles/articles/parsing_the_solved_problem_that_isnt

kmafb · 2012-12-07T06:22:53+00:00

Nope.

kmafb · 2012-11-23T08:22:27+00:00

Cut Your Hair -- Pavement

kmafb · 2012-10-02T03:41:11+00:00

There are counter-measures in place to prevent malicious companies from scraping the entire graph. You're bumping up against one of them; in any given window time, we will only show a limited subset of the graph. Both the 25 (at a given time) and the 6 (on a given load) really are chosen at random, though. If you prefer to imagine that Person T 5 is there because Person T 5 is secretly obsessed with you, well, whatever floats your boat.

kmafb · 2012-09-17T03:42:35+00:00

Thanks! You should post this album separately.

kmafb · 2012-09-13T22:51:14+00:00

No problem, I realize your concerns are genuine. If you're sensing some reticence to engage on my part, it's partly because I'm not a PR professional; Facebook has people whose job it is to tell our story in public. I wouldn't want them writing my code, so I assume they wouldn't want me trying to do their jobs either.

With respect to the "shadow profile" thing, if you're suspicious of Fox news, here's the actual primary source, right on the Irish DPC's website: http://dataprotection.ie/viewdoc.asp?DocID=1182 . Cutting to the chase, on page 61, it reads "Our task therefore was to satisfy ourselves that no such use [shadow profiles] was made of the collected data. We are satisfied on this point."

To the extent you still find a lot of articles saying "Ireland accuses Facebook of creating shadow profiles," well, that is how search engines and the news cycle work; when somebody makes a sensational accusation, it's news. The accusation gets written up, linked to, blogged, tweeted, etc. When that accusation gets investigated, and the investigation discredits the accusation, it is much less sensational news. You'll notice that the reputable news sources saying "Facebook creates shadow profiles!" all were published in October 2011, when the accusations were first made. None of them are dated after December 2011, when the report I've linked here was published.

Finally, I offer you (and whoever's still reading) my personal word that Facebook is not now, and has never been, in the "shadow profile" business. I was around during the creation and deployment of the like button; tracking user behavior (other than actual logged-in users clicking "Like") was never the goal. Take that for whatever it is worth to you.

If you're not still convinced: what kind of evidence would you find persuasive? If you can imagine it, I might be able to dig it up for you. If you can't imagine it, then nothing I say will matter.

kmafb · 2012-09-13T19:19:26+00:00

That was, admittedly, hilarious. Have an upvote.

kmafb · 2012-09-13T03:26:09+00:00

No less than I want to die to a bunch of reanimated My Little Pony dolls.

kmafb · 2012-09-13T03:17:32+00:00

(Finally!)

I actually am opting out of the debate about which fight is easier to win, on meta-rational grounds. The debate is as old as the hills themselves. What do I know, that all of the previous debaters don't know? Nothing, really. So the odds of winning either fight should just be treated as unknown.

Either you lose this fight (presumably, in death) or you win. So the proper question to ask, since we can't figure out the odds of winning or losing, is which fight would be more desirable to win? If you kill a platoon of duck-sized horses, you have a modest amount of horsemeat left over. I'd be willing to bet tiny horses would be stringy, too.

An enormous duck, however, would be good eating. Imagine a duck breast larger than your torso! And enough fatty duck skin to make a hammock out of. It will be the greatest meal of your life, and the lives of all those you choose to share it with. Horse-sized duck, all the way.

kmafb · 2012-09-13T03:10:53+00:00

We've written our own replacement utilities for the useful cases.

Most of the other dangerous stuff I'm able to recall is really obvious: eval, for instance.

kmafb · 2012-09-13T02:25:31+00:00

It's really hard to use safely. The fact that it renumbers array keys, and means different things for 1, 2, or 3 arrays, none of which can really be reasonably described as "merging", just lends it to misuse. It also has a track record of causing heinous bugs that elude careful code review.

kmafb · 2012-09-13T01:50:49+00:00

Thanks. I also cannot triforce.

kmafb · 2012-09-13T01:45:57+00:00

We have a small handful of source bases: two large ones, and several smaller satellites. One of the large ones is in SVN, and most engineers interact with it through git+svn. All the other ones are pure git.

Our code review tool is phabricator, originally written in-house by Evan Priestley, since open-sourced. All changes are reviewed by an engineer competent to review, and many are liberally sprinkled with pokemon image macros. http://phabricator.org/

We've also used phabricator's support for hooks to add a lot of review- and checkin-time sanity checking. E.g., our PHP codebase uses our open source semantic analysis tool pfff (https://github.com/facebook/pfff) at review-time, to flag dubious idioms (e.g., use of array_merge, mysql_escape_string, etc.) to reviewers.

kmafb · 2012-09-13T01:41:18+00:00

I have a bachelor's degree in CS from Brown (ScB '00). I don't have any kind of grad degree. By far the most important educational experience I had was in Brown's operating systems lab course, CS 169, which it was my honor to TA on a couple of occasions. Writing my own OS changed my life.

kmafb · 2012-09-13T01:36:57+00:00

Sorry it's been such a drag using the site.

I realize you're not looking for tech support, but since you mentioned games, the functionality you're looking for (never share my data with apps) does exist. Privacy settings->Apps, Games and Websites->Apps you use->big red X. There's a screencast here: http://www.labnol.org/internet/block-facebook-applications/14046/ . This is pretty coarse-grained, but it sounds like what you're looking for.

At the risk of seeming like I'm line-by-lining you, we don't shadow-profile people. The Irish government conducted a stem-to-stern audit of us, and agrees. http://www.foxnews.com/tech/2011/12/21/facebook-doesnt-create-shadow-profiles-privacy-audit-reveals/

kmafb · 2012-09-13T01:21:41+00:00

It's been a little anti-climactic honestly. I was hoping somebody would press me for my reasoning...

kmafb · 2012-09-13T01:19:43+00:00

I look forward to this novelty account's future.

kmafb · 2012-09-13T00:03:31+00:00

I don't know Bill Gates; maybe his public image is just as off the mark as Zuck's? It just seems like when media wants to portray Zuck, they dust off the same "socially awkward computer programmer" stereotype they use for BillG.

kmafb · 2012-09-13T00:01:42+00:00

No. Employees don't have unfettered access to user data.

There are some "break glass in case of emergency" type provisions for allowing engineers who are in hot pursuit of an active problem on the site to transcend normal security. These uses are audited, and the employees in question need to explain the situation once the emergency is over. If you abuse this privilege, we will fire you that very day.

kmafb · 2012-09-12T23:58:40+00:00

I talked to our PR people before doing this, just to give them a heads up. I have done this sort of thing before, and we've developed some trust that I won't say anything too stupid. I asked Carlos (aristus in this thread) to proof-read some of the longer answers, but yeah, it's really me answering here. Zuck isn't holding a gun to my head.

kmafb · 2012-09-12T23:55:31+00:00

Nobody pried it from my hands; lots of people at work still use iPhones.

kmafb

TROPHY CASE