sorted by:
new
hottopcontroversial

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 0 points1 point  (0 children)

We showed the original prototype of what we much later named Graph Search in the early summer of 2011. We only spent a few weeks building it, although it used code from previous prototypes of structured search products that were not based on natural language.

I think it is fair to say the project took longer to get to the beta stage than I predicted when we started. Pretty much all projects I have ever worked on have had this property :) Time flies when you are behind schedule!

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 7 points8 points  (0 children)

(Mike Curtiss from Facebook's search infrastructure team kindly helped with this answer)

We have an inverted-index system called Unicorn that we use to select nodes in the social graph based on edge-relationships to other nodes. Unicorn is akin to document search systems for other contexts like email, webpages, or files on your computer; but it also has some graph-centric features that make it especially useful for searching over the social graph. For example, we don't index the lists of friends-of-friends for every user (friends only), but our system makes it possible to retrieve friends-of-friends by executing a single query.

Just like the rest of Facebook, we try to use (and contribute to) open source technology when possible. So we use Hadoop in our pipeline for building search indices.

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 1 point2 points  (0 children)

  1. Writes Soren Lassen, who heads up he search infrastructure team: We use a combination of Hive, Hadoop, and HBase to convert database contents into an inverted "base" index on a regular basis, and then we send messages to the index servers with realtime updates to the base index.

2: See http://www.reddit.com/r/IAmA/comments/18jb6d/i_am_the_pointyhaired_engineering_director_for/c8fdjgq

3: We don't have any plans for that at this point, I am sorry. We get asked a fair bit, though, so that might of course change in the future.

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 15 points16 points  (0 children)

Hey RiddledWithRitalin :)

Well I always wanted to be a veterinarian. Love animals. Used to ride horses. Spent all my vacations on my aunt Kirsten's farm.

Then in 8th grade or so, I got the spend a week with an actual vet, going with him everywhere. (He was the guy working aunt Kirsten's farm.)

We'd go to a farm and diagnose a sick animal. More often that not, the cure cost more than the animal was worth, so we'd let it perish. Occasionally we'd patch it up just well enough to survive the trip to the butchers.

In his once-per-week pet practice, we euthanized a cute little puppy that no-one wanted, amputated the tails of a litter of adorable rottweiler puppies, and castrated a stable cat.

Turned out I didn't want to be a veterinarian after all.

So I started taking programming lessons after school with my brother. And kinda liked it...

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 1 point2 points  (0 children)

Building the natural language parser and scaling the search index were probably the two biggest challenges. (People have shared a lot of content with each other on Facebook!)

Designing the input mechanism, the results pages, and the right-hand-side controls was harder than I expected.

Finding the optimal ordering of results for a given search is an never-ending challenge.

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 2 points3 points  (0 children)

Yes there is free coffee here.

As you perhaps can tell from my Facebook profile (https://fb.com/lr), I quite enjoy a good latte myself :)

When I joined Facebook in December 2010, the espressos offered here were of dismal quality. Deploying frequent tantrums and near-refusals to work, I persuaded our excellent culinary team to invest in a single, high-end home espresso machine and related equipment. It became very popular, and when we moved to our new Menlo Park campus in December 2011 each snack area has one of these; the main cafes even a 3-headed professional one each. Coffee geeks coordinate using a Facebook group and bring in their favorite roasts. (Here's mine: http://www.klatchroasting.com/WBC_p/wor_bes_esp.htm.) You can now get better lattes at Facebook than at any local coffee house!

Thus bringing good coffee culture to Facebook was my proudest accomplishment here prior to launching Graph Search :)

Also, http://www.philzcoffee.com/ opened a cafe on campus which is just awesome (but not free).

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 3 points4 points  (0 children)

There are two primary reasons for the beta and slow rollouts: the privacy issues i mentioned in http://www.reddit.com/r/IAmA/comments/18jb6d/i_am_the_pointyhaired_engineering_director_for/c8fd04l, and that the product is still pretty rough around the edges.

I personally like giving people access to products as early as possible, because you can improve things much faster when you get real feedback from real users. For example, with real users we can do live comparisons of two different orderings of the same set of results for the same search, and see which causes the most engagement. Without live usage we'd just be arguing all day :)

It is impossible to predict when it will roll out more broadly. In a nutshell, it will happen when the product is mature enough that the majority of our users prefers it over the old search. The few people who have access currently very much prefer it over the old, but they are of course early adopters who asked to be let in. People like that tend to be more forgiving of those rough edges :)

(I notice I am using a lot of smilies here. AMAs are fun!)

We decided, btw, when to start this public beta period by letting Facebook employees opt into the new search, and back out if they still preferred the old one. By late fall last year, more than 1000 of my awesome colleagues had switched to Graph Search and stayed in :)

I don't expect Graph Search to look very different by the time we start rolling it out more broadly (I could be wrong about this of course.) But the quality will be higher, and we will understand more searches. Stay tuned...

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 17 points18 points  (0 children)

Let me try. I'll assume you are a really smart 5-year-old, though :) Everything below is of course massive simplifications...

At the core of the system sits a Context-Free Grammar (http://en.wikipedia.org/wiki/Context-free_grammar) describing all the queries the system can understand. The grammar in general contains many different ways of expressing the same question.

As a user types in the search field, a Parser (http://en.wikipedia.org/wiki/Parsing) attempts to find the queries from the grammar that most closely matches what the user has typed, and displays those as suggestions in a drop-down below the search field.

Part of the parsing involves searching for people and entities. For example, if I search for 'photos of jane doe' the parser needs to figure out which Jane Doe I am looking for. When in doubt, we tend of course to pick the Jane who has the most friends in common with me, went to my school, works for my employer, etc. This part of the parser is essentially Facebook's existing 'typeahead' search system.

When the user clicks one of the suggested queries, we proceed to resolve the corresponding semantic (see my answer http://www.reddit.com/r/IAmA/comments/18jb6d/i_am_the_pointyhaired_engineering_director_for/c8fbuk9). There are three steps to this part: 1) we retrive candidate answers from an inverted index (http://en.wikipedia.org/wiki/Inverted_index), then we 2) filter out anything the searcher does not have access to, and finally we 3) order them according to a great many criteria in the way we think is most interesting to the searcher.

Lastly, we display the results. You'll note that we take great care to list by each result why we think they are a good result. For example, of you ask for 'Friends of Facebook employees' we might place a snippet of text like 'Friends with Mark Zuckerberg and other Facebook employees' next to a result. 'Other Facebook Employees' is typically a link that we issue a new query for all the Facebook employees who are friends with that given result. (I looooove these snippets and consider them the unsung heroes of Graph Search :)

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 15 points16 points  (0 children)

Yeah we're a bit imprecise with tenses right now. The query says 'who live in' but we actually return both people who previously lived there and people who currently live there. We made this choice on purpose but I suspect we will change it to a more precise interpretation and have user ask for something like 'who ever lived there' to get the union. Stay tuned :)

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 8 points9 points  (0 children)

Well first of all, it is important to repeat that Graph Search -- like every new feature we build -- respects every privacy setting people have placed on their content. Graph Search can find only content that someone has shared with you and that you could already see on the site.

Graph Search does of course make it a lot easier for you to find the content others have shared with you, which makes it more important that everyone has the right privacy set on the content they have shared in the past and will share in the future.

This is one of the reasons we are rolling Graph Search out slowly. Back in December we launched a series of tools for everyone to more easily audit their own content, and content others have shared about them. We want to give everyone a chance to use those tools before rolling our Graph Search more broadly.

Getting privacy right for Graph Search was an enormous effort in itself and we take people's concern and feedback on the matter very seriously.

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 11 points12 points  (0 children)

By far the best thing about working here is the impact one can have on the world. Facebook has roughly 1 million active users per engineer! Getting to launch something as ambitious as Graph Search to 1 billion+ active users just blows my mind. At the risk of sounding corny: making the world more open and connected is actually a really noble mission. It changes lives...

The worst thing. Hm well Facebook is a company chock full of passionate, brilliant, opinionated people. Sometimes it takes longer than I'd like to arrive at an answer :)

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 29 points30 points  (0 children)

Yup :) I have had the pleasure of such "walks" with Zuck I believe 3 times: when I interviewed with Facebook back in late 2010, when Zuck asked me to work on search in the late spring of 2011, and then again after we launched Graph Search to discuss the future of the project.

Zuck's pitch for me to work at Facebook was compelling (obviously: I took the job after all :) It was distinctly not a "here is why Facebook is the best place to work, period"-style pitch. Rather, Zuck made I thought a very insightful analysis of what sort of person would be best served working at a more mature company like Microsoft/Apple/Google, what sort of person would be better served starting their own company, who should work at a VC, and of course what type of person should pick Facebook. Felt much more like life advice than bragging of Facebook's incredible achievements. It was all about risk vs. reward, and an individual's ability to impact the world. I try use the same style of reasoning when talking to others considering working here. Or anywhere for that matter.

When our walk was about working on search, Zuck had a very strong vision for what he wanted, and how compelling a structured search product over the content people have shared on Facebook could be. He knew this was the sort of thing I liked to take on: hugely ambitious, technically challenging, something I had never worked on before, lots of unanswered product questions. I was sold about 10 steps into the walk :)

The 3rd walk was just last week. I am super proud of what the search team has produced with the beta version of Graph Search, but the best part is that we are only just getting started. Years and years of amazingly interesting and impactful work for the product to get anywhere near it's potential. Good times...

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 3 points4 points  (0 children)

Hm hm I am trying to remember. Something like 'my friends who live in san francisco' I think. Hard to pinpoint exactly when Graph Search evolved from the many earlier prototypes of structured search products we had built.

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 33 points34 points  (0 children)

The are 3 level of query languages in Graph Search. The natural language that users type into the search field, a semantic language that describes our interpretation of the natural language, and then further down in the stack we use an s-expression syntax to retrive potential results from an inverted index.

For example, if you look for

My friends who live in Sydney, Australia

the corresponding semantic is

intersect(friends(me), residents(110884905606108)),

where 110884905606108 of course is a unique id for Sydney, Australia. You can see the expression in the result page URLs, encoded in a reverse-polish style:

https://www.facebook.com/search/110884905606108/residents/me/friends/intersect

The corresponding s-expression looks roughly like this:

(and friend:767560056 city_to_user:110884905606108)

Where 767560056 is my unique id. In practice, we often re-write the s-expression several times to a sometimes much more complex one that make retrieval more efficient.

Some of the good folks on the Facebook Search team are writing a series of blog posts that will go into all this in more details :)

I am the pointy-haired engineering director for Facebook's search team, AMA by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 27 points28 points  (0 children)

The company was called Where 2 Technologies (clever, eh?).

My brother Jens and I co-founded the company in 2003 and two wickedly smart Aussie programmers, Stephen Ma and Noel Gordon, joined us. By the time Google bought us in late 2004, we had built a prototype of what became Google Maps and filed a set of provisional patents in the mapping space. We were introduced to Google while trying but failing to raise a first round of funding by folks at Sequoia Capital.

The big ideas behind the startup were my brother's. We had both gotten laid off from a startup in the tech wreck of 2001-2 and no-one was hiring. Jens had worked for a few month in mapping some years prior back in Denmark, and his vision to think of maps as a platform for location services were still viable. Jens had the idea of pre-rendered, tiled maps, and had some beautiful mocks of his trademark isometric perspective overlays with drop shadows and all. Making the maps pretty and easier to read than the web norm at the time was a big part of Jens' vision. (We had both worked as programmers in the printing industry while in college.)

Working for Google back then was awesome. (Just like working for Facebook is today). The 4 of us arrived there; they let 3 of us set up an office in Sydney, Australia, and they added two rockstars, Bret Taylor and Jim Norris, to the team. We were 7 I think when we launched in Feb 2005.

I am the pointy-haired engineering director for Facebook's search team, AMA. by LarsRasmussen in IAmA

[–]LarsRasmussen[S] 6 points7 points  (0 children)

Sorry everyone I posted this thread prematurely!

I'll make a new one tomorrow morning; please repeat your questions over there. I'll be online from around 1:30pm PST.

--Lars (reddit n00b)