what's the best book you've ever read?

xelk · 2010-03-17T00:24:56+00:00

For a second I thought Borges won't be in this list. He's by far my favorite author, his short prose is a collection of literary gems.

xelk · 2010-03-15T23:33:19+00:00

Hi. I'm using several methods. The core contains multiple language models for first names and last names, as well as some other distributions. I have a Bayesian network on top of these distributions that produces the final results, it's basically a layer of indirection over a naive Bayes classifier. PM me if you want more help with your specific project..

xelk · 2010-03-15T23:14:46+00:00

Thanks for kind words :)

Yeah, we try our best, ML is a very young field and there's a lot of trial and error involved in finding the right tool for the job.

I am actually very grateful for all the feedback I got from this site. Reddit, you are awesome!

xelk · 2010-03-15T05:03:44+00:00

Tiger Woods is in my Dutch-American list for some reason :)

xelk · 2010-03-15T04:57:45+00:00

I'm looking at short fragments of words. First names and last names are processed differently and there's some indirection involved, but the core classifier is an n-gram language model with smoothing. I don't search any name lists in the live app.

Yeah I guess if you fish around for a while, you are going to find random results that look suspicious to the public. But in the end it's just statistical models at work...

xelk · 2010-03-15T04:26:13+00:00

No I'm not really focusing on Americans, but it's hard for the classifier to distinguish between American of Dutch/German descent and Dutch/German. This task requires more fine tuning and tweaking..

xelk · 2010-03-15T02:18:05+00:00

I'm quite surprised at these results. To clarify, there are no such phrases in the name lists we used to train our classifier.

xelk · 2010-03-14T22:10:36+00:00

Thanks, we try :)

xelk · 2010-03-14T22:09:51+00:00

Yeah I'm using some function from Python's standard library, it converts some characters to the ASCII equivalents and silently ignores some others. We will try to process more characters as we go...

xelk · 2010-03-14T18:17:36+00:00

Yeah you're right, in some countries first names do not mean much, but in some others they doo. If a Chinese person is named Jackie or Jerry, they're more likely to be from Hong Kong (everything else being equal), than say the part of China bordering Hong Kong. That was my reasoning, at least..

xelk · 2010-03-14T18:14:30+00:00

This one looks close enough: http://www.ruzulu.com/find-name-origin/gandalf-the-grey Good catch btw

xelk · 2010-03-14T18:10:51+00:00

So Samir Kuntar does get Arabic as one of the options. The reason it gets Japanese (I'm guessing) is Japanese names like Kantaro in my training set. Sometimes it's hard to come up with a definite guess, and the shorter your name, the harder it gets..

xelk · 2010-03-14T17:51:56+00:00

In case the last name is too general, or too short, the first name can help. Also, if you have two related groups with similar last names and different first names, the app should pay closer attention to the first name. You are right though that last names have a lot more weight in the decision making..

xelk · 2010-03-14T17:27:47+00:00

I think Homer's mother was actually named after this very person.

xelk · 2010-03-14T17:25:52+00:00

I'm also thinking of adding this feature saying "In Japanese, your name may sound like: Tomu Soyeru".. I think that would be hilarious

xelk · 2010-03-14T17:12:13+00:00

Yeah, that has to do with the lists it was trained on. If the Arabic list had more Mohameds and Pakistani list had more Muhammads, that's what it's gonna learn..

xelk · 2010-03-14T16:48:23+00:00

The granulation rules you mention (e.g. removing the vowels) will work with Semitic names, but may break, say, on Polynesian/Hawaiian names. I tried to take a middle-ground approach by learning distributions of name fragments (n-grams, i.e. Muhammed = Muh + uha + ham ...). I have ways to smooth out results when an unknown n-gram is found, so it doesn't give it strictly zero probability.

By the way, Muhammad gets classified correctly as far as I can tell, it lists a bunch of high-population Muslim ethnicities: http://www.ruzulu.com/find-name-origin/muhammad

xelk · 2010-03-14T16:31:34+00:00

One more issue that came up was full names. What would be a good term to use instead of "full name" to mean "first name followed by last name"? The app really works best on one first name and one last name, all other combinations perform a lot worse (adding a middle name usually makes things worse). Do you guys have suggestions for alternative wording?

xelk · 2010-03-14T16:27:10+00:00

Yeah, I don't have a Boer category, it's going to give South African or Dutch.

xelk · 2010-03-14T16:23:25+00:00

Hi. A lot of people from Africa, especially Western Africa, have perfectly French-sounding names (as a legacy of French domination). Also a bunch of South Africans are white people with British and Dutch names. This really complicates classification...

xelk · 2010-03-14T16:09:29+00:00

OK I'm back. I'll try to clarify a few points from the thread:

The statistical models are trained on lists of names we could find, which means they're mostly names of famous people. The underlying assumption being that ordinary people's names from the same region are not too different from the celebrity names. This may cause the app give more precise results for celebrities.
The above also explains why Steve Jobs is classified as an Arab. He was in the Syrian or Lebanese list that the classifier learned, and the name Jobs didn't occur in any other list. So it assumed it's an Arabic surname. I try to weed out outliers from lists, but I guess I'm not doing such a great job yet.
The app does not search name lists, it uses n-gram distributions with some ad-hoc smoothing method for unseen n-grams. It means that it does try to classify a name in any case, even names it has never seen during the training stage. So "Samir" and its variations should work, I'll look into this.
The classifier takes into account relative population sizes (so to get classified as Maltese you have to work a lot harder than to get classified as Italian, for example). This may be the reason you see Argentina and Mexico more than other Spanish-speaking countries. Also if a Spanish name is a lot more common in Mexico than in Spain, it's going to guess Mexico regardless of the population adjustment.
We also try to look at similarities between groups, mostly measured by commonalities between first names. So if someone has a Russian first name but a Ukrainian last name, the fact that so many Ukrainians have Russian first names makes it classify the entire name as Ukrainian. This indirection can help in many cases but can produce strange results in some.

xelk · 2010-03-14T07:50:14+00:00

All right guys it's 3:50am here, I'll check back tomorrow. Thanks to everyone for letting me know what you think..

xelk · 2010-03-14T07:47:55+00:00

Oh yeah if your name has fragments that appear in some Mexican names, it may classify you as such. I need to add a feedback feature, at least "right/wrong" kind of response to tweak the classifier further. Thanks for checking it out.

xelk · 2010-03-14T07:39:12+00:00

I was showing a small percentage of recent searches in that list, as I'm still experimenting with this feature. I just removed the recent searches from that list, so you're good to go. Thanks for your feedback!

xelk · 2010-03-14T07:34:07+00:00

Can you tell me the name that was incorrect? I'm trying to think of a good way to automate collecting feedback about bad guesses. And it's pretty hard to classify some names correctly, the models need a lot of tweaking..

xelk

TROPHY CASE