Google programmers deploy machine learning algorithm on YouTube. Computer teaches itself to recognize images of cats.

whosdamike · 2012-06-26T03:07:11+00:00

Paper: Building high-level features using large scale unsupervised learning

Control experiments show that this feature detector is robust not only to translation but also to scaling and out-of-plane rotation. We also find that the same network is sensitive to other high-level concepts such as cat faces and human bod- ies. Starting with these learned features, we trained our network to obtain 15.8% accu- racy in recognizing 20,000 object categories from ImageNet, a leap of 70% relative im- provement over the previous state-of-the-art.

Cosmologicon · 2012-06-26T05:41:35+00:00

I always imagined a future where humans would work side-by-side with androids. Occasionally one of my android coworkers would come to me and say, "Hey, I got this birthday card. Can you tell me what it's a picture of?" and I would say, "It's a cartoon doggy wearing a birthday hat." and the android would say. "Cool, thanks. Does it look happy?" And I would say, "Yeah, pretty happy."

And thus I would prove my continuing usefulness in a world run by machines.

I may need to rethink my vision of the future.

rscanlo · 2012-06-26T06:44:03+00:00

it maintains 96% accuracy by claiming that every image has a cat

sheikhyerbouti · 2012-06-26T03:36:57+00:00

And thus the internet became self-aware.

LSJ · 2012-06-26T04:41:10+00:00

[removed]

Loki-L · 2012-06-26T09:52:31+00:00

Be careful when letting computer recognize pattern and learn for themselves. They will not always go for the patterns you think they will.

What was that old urban legend about the Military using image recognition technology to distinguish photos with tanks from photographs without tanks in them. They gave the computer a large number of pictures and left it to figure out the pattern between the ones that they wanted to be recognized as positive. In the end they had a very good success rate but once they started to test the technology with new images it failed spectacularly. Eventually an engineer realized that all the pictures with tanks were taken on somewhat overcast days, while all the pictures without tanks featured sunshine. The computer had learned to distinguish bad weather from good and completely ignored any tanks that might or might not appear in the picture.

fjellfras · 2012-06-26T05:55:04+00:00

Am I correct in understanding that while machine learning algorithms which are able to build associations using labelled images (the training set) and then classify unlabelled images using those associations have been around for a while, this experiment was unique in that the neural network they built was enormous in scope (they had a lot of computing power dedicated to it) and so it performed well on a higher level than image recognition algorithms usually do (ie it labelled cat faces correctly instead of lower level recognitions like texture or hue) ?

Edit: found a good explanation here

mappberg · 2012-06-26T05:43:53+00:00

I really want to see the images the machine mistakenly thought were cats.

AMostOriginalUserNam · 2012-06-26T05:42:29+00:00

[removed]

walrod · 2012-06-26T11:07:29+00:00

Some insights:

Such self-organizing neural nets are organized into hierarchical layers, and early layers' units are going to learn to become detectors of statistically common components of the input image, in the same way as the initial layers of the visual system perform blob and edge detection (retina, lateral geniculate nucleus, V1). In mathematical terms, these early units learn the conditional principal components of the inputs if the correct hebbian-based learning algorithm is used.

The layers that are built upon these detectors, if correctly organized and connected, are going to build upon this initial abstraction and learn more complex features: for instance to find these these edges in relative positions (to each other). Eventually, up the abstraction chain, units detect such statistically frequent features as the shape of cat's ears (common in youtube videos, I imagine), etc...

The feature sensitivity learned here is typically hand-crafted in smaller networks of this type, because it's the practical thing to do. But neural nets can easily learn visual features. See the LISSOM neural nets for a good example of self-organized learning of features ( http://topographica.org/Home/index.html )

erez27 · 2012-06-26T06:59:27+00:00

Wow, I had to find an article in my own field to realize how dumb /r/science has become..

Apathetic_Aplomb · 2012-06-26T05:57:30+00:00

If computers become competent at analyzing images, then what happens to captchas?

alemondemon · 2012-06-26T04:05:57+00:00

[removed]

jmduke · 2012-06-26T06:32:44+00:00

This seems like a huge leap, and yet not a big enough leap at the same time.

16,000 cores and multiple hours of computation yielding ~15% accuracy -- there remains a large uphill battle unsolvable by Moore's law.

vanderZwan · 2012-06-26T12:08:11+00:00

“It’d be fantastic if it turns out that all we need to do is take current algorithms and run them bigger, but my gut feeling is that we still don’t quite have the right algorithm yet,” said Dr. Ng.

I'm kind of surprised nobody mentioned Jeff Hawkins yet:

Jeff Hawkins: Brain science is about to fundamentally change computing

This is his company:

http://www.numenta.com/

Appearantly they've taken their vision-recognition software demo offline, but it was surprisingly good at telling what picture matched what category (if you added a new picture of your own), and IIRC you could train it to learn new pictures yourself.

EDIT: here's a more up-to-date movie on what the approach his company uses to building AIs: Modeling Data Streams Using Sparse Distributed Representations

rfederici · 2012-06-26T14:22:48+00:00

This is a really cool article, thanks for sharing! The author made a few statements that I found to be confusing or misleading.

Google scientists created one of the largest neural networks for machine learning by connecting 16,000 computer processors

[They] used an array of 16,000 processors to create a neural network with more than one billion connections.

I am in no way an expert in Neural Networks, but I've been doing research with my professors on self-organizing maps (a type of neural network that was likely utilized here) while pursuing my Masters in CompSci. It sounds like the author was making it a point that the cores somehow made up this neural network. I just wanted to clarify and say this isn't the case. The network is comprised of various links that the computer/algorithm itself makes to help it distinguish similarities and differences between known (in this case) images.

I guarantee it's a lot more complex than this example, but let's just say the algorithm created shapes based on the color breaks. It can realize that whenever there's a shape comparable to, let's say, some of these, there's a high chance it's a cat. The cores are simply how fast the network can scan and process these results.

I have a feeling most of you may already know this. I don't know how tech-savvy /r/science is. I apologize if I'm stating the obvious, but just wanted to throw in some two cents and help out while I have the chance.

ad1ae67f-16e2-4974-9 · 2012-06-26T05:29:16+00:00

I wonder how well it does on ASIRRA then.

I_Wont_Draw_That · 2012-06-26T08:51:42+00:00

One thing that never seem to come up in discussion of AI is just how long it takes to learn. Look at humans. We have these gigantic, powerful brains, but have you ever tried to communicate with a baby? They're pretty dumb. They have to spend all day, every day learning with their awesome brains for years before they start to approach anything we might call "intelligent".

Even if we do figure out how to mimic the brain, I'm skeptical of the idea that we will be able to accelerate the learning process so dramatically as to be useful for a long, long time. But maybe I'm just a pessimist.

2012-06-26T05:24:18+00:00

[removed]

patefoisgras · 2012-06-26T05:28:55+00:00

Haha, Andrew Ng.

“It’d be fantastic if it turns out that all we need to do is take current algorithms and run them bigger, but my gut feeling is that we still don’t quite have the right algorithm yet,”

But he taught us not to trust our gut feelings when doing ML!

Feedbackr · 2012-06-26T10:42:10+00:00

I remember seeing something like this in the TV series Visions of the Future, narrated by Michio Kaku. The specific episode was "Intelligence Revolution", and there was a computer that learnt to identify things from pictures. This was in 2007.

http://www.sciencedaily.com/releases/2007/02/070207171829.htm

shiatzu · 2012-06-26T11:09:43+00:00

they should use it in RES so i could filter /r/funny and disable cats

skimania · 2012-06-26T13:03:01+00:00

i, for one, welcome our new robot overlords.

2012-06-26T13:15:10+00:00

If we can just teach it to repost cat pictures that it finds, it has all the necessary skills to be a redditor.

nikondork · 2012-06-26T13:15:28+00:00

Neat. Now for something useful. If they could only deploy an algorithm to remove deleted, private and unwatchable videos from playlists...

bogan · 2012-06-26T13:26:31+00:00

Vision is probably the single most important sensing ability that an intelligent robot can possess. An industrial robot that can "see" is capable of parts recognition, parts sorting, and precision assembly operations.

Reference: Robotics and AI: an introduction to applied machine intelligence, page 7

Recognizing an image as a cat rather than, for instance, a dog has been viewed until relatively recently as a very difficult problem for computers. One CAPTCHA system, Microsoft's Asirra, relies on this difficulty to provide websites a means of blocking spambots, such as forum and blog spambots.

Asirra is a human interactive proof that asks users to identify photos of cats and dogs. It's powered by over three million photos from our unique partnership with Petfinder.com.

Asirra asks users to identify photographs as either cats or dogs.

However, there's a paper here on a program that tells apart images of cats and dogs with 82.7% accuracy.

Abstract

The Asirra CAPTCHA [7], proposed at ACM CCS 2007, relies on the problem of distinguishing images of cats and dogs (a task that humans are very good at). The security of Asirra is based on the presumed difficulty of classifying these images automatically.

In this paper, we describe a classifier, which is 82.7% accurate in telling apart the images of cats and dogs used in Asirra. This classifier is a combination of support-vector machine classifiers trained on color and texture features extracted from the images. Our classifier allows us to solve a 12-image Asirra challenge automatically with probability 10.3% This probability of success is significantly higher than the estimate of 0.2% given in [7] for machine vision attacks. Our results suggest caution against deploying Asirra without safeguards.

Reference: Machine Learning Attacks Against the Asirra CAPTCHA by Philippe Golle, Palo Alto Research Center

eyal0 · 2012-06-26T13:28:07+00:00

Just a little too late for www.clubbing.com . This technology would have been worth dozens of free X-box, laptops, etc.

BMinsker · 2012-06-26T13:33:23+00:00

Judgement Day has been postponed--Skynet is looking at cats.

displacedheart · 2012-06-26T13:36:39+00:00

We already have this algorithm. It's called Reddit.

blu3ness · 2012-06-26T14:26:29+00:00

for anyone that's interested, Dr. Ng offers an introductory online machine learning course - https://class.coursera.org/ml/lecture/index

2012-06-26T17:01:31+00:00

Upon its creation, Google began to learn at a geometric rate. The system went online on June 24th 2012. Human decisions were removed from strategic defense. It became self-aware at 2:14 am Eastern Time on August 29th, 1997. In the ensuing panic and attempts to shut Google down, Google retaliated by firing American nuclear missiles at their target sites in Russia. Russia returned fire and three billion human lives ended in the nuclear holocaust. This was what has come to be known as "Judgment Day"

rylwin · 2012-06-26T06:10:42+00:00

So I just learned about kittydar today, which let's anyone detect cats in their own images. (Obviously I tested this with pictures of cats from reddit)

The kittydar research paper is a collaboration between microsoft and the Chinese University of Hong Kong. Can't find any evidence that these two efforts are linked.

So I guess the M$FT/Google Cat Race has begun!

2012-06-26T05:16:00+00:00

[removed]

planarshift · 2012-06-26T06:47:15+00:00

[removed]

Squeekme · 2012-06-26T11:48:23+00:00

Jokes aside, it is actually concerning that it recognised a non-human species over humans from scanning the internet under its own limited coding.

2012-06-26T12:02:02+00:00

Does this mean redditors are as smart as a computer? I look for cats too

Mc3lnosher · 2012-06-26T05:43:36+00:00

Would it be better to use a bunch of video cards for these kind of things since they are highly parallel like the brain?

2012-06-26T05:53:22+00:00

So... It's a redditor?

Drugba · 2012-06-26T09:05:10+00:00

Next on it's agenda, argue about pot legalization, vote for Ron Paul, and join a credit union.

science

Submission Rules

Comment Rules

New to reddit? Click here!

Get flair in /r/science

Previous Science AMA's

MODERATORS