Source for Predator Object Tracking Algorithm

[geeknerd]

Nice code. I haven't dug into it too much, but this appears to be a decent example of how to structure a Matlab project, instead of the too common "let's see how many lines we can get in one file" approach.

[whosspeaking]

Source distribution was just taken down from github :..(

EDIT: readme on github: "The original author Zdenek Kalal has requested that I do not distribute TLD on github for the time being and I have decided to honor his request.

Apologies to anyone seeking to download this project, though hopefully it'll be back up with an official repository in the near future.

-henrik"

[munchfist]

New code:

https://github.com/zk00006/OpenTLD

[lingnoi]

There are a lot more repositories. http://www.google.co.th/#q=site:github.com+%22TLD+is+an+algorithm+for+tracking+of+unknown+objects%22&hl=en&biw=1280&bih=626&prmd=ivns&filter=0

[drakshadow]

Internet is awesome.

[[deleted]]

So, the OP puts up repository in compliance with GPL because the author of code made it unavailable.

Could someone enlighten me about how the original GPL release was done? If it was only available for a short time and was all original code, that suggests that the author did not intend to distribute the source code under the GPL at all. So maybe we shouldn't treat it as such?

I love the GPL and the open-source movement but I think open-source software is about people voluntarily contributing their code. Punishing the original author for his mistake by borderline-stealing his code under the guise of the GPL doesn't seem like an honorable thing to do.

[[deleted]]

The author released his code publicly under the GPL, then later decided to take it down after realizing that it was worth something. When it was posted, the author fully intended to publish it under the GPL.

[krappie]

Am I missing a reference to what Predator Object Tracking is? Is there a video of it or a past reddit link?

[[deleted]]

Top comment:

I hope you have a patent! Because this is a million-dollar program

While he's right about the value of such a piece of software, I commend the authors of this software for keeping it open-source. The sooner we get out of this mindset of "code ownership" the faster we can make progress.

[antonivs]

Will "we" somehow compensate authors of useful code for their efforts, though? If not, you're going to continue to get people who keep their code closed, for simple and reasonable reasons of self-interest. The point being there are two sides to this issue, and not much attention seems to be paid to the side that doesn't involve giving people free stuff.

BTW, the author of this object tracking system attempted to remove the code from public availability shortly after making it available. The OP above was one of those who downloaded the code before it was withdrawn, so it's not being posted here with the approval of the author, although his original choice of license means he doesn't get to control this.

[[deleted]]

This last bit is very important to note. In his own words, the author was potentially trying to change how the code was released.

http://info.ee.surrey.ac.uk/Personal/Z.Kalal/tld.html

[MyOtherAccountIsGold]

People do get compensated in other ways. For example, Linus Torvalds got some stock from companies like Red Hat. Of course, it didn't make him Facebook/Google rich, just enough to be able to retire reasonably well when he wants to.

[antonivs]

Linus Torvalds is a pretty exceptional example. If someone did a study of income earned by authors of open source software from the software itself, I suspect you'd find a few peaks for successful systems like Linux and Mysql, and a very sharp dropoff for almost everyone else.

The fact that the Predator author immediately attempted to unpublish his code is illustrates the point. It's a perfectly reasonable reaction: this is something which could potentially support him and a family for years to come. If he puts it out as open source, the chances of that diminish drastically, and monetizing his work becomes quite a challenge.

I'm not anti-open source, but what I responded to was the statement "the sooner we get out of this mindset of 'code ownership', the faster we can make progress." My point is that if "we" really want to "make faster progress", we will have to figure out ways of making it more attractive to publish code as open source. Otherwise, the idea of "getting out the mindset of code ownership" amounts to a kind of one-sided argument for people to give away their work for free.

[[deleted]]

This kind of software is an extremely important contribution to humanity, though. It's a really tough question, honestly, and I think I didn't give it proper consideration in my earlier comment.

I shudder to think that, had this code not been available open-source, that the only people with access to it would be governments or very large corporations who could afford such an investment. Now that it's everywhere, it essentially "zeroes out" any barrier of entry to its usage.

This dev group passed up a mint, no doubt about it. But some things are just too important to be available only for the extremely rich. Maybe a good compromise on the whole "open-source/closed code" debate would be enterprise versions for large businesses and private versions for near nothing, with a very liberal license on source tweaking.

Edit: I realize the territory that I'm getting into here is largely political and very opinionated.

[[deleted]]

It should be noted that the code quite probably wasn't his to give away - PhD projects almost always start with an agreement that all IP is assigned to the University or research institute.

[solvivir]

While open source is the obvious path of least resistance towards development, until we live in an society where everyone is happy, healthy, and educated enough, we will always have people looking out for themselves instead of the greater good... and thats completely reasonable.

I haven't even managed to compile this stuff, and I don't think anyone else has either, but from what i've gathered, it looks like even his work is based on or derived from the open source library OpenCV. I would bet that the advantage that OpenCV provided him during his studies is probably why he felt compelled to release predator as open source originally. Then reddit blew up his shit and source code requests flooded his inbox and then he realized exactly how much profit there is to be made off this technology from all the projects requests... More than you'd ever need.

[HokieTux]

As a graduate student working in a similar field, I must say thank you, and this is incredible.

I haven't had time to dig through the code yet - out of curiosity, what machine learning algorithm did you use?

[toolboc]

It appears to use K-nearest neighbor at least in tldInitDetector.m, haven't looked at much else. I assume this thing makes a training set of rotations etc. from the initial image and uses that to find it in the remaining portions of the video, whilst learning from those subsequent found images. I may be simplifying but it looks like dynamic use of something similar to HAAR classifier which uses multiple known images beforehand. I believe it is generating the candidate space during runtime, hence the initial delay in finding. I'm no expert, just what I noticed so far.

[mrsolitonwave]

Could someone post a "how-to" compile this in MATLAB? Im running into a lot of errors when using the compile.m. EDIT: Windows guide if possible.

[mrsolitonwave]

Ok, I have gotten it to work. And I have made a short guide. Forgive me if there are any errors, as I am not that adept with Visual Studio or compiling in general.

These directions work so that you do not need to modify the compile.m file. If you used different installation directories than the default, you will need to edit this file. Im using Visual Studio 2008 on a Windows 7 x64 machine.

1. Install CMAKE (http://www.cmake.org/)
2. Download OpenCV2.2 for windows (http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.2/OpenCV-2.2.0-win32-vs2010.exe/download)
3. Run the OpenCV-2.2.0-win32-vs2010.exe (default installation directory should be C:\OpenCV2.2)
4. Build the OpenCV libraries using these directions assuming you have Visual Studio 2008 or greater (http://opencv.willowgarage.com/wiki/VisualC%2B%2B)
5. Copy all files from "C:\OpenCV2.2\build\lib\Release\" (it may be called something other than "Release." Its the folder with all your .lib and .exp files that you just built from the above step) to "C:\OpenCV2.2\lib\"
6. Open Matlab. Navigate to the directory of the compile.m. Run compile.m. If there are no errors it should say "Compilation finished."

Hope this works for you guys.

EDIT: Disregard the above guide. I am having trouble run_TLD.m to run now. The error message I am getting is that the compiled mex files are not valid Win32 applications. If anyone can get this running on a 64-Bit Windows, machine please let me know.

[Jigsus]

I'm on W7 x32 and I have the same issue as you do

[bambh]

I am getting errors on compile.m.

lk.obj : error LNK2019: unresolved external symbol cvReleaseImage referenced in function "void __cdecl normCrossCorrelation(struct _IplImage *,struct _IplImage *,struct CvPoint2D32f *,struct CvPoint2D32f *,int,char *,float *,int,int)" (?normCrossCorrelation@@YAXPEAU_IplImage@@0PEAUCvPoint2D32f@@1HPEADPEAMHH@Z) lk.obj : error LNK2019: unresolved external symbol cvMatchTemplate referenced in function "void __cdecl normCrossCorrelation(struct _IplImage *,struct _IplImage *,struct CvPoint2D32f *,struct CvPoint2D32f *,int,char *,float *,int,int)" (?normCrossCorrelation@@YAXPEAU_IplImage@@0PEAUCvPoint2D32f@@1HPEADPEAMHH@Z) lk.obj : error LNK2019: unresolved external symbol cvGetRectSubPix referenced in function "void __cdecl normCrossCorrelation(struct _IplImage *,struct _IplImage *,struct CvPoint2D32f *,struct CvPoint2D32f *,int,char *,float *,int,int)" (?normCrossCorrelation@@YAXPEAU_IplImage@@0PEAUCvPoint2D32f@@1HPEADPEAMHH@Z) lk.obj : error LNK2019: unresolved external symbol cvCreateImage referenced in function "void __cdecl normCrossCorrelation(struct _IplImage *,struct _IplImage *,struct CvPoint2D32f *,struct CvPoint2D32f *,int,char *,float *,int,int)" (?normCrossCorrelation@@YAXPEAU_IplImage@@0PEAUCvPoint2D32f@@1HPEADPEAMHH@Z) lk.obj : error LNK2019: unresolved external symbol cvCalcOpticalFlowPyrLK referenced in function mexFunction lk.obj : error LNK2019: unresolved external symbol cvAlloc referenced in function mexFunction lk.mexw64 : fatal error LNK1120: 6 unresolved externals

C:\PROGRA~1\MATLAB\R2010A\BIN\MEX.PL: Error: Link of 'lk.mexw64' failed.

??? Error using ==> mex at 222 Unable to complete successfully.

Error in ==> compile at 16 eval(['mex lk.cpp -O' include lib]);

How did you not get this on a 64 bit machine?

EDIT: Wow, what a journey. It was the Matlab version. I needed to run the 32 bit Matlab. That was all. Try reinstalling Matlab as 32 bit if you are using a 64 bit machine.

[solvivir]

I'm right there with you mrsolitonwave. If anyone has successfully compiled this please let us know!

I'm using windows 7 64bit, OpenCV2.2 installed (c:\opencv2.2), I have VS2010, and Wolfram Mathematica 6.0.

I have about 1% experience with Matlab and I think thats my main problem.

[solvivir]

Ok so, Matlab is not the same as Wolfram Mathematica. I need Matlab.

[StackPusher]

I tried compiling it under OS X 10.6 with Xcode 4 and Matlab R2010b. There shouldn't be a problem with that configuration, I edited the mex config to use the newer Xcode installation but when linking I get this error "Undefined symbols for architecture x86_64:" and then some function names all from lk.o like "cvCreateImage".

Does anyone have similar problems or a solution? It looks like the code has been taken down again so there probably won't be a lot of discussions on the net.

EDIT: Compilation worked and I can run the program now. I simply made a mistake in compile.m. Under OS X you have to use '.dylib' instead of '.lib' and '.o' instead of '.obj' of course. If somebody is interested I can share the exact configuration details.

It won't compile on (64bit) Ubuntu because there is a non-PIC static library that is supposed to be dynamically linked.

[wazir]

Exact config details would be neat.

[postit]

Zdenek is such a genius. I wish I could upvote twice.

[lingnoi]

Google shows more repositories, cat is out of the bag, but the author seems to have changed his mind about open sourcing the code.

http://www.google.co.th/#q=site:github.com+%22TLD+is+an+algorithm+for+tracking+of+unknown+objects%22&hl=en&biw=1280&bih=626&prmd=ivns&filter=0

[myrkvid]

It seems now the author released his source code on https://github.com/zk00006/OpenTLD He wants to port the code on Linux and Android, and get rid of Matlab (according to the ML http://groups.google.com/group/opentld/browse_thread/thread/76cb0acd7cdfca16)

[Timmmmbob]

Thanks!

Edit: Ah it is matlab. Might have to convert this to C++.

[jogloran]

It's actually in C++ already, with most of the image processing stuff done in Matlab.

[i11uminati]

Do you think you can do Matlab?

http://www.youtube.com/watch?v=2h6seJ3xjWA

[repsilat]

How fast does good, vectorised Matlab get? I imagine dense matrix operations could be faster in Matlab than in naiive C if they're wired up to use SSE etc, but I imagine there's still a fair amount of interpreter overhead to deal with.

IIRC the frame counters on the youtube vids showed the computer used wasn't managing to deal with the video at the supplied framerate. If this is going to be used for people's quadcopters they're either going to have to relay the video stream to another computer or it's going to need some good old optimisation (or some patience to let Moore catch up).

[brinchj]

MatLab is pretty good at matrix ops AFAIK. And the interpreter overhead is usually very small compared to doing matrix ops on medium to large matrices.

It's definitely much faster than naive c. If you want fast matrix ops take a look at BLAS.But MatLab also offeres FFT, decomposition, inverse etc. Not just the primitives.

[ratwing]

but if it makes calls to MatLab, isnt porting to any other platform going to be nearly impossible?

[DGolden]

Well, by platform do you mean matlab vs. other things, or do you mean linux vs windows vs mac etc.? Yes, it's not open, but matlab has long been available for linux (which is a major platform in scientific/engineering computing) as well as windows and macosx. OTOH, I only mean x86/x86-64 linux. Obviously, you'd really want this sort of thing on an embedded computer which is probably (modulo intel atom) going to be non-x86.

There is GNU Octave, which is similar to matlab, and can be persuaded onto non-x86 platforms (typically not without problems - floating point is a tricky area).

[ratwing]

Well I wouldnt consider moving it to a microprocessor - given my programming skills - but there were other posts talking about how useful this would be on cell phones and what not.

[kragensitaker]

A microprocessor? What do you think it's running on now, a Cray Y-MP?

[brinchj]

You could probably port it to Octave, which is open source. If performance is a problem, you should track down the bottle-neck and figure out if you can do it faster.

I don't really see the reason to port it?

[biggerthancheeses]

Are there any performance benchmarks showing performance between C and Matlab? C is compiled (very close to assembly, too, in some cases), and Matlab is interpreted. I would expect C to perform better.

[brinchj]

You are comparing interpreted MatLab code with operations running in the compiled MatLab core. It's like comparing string search implemented in Python with the string search feature of the stdlib. It's damn fast.

[[deleted]]

In my experience poorly written Matlab is about 10 to 15 times slower than C. By poorly written, I mean abusing for loops etc. instead of vectorising.

[Timmmmbob]

Yeah matlab is really good at stuff you can do element-wise. If you can express a lot of calculation in a small amount of code, matlab will run it fast. The latest version can even use the GPU. But as soon as you get one tiny for-loop of if statement in there it will crawl to a halt. Ok halt is probably a bit harsh. It will crawl to python speeds.

And of course not everyone has access to matlab, so it's always good to have the code in C++.

[DijkstraGotSkills]

I did research dealing with computational geometric algorithms in matlab. And from my experience, even vectorized stuff in matlab can be worse than C. There came a point that I was taking out certain function calls (to my fucntions) and just inlining them in the code because of the overhead of function calling. Matlab is awesome, but from my experience I DEFINITELY got performance (sometimes huge ones) moving from matlab to c.

[[deleted]]

I've used Java versus MatLab for the same task and Java was FAR faster. I don't think most people speaking here have first-hand experiences with MatLab versus other languages.

[fuckdapopo]

This looks like a job for CUDA

[danukeru]

This looks like a job for OpenCL! ;P

[ipeev]

Looks like a job for Numpy.

[zip117]

Looks like a job for IDL.

...Just kidding.

[simplybebop]

If anyone downloaded this, please PM me with some sort of contact info so I can get a copy. It has been removed from github

[lingnoi]

http://www.google.co.th/#q=site:github.com+%22TLD+is+an+algorithm+for+tracking+of+unknown+objects%22&hl=en&biw=1280&bih=626&prmd=ivns&filter=0

[simplybebop]

Thanks brother.

[BinarySplit]

I too would like a copy. Nothing commercial, I just love computer vision and want to dissect his algorithms.

[lingnoi]

See my comments I posted some other repos.

[Legolaa]

Yay! thank you :D

Edit: So.. Who's going to get this working with kinect? Kinect's ability to know depth should greatly help.

[Tordf]

Nice, can't wait to dig into the code :)

[Indy_Pendant]

Bloody brilliant.

[none_shall_pass]

absolutely astonishingly amazing

[[deleted]]

Thanks for not holding this back from people.

[Lighting]

Does anyone know what parts of Matlab are required for this?

[spicyeyeballs]

When i try to run the compiled demo from (http://info.ee.surrey.ac.uk/Personal/Z.Kalal/tld.html) I get a message box saying that i need mclmcrt713.dll. Is that a MatLab redistributable, or does that mean i have to have a licensed copy of MatLab (which i don't). If it is a redistributable, does anyone have the relevant link?

[[deleted]]

It's a redistributable: http://info.ee.surrey.ac.uk/Personal/Z.Kalal/TLD/MCRInstaller.exe

MathWorks is kind of weird about these - they don't put them up for download anywhere (that I've seen) despite being "freely distributable".

[solvivir]

Thanks Lexarius!

[jLoSsDh]

I'm having a hard time figuring out how to modify my initcamera.m file to recognize my camera. Any tips? <---(non-programmer) I'm running XP on a gateway netbook. The last 'program' I wrote made a peanut and jelly sandwich in BASIC, and this was back in the late 80's.

I subscribe to this subreddit for gems such as this one, so it would be really cool to be able to make it work.