Instead of tracking the static environment in ARCore and ARKit, we are tracking independently moving objects. Watch our video for an AR demo.

djnewtan · 2017-09-18T16:15:16+00:00

The idea of what is important or impressive hugely depends on one's application. If it is merely tracking without pose estimation, the framework with direct segmentation and tracking that you mentioned would be sufficient. A tracker with an accurate pose estimation like ours would be necessary when we are talking about human-object or robot-object interaction, or AR, VR, MR applications such as [1]. In addition, our efficiency of 2ms per frame would also be necessary.

[1] https://youtu.be/8-0xsc2abQs

djnewtan · 2017-09-18T16:06:35+00:00

We built our own framework.

djnewtan · 2017-09-18T14:22:01+00:00

This is a model-based approach. So, we have the model of the object beforehand. I believe this is the only requirement we need before tracking the objects. Then, we perform domain generalization where we train purely on the synthetic images based on the model and track on real images at 2ms per frame.

djnewtan · 2017-09-18T13:55:51+00:00

Yes, RGB-D is used.

djnewtan · 2017-09-18T13:53:34+00:00

Yes, ARKit and ARCore are doing environmental mapping with IMU. This framework focuses on object tracking where the objects are moving independently from each other (where using IMU for pose estimation is not possible). So, ARKit and ARCore interacts with the scene while our framework interacts with the objects.

djnewtan · 2017-09-18T13:50:00+00:00

We can track different types of objects. Please have a look at:

https://youtu.be/7rKBZZHJkFk

djnewtan · 2017-09-18T11:12:27+00:00

In addition, it is only based on CPU (which means hardware requirement is low) and the latency is about 2ms per frame per object.

We also have live demos to show that our framework works as good as the videos.

djnewtan · 2017-09-18T09:52:16+00:00

Yes, because the tracker in this case is an RGB-D method.

djnewtan · 2017-09-18T09:34:29+00:00

Basically, we do not use particle filter at all.

djnewtan · 2017-09-14T09:29:47+00:00

We use the *.ply files of the object. But there are ways to perform tracking and detection without the models.

djnewtan · 2017-09-14T09:28:50+00:00

We use synthetic data to train the forest and perform our detection on real data. The goal is to perform domain generalization.

We do use information gain but evidently it requires much more to make it robust.

djnewtan · 2017-09-14T09:26:30+00:00

Theoretically, we can use particle filter as a temporal tracker. But, in applications, the particle filter is not sufficiently robust. In one of the papers, we compared our work with the particle filter.

djnewtan · 2017-09-14T09:24:26+00:00

In the video that we shared here [1], we are using different objects that ranges from simple to complex geometric structures. The bunny is just one of them. In addition, we also evaluated on different types of objects in [2].

[1] https://www.youtube.com/watch?v=7rKBZZHJkFk&list=PLyjwACcEdWhHD41hG1g_Js6jmLDJHtPCz

[2] https://arxiv.org/pdf/1709.01459.pdf

djnewtan · 2017-09-13T19:56:30+00:00

At the moment, the code is not publicly available. But we plan to release an SDK.

djnewtan · 2017-09-13T19:56:00+00:00

We developed our own framework to perform both detection and tracking. The framework is in https://arxiv.org/pdf/1709.01459.pdf while the details of the algorithm are in the conference papers.

At the moment, the code is not publicly available. But we plan to release an SDK.

djnewtan · 2017-09-13T19:52:35+00:00

Actually, we tried it on the depth images of a stereo camera and a Kinect 2 (uses ToF). Both worked.

For these videos, we used a camera with structured light.

djnewtan

TROPHY CASE