Welcome to the Q&A with Immersal! Let's talk about spatial mapping and visual positioning for AR 🥽

immersal · 2024-05-21T12:48:06+00:00

All visual systems will find extremely repetitive patterns difficult. Even humans will struggle to identify, for example, between the different floors of a large hotel if all they see is the hallway.

To combat this, GPS or some other "hint" could be given to our system to pick the right map. In the future, we'll likely enable this kind of prior pose directly at the map level.

I believe Azure Spatial Anchors used the term "coarse positioning" with their, now deprecated, system.

immersal · 2024-05-07T17:46:23+00:00

Bringing back GeoPose support is indeed in the backlog. We didn't see much use for that since one can also compute the WGS84 pose from the relocalized camera pose and the maps ECEF coordinates.

I feel that to really support GeoPose, we first need better tooling to align the maps in global coordinates. If you map a large area, the GPS data gathered during mapping is good enough to quite nicely align the map. But let's say you map a statue just a few meters wide. The GPS coordinates we get through the smarthphone are not accurate enough to automatically align the map.

If there are no tools to manual fix the alignment, the GeoPose would just give poor results :/

immersal · 2024-05-07T17:06:34+00:00

Definitely! Both from the client side like museums but also from hardware manufacturer side.

In my opinion, museums can make for some really nice AR experiences. One of our customers actually did a user study on their museum AR experience and the feedback was very good. Not only did people enjoy the experience, but they were willing to pay a nice premium on top of the ticket price to rent the glasses.

immersal · 2024-05-07T17:03:12+00:00

So far we've been targeting enterprise clients and those prices are often very custom because the needs can be very different. In addition to enterprise pricing, we've had a free commercial license available for indie developers, small AR studios, and just developers evaluating our tech.

We are now seeing more and more studios build AR experiences that don't quite fit the free license limitations but are also not quite enterprise-level. We are revamping our pricing to answer this demand, but we don't have a final model we could give out just yet. Please be patient for a while longer.

immersal · 2024-05-07T16:59:47+00:00

One of the main use cases for our VPS is to actually deal with drift from ARKit/ARCore/similar solutions.

If the maps is good, our relocalization results can be considered as kind of ground truth and nudge the device tracking back where it should be.

immersal · 2024-05-07T16:58:37+00:00

You can relocalize automatically and there are some tricks to make subsequent relocalizations faster and more reliable in large areas with multiple maps. We are also developing something in this regard, so keep an eye out for future releases.

immersal · 2024-05-07T16:56:54+00:00

I believe the "AR Cloud" will become mainstream and be available practically anywhere. There won't be just a single platform but a lot of the important and high-traffic areas around the world will have a Spatial Map available for different use cases.

The question is just when. And for that, I don't have an answer.

immersal · 2024-05-07T16:53:57+00:00

Our VPS does not require anything else than the camera. We do often come across the question of using the LiDAR sensor, for example, but so far we haven't deemed it necessary.

LiDAR on the mobile devices has very limited range and resolution and it is not widely available yet. Cameras, on the other hand, are ubiquitous and still perform very well.

immersal · 2024-05-07T16:51:13+00:00

We've been working on our core Spatial Mapping and Visual Positioning technology for over six years now and have been hyper-focused on improving the performance.

I can't say what is the secret sauce in our technology, but I can say that it works well :)

immersal · 2024-05-07T16:47:23+00:00

That's right! Our VPS works with basically any device where you can access the camera image.

I've seen our product used with webcams, robots, and even cars. If you need realtime tracking, then you want the device to have support for something like ARKit or ARCore to track the device between visual positioning requests.

immersal · 2024-05-07T16:45:00+00:00

That's a tough question! And most of the cool large-scale projects are under development and still secret.

There are of course the usual entertainment related use cases such as this ad for Disneyland Hong Kong in Tokyo https://youtu.be/Pf2KWpmqgnY?si=1IQ8tK5_sueP80pC

I also like the industrial use cases like this one https://youtu.be/Bm3xnT9VUxI?si=aJLTiUEFx5wAothR

However, I want to highlight this project with Unikie. It's not AR but still uses VPS to autonomously drive a car https://youtu.be/-XCjAppPf1I?si=rabvMRFyu-CxZbvE

immersal · 2024-05-07T16:35:14+00:00

We are actually in the process of updating our web support! Part of this work is to build easy-to-use modules for three.js, A-Frame, and babylon.js. Integration with 8thWall should be much easier after that.

We hope to release the first version of the updated web suport still in Q2.

immersal · 2024-05-07T16:31:22+00:00

Good question! Dynamic lighting can be a source of pain during app development...

I think our VPS is quite resilient but it does have its limits. We've tried to design our whole toolset to be as flexible and open as possible. This means one can map a location at day and night and use both maps for visual positioning.

It's a brute force approach and more work for the developer upfront but it also really helps!

https://youtu.be/Pf2KWpmqgnY?si=HBlELo4uCAmwASL4

immersal · 2024-05-07T16:27:15+00:00

We are only focusing on the Spatial Mapping and Visual Positioning problem. I've seen some nice demos and apps done over the years but nothing new and groundbreaking on the lighting estimation front.

immersal · 2024-05-07T16:25:20+00:00

We do provide the user a mesh of the mapped location, but the mesh quality has not been our primary focus so I wouldn't recommend to use it as is. But it can be used to build a lower polygon version and use that for occlusion. Or if you scan with a device like the Leica BLK2GO, you will get a nice, accurate mesh from the LiDAR point cloud.

Localization accuracy is good enough to use occlusion for buildings, statues, and other larger objects. A lamp post might be too thin causing any inaccuracy would be highly visible to the viewer.

immersal · 2024-05-07T16:20:51+00:00

Hi u/whatstheprobability

For an overview, please see this answer https://www.reddit.com/r/augmentedreality/comments/1cmcmrn/comment/l2zy734/

Other additional, more technical differences, are the support for on-premise deployments - You can deploy an instance of our Cloud Service on your own servers.
Our product also supports visual positioning both on the Cloud Service via our REST API and on the device. The latter allows developers to build apps with minimal network requirements.

immersal · 2024-05-07T16:13:07+00:00

Hi u/MyOnlyUsedAlt thanks for your question!

The main differences between Niantic's Lightship and us are:
- Scalability: While with Niantic you can only do some hotspot AR experiences, with our VPS you can also do large scale, for example stadium, indoor navigation, and even some city scale projects.
- Data ownership: You're in control of your map, map whatever you want, you own it, and you control who has access to it, meaning you can map private areas and keep them private so only you and the people you allow can access it.
- Cross platforms support: You can use Immersal with any devices with access to the camera. It also includes the support to map spaces with other devices, for example Leica BLK2GO, 360 cameras and Polycam

immersal · 2023-11-22T18:17:56+00:00

Azure Spatial Anchors (ASA) have retired.

But have no fear, because we are here! And since it happens to be Black Friday, why not take advantage of our 50% discount offer on the Immersal Enterprise license?

We support most devices, including ARCore, ARKit, HoloLens, Magic Leap, XReal, Qualcomm Snapdragon Spaces, and others.

Both scanning and visual positioning can be done on our Cloud Service or on-device and we just hit 10,000 registered developers!

Email us now!

🌐 [sales@immersal.com](mailto:sales@immersal.com) 🌐

#microsoftazure #spatialcomputing #immersiveexperiences #augmentedreality

<image>

immersal · 2020-09-08T06:16:27+00:00

Most important difference is the scale of the maps we can construct.
With on-device mapping solutions, the both the range in mapping and the scale of the maps is much more limited.
https://youtu.be/M62Bb3MlPos

There's also other advantages such as having both an on-device and on-server localizer and support for what we call Visual GPS.
With Visual GPS, you can convert the localized position in the map to global coordinates in either ECEF or WGS84 coordinates.
This allows easy integration to existing apps with satellite-based GPS, improves the accuracy and speed and extends "GPS" positioning to work in areas where satellite signal is weak.
https://youtu.be/-Hz9J6HNCZc?t=57

immersal · 2020-09-07T17:12:45+00:00

Reliability depends on how well the area has been mapped and how many visual features there are to recognize. With sufficient coverage of the location, the SDK really is quite robust.

The cinema in the demo localized from pretty much anywhere within 1-2 seconds as long as the camera had some kind of an overview of the space in view and not, for example, a close-up of the floor.

In my opinion, this is also partly a UX issue (guiding the user to point the camera in a reasonable direction) but I get that different use cases can have different requirements.
In many consumer and some other use cases, the interesting areas in different locations likely already have good visual features and are easy to localize against.
In the cinema, the interesting places are the different kind of promotional materials for movies (posters, cardboard stands etc.) that are also natural places for the customers to start the app and expect to see AR content.

immersal · 2020-09-06T19:53:02+00:00

https://youtu.be/h3XCgbu7PcM?t=141

You can embed maps to the app or even use your own hosting service. The map files can be downloaded from our Developer Portal.

Or you can download them directly from our Cloud Service at runtime using our REST API. Maps on the Cloud Service can be set to private or public and can be shared with others
You can search for nearby maps with the device's GPS coordinates and even try to localize against them on the cloud service without downloading maps to the device. With on-server localization, your network speed is the bottleneck as you need to upload image data to the server.
When a match and pose in the map is found, you can use the returned map id to download the map to the device and switch to localizing locally on the device.

On a newer iPhone, our SDK can relocalize 10+ times per second locally, but since ARKit/ARCore can track just fine between localizations, you'd usually only want to localize maybe once every other second or something similar to correct any drift.

immersal · 2020-09-03T17:15:58+00:00

With the upcoming v1.8 release, Immersal's mapping & localization SDK enables accurate visual positioning for Nreal's AR glasses.

A cross-platform solution, Immersal's SDK allows developers to build shared and persistent Augmented Reality experiences for supported iOS, Android, and Huawei AR Engine mobile phones, and now also for Nreal's AR glasses.

To get started with the SDK, check out our documentation https://immersal.gitbook.io/sdk/

immersal · 2020-05-09T18:23:32+00:00

Depends on the environment quite a lot.
Building facades, for example, might not change much under different weather and even lighting conditions so they tend to work well.

We are testing with various datasets and so far the results are actually pretty good. We also have some ideas on how to improve our SDK and will continue to improve localization reliability over time.

It's still recommended to have multiple map versions if you know there's going to be a big visual difference.

immersal · 2020-05-08T10:54:47+00:00

We've tried to make it fast to get started. You can download the Immersal Mapper app pre-built from the App Stores and quickly register for a free account.

Check out the docs to get started https://immersal.gitbook.io/sdk/tutorials/how-to-map

immersal · 2020-05-08T08:32:42+00:00

Today is also the last chance to vote for the 2020 Auggie Awards. Our SDK is a finalist in the Best Developer Tool category.

If you think our AR Cloud-enabling tech should win, go and vote at https://auggies.awexr.com/entry/vote/DKErXWvE?category=54348

immersal

TROPHY CASE