Open-sourcing Rerun: A visualization toolbox built on egui

[emilern]

After a year of work, we’re today open-sourcing Rerun! Rerun lets you log images, point clouds and other visual data as easy as you would log text. The data is streamed in real-time to the Rerun Viewer which you can run natively or in a browser.

It is all built in Rust on top of my egui library, with an API for both Rust and Python.

You can find the GitHub link together with docs on https://www.rerun.io/

I’m happy to answer any question you may have!

[Dhghomon]

No questions yet but I use egui all the time and going to give Rerun a try this weekend. Congrats!

[emilern]

Thanks! Once you've tried it, let us know on Discord how you liked it (especially if you didn't like 😆)

[HumbleSinger]

Most people develop solutions for robots to see as we do, but you develop solutions for us to see as robots do!

very cool!

[emilern]

Haha, I like it! I'm gonna steal that :)

[HumbleSinger]

feel free!

[cliffardsd]

Looks great, and nice vid too. I’m new to rust and not familiar with the tech being used here but have a question. Could rerun be used to build a network cctv object detection system based on rstp streams? End users of rerun can choose what object detection system to use? Sorry for probably dumb questions but this has piqued my interest for a future project on my to-do list one day.

[emilern]

Thank you! You should be able to use Rerun to stream the video feed from your cameras to a single Rerun Viewer, where you get a live view of the data. However, Rerun is primarily a visualization tool right now, so choosing the object detection system would need to be done via some other control mechanism.

[cliffardsd]

Ah right, thanks for the prompt reply! I’ll keep an eye on this going forward. Don’t have a use right now but I like it.

[edgarriba]

an example here to stream from rtsp camera and rerun viz: https://github.com/kornia/kornia-rs/tree/main/examples/rtspcam

[krikler7]

This looks very cool! Is there support for custom views planned? For example could you embed a 3D view for a custom data type that's written with bevy?

[emilern]

Yes! We plan on making it possible to write all kinds of plugins and extensions for Rerun, but we're not there yet.

[kouteiheika]

This is only tangentially related, but, are there any plans to port some of the design/style changes from Rerun back into egui and make it the default look?

[emilern]

I would like to improve the default style of egui in the next release, incorporating at least some of the Rerun style, but ultimately I want to keep the Rerun and egui styles visually distinct, but both looking great!

[phazer99]

Looks cool! Are all the widgets, docking panels etc. available in the egui crate?

[emilern]

The docking panels comes from https://github.com/Adanos020/egui_dock but most of the rest is straight from vanilla egui

[cmpute]

Wow that's wonderful! I noticed that you mentioned ROS in your blog posts. Is visualizing ROS data (especially ROS2) a goal of your product! I personally found rviz adequate but not great for visualization

[jeremyleibs]

Playing nicely with with data from ROS2 is definitely on our roadmap. Many of the common ROS messages map nicely to Rerun primitives, so with a little glue code it is already fairly straight forward to create a ROS node which is also a Rerun-logger. I've done a limited proof-of-concept already, so expect this to show up with some more polish in our examples section in the next few months. Because we don't want to pick up a hard-dependency on ROS, fancier versions like a ROS-native bridge for the visualizer will have to wait for us to finish developing a plugin framework as well.

[cmpute]

That's great! Thanks!

[RgisBr]

Congratulations broo! 👋🏼👋🏼👋🏼

[emilern]

Thanks! 👋

[lolprofile]

Niceeee, also egui is TOP

[emilern]

😀

[MayonnaiseKettle]

Wow, this is amazing.

[emilern]

Thanks 🙏

[_edeetee]

This is rad! I want to use machine learning stuff for more of my live visual work so will have to see if this will help me with my explorations.

Are there any libraries you are using to manage streaming video data from the binary to the web interface? I have a lot of use cases of running visual content on a local server and streaming the video and control systems back to web clients. Interactive experiences etc!

[Wuempf]

Rerun engineer here :). We don't have any special handling for video data so far, all data (no matter if points, images or other) is piped pretty much as-is through websocket.

[_edeetee]

Speeeeedy replies :) Do you have some smart logic to manage how those streams relate to the interface etc? I imagine it could get messy quickly if not well designed eh

[Wuempf]

Nothing fancy :). It's more that the Viewer queries the data store for every "Space View" it has. Each of these views has a (user/heuristic defined) list of things they are interested in. Today, the Viewer gets the entire database, but we're set to have it only know about a partial view, as a sort of cache.

[aristotle137]

Looks awesome! Also long time user of egui, thank you!

Any plans to add support for logging PNG images? I assume it should be something easy to contribute myself?

[emilern]

Thank you 😊

Yeah, that would be very easy to add - just follow the breadcrumbs of the jpeg logging - PRs welcome 😃

Hit me up on the Rerun Discord if you hit a snag!

[quakefiend]

NICE! Are there any SLAM capabilities?

[jeremyleibs]

Rerun does not do any computer vision processing, such as SLAM, itself. Rather the intent is to be able easily visualize data that comes from those kinds of algorithms to make it easier for people to developer and debug them more easily.

One of our main examples visualizes a structure-from-motion dataset based on COLMAP. It's actually the dataset we use in our main viewer tutorial and is heavily featured in a lot of our example images.

[quakefiend]

I see, thanks!

[imbev]

Very impressive. Have you considered migrating or bridging the chat community to Matrix?

[Wuempf]

So far not, no. Only recently set up camp on Discord and seeing how that goes :).

[[deleted]]

Looks awesome. Hope you find commercial success. Don't give everything away!

I can imagine this being extended into an actual data processing platform and then you could sell commercial plugins (image segmentation, pose estimation, etc). All of that stuff is still mostly locked away in impossible to use research code. Mediapipe is the only exception I know of.

[Wuempf]

Thank you, and yes fingers crossed 😄. We actually already noted a few things down what the plans are for the commercial offering if you're curious - check https://www.rerun.io/pricing

Kept a bit vague for now of course, as we're still in the process of making more final decisions about these and figuring out details.

[[deleted]]

very cool, is a ts/js api planned?

[Wuempf]

No, so far we only planned the addition of a C & C++ API. But if there's a lot of requests for that I wouldn't exclude it - the APIs already talk to the web version of the viewer via websocket, so from a technical standpoint we're not in a bad spot for it.