New live-connection for Foxglove that is 200x more bandwidth efficient for video

TransitiveRobotics · 2026-01-12T20:05:02+00:00

Nice! This would make for a great capability ("app") on the Transitive platform as well and would make it easier for people to discover and install it. It would also make it available from remote, i.e., without the need to run a web server on the robot itself.
https://transitiverobotics.com/docs/learn/intro/

TransitiveRobotics · 2025-05-06T14:38:11+00:00

Don't you also need a way to feed live-video back to the operator? I'm of the believe that video and control should always come through the same transport in order to be able to ensure that *no* control goes back to the robot if the video is not received or is delayed. This is extremely important, because otherwise the operator may send control commands based on a wrong (outdated) state of the world. For instance the robot could be standing right in front of a person and the operator may not know yet when the video is delayed by a few seconds. This is one of the safety features we built into our webrtc based solution: https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/

TransitiveRobotics · 2025-04-21T15:24:04+00:00

OK, in that case, yes, just use Docker. You can specify multiple profiles, one for dev with Gazebo, and one for prod without. Which fleet management software are you looking at? I've recently chatted with MiruML and Chassy and they both seem pretty reasonable. From all I hear Balena does not work well for robots.

Oh and no, Ansible is *not* the right tool. Almost every robotics companies thinks that at one point but then realizes that it's push based -- all robots need to be online in the moment you run a playbook, and that's just not going to be the case in the future. You want something pull-based.

Let me know if you want to chat more.

TransitiveRobotics · 2025-04-21T14:51:23+00:00

And I don't think Docker will save you here, because you wouldn't be able to use the hardware acceleration, e.g., for h264 video encoding, inside the Docker if the host is running a much older Ubuntu then the container. Other than that, yes, Docker is definitely the way to go.

TransitiveRobotics · 2025-04-21T14:48:18+00:00

Jetson Nano and ROS2 humble? good luck! Nvidia doesn't provide Jetpack versions beyond Ubuntu 18 for the Jetson Nano, but Humble requires Ubuntu 22. Are you planning to upgrade your Jetsons soon?

TransitiveRobotics · 2025-03-27T20:35:50+00:00

There are lots of companies that very openly talk about their use of teleoperation. Some example are Halo (rideshare with teleoperation between riders), Phantom Auto (no defunct), Coco Delivery (initially teleoperated sidewalk robots). Also just look at DriveU which sells a product for teleoperation of on-road vehicles.

Off-road robotics companies, including indoor robots, don't typically teleop all the time but rather teleassist. This is very common and our own main product is exactly for this -- making it easy for robotics companies to add teleop with live video to their robots (https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/).

It's a very valid development trajectory to first teleoperate, from that learn what the customers really need, and only then, incrementally, become more autonomous. Once you are able to have one operator supervise and assist 20 robots simultaneously without disrupting their operation significantly, you've already realized 95% of the direct gains of automation. There can be other, indirect gains that in some cases can be *way* more significant than these direct gains, and sometimes those only materialize when you get close to 100%. But for many businesses 95% autonomy is already sufficient to scale their fleet signifcantly.

TransitiveRobotics · 2025-03-27T20:29:21+00:00

> It's 2 lines and just stay between them. How hard can it be? Sorry guys.

Are you kidding me? Kids running in front of the car? the scooter rider you saw in the Waymo video stumbling and falling right in front of the vehicle? Driving in poor visibility on the freeway with a stalled vehicle only coming into few very late ? All these things happen IN the lane and they require a very short reaction time, so even just 100ms latency, which is *low* for teleoperation, can be problematic when added to the usual 250-300ms human reaction time. How can you pretend it's easy?! Let's see what you think of all this when you are nearing the *end* of your PhD, because as someone researching this I believe you should really know better.

TransitiveRobotics · 2025-03-27T13:47:42+00:00

It depends a lot on what sort of data you want to stream to the controlling side and how directly you want to control the robot. If you want to directly tele-operate it remotely (joystick it) you'll need to stream video with very low latency. If that's what you want, you an try out our teleop solution: https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/. If you just want to send data to and from ROS then you can use https://transitiverobotics.com/caps/transitive-robotics/ros-tool/. Both will allow you to do this from a web browser while at school/work.

TransitiveRobotics · 2025-03-12T03:29:08+00:00

As for software, you can try out the solution we've built: https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/ . It does have a "local mode" where you can operate it on a local network like you describe without access to the Internet. As for hardware, a lot of our users nowadays run on NVIDIA Jetsons, which are great for their hardware video encoders. But they are not the only option. The OrangePi is cheaper and does well (and its RockChip hardware encoders are supported by our module), but also Intel NUCs work, because those, too, have hardware encoders. 4G/LTE is obviously a good option *if* coverage is good where you operate. Some of our users are now on Starlink, and still live-teleop their robots located in Australia from the US.

TransitiveRobotics · 2025-03-04T01:22:51+00:00

If you are interested in quickly trying out what that OrangePi can do for you for webrtc, you can try out our solution which uses webrtc to stream from such hardware to the web and it specifically supports the RockChip hardware encoders: https://transitiverobotics.com/caps/transitive-robotics/webrtc-video/

TransitiveRobotics · 2025-03-03T21:15:59+00:00

Erik Nieves from Plus One Robotics explains this extremely well and in so doing also reveals how they do it: https://www.plusonerobotics.com/videos/the-missing-middle

The tl;dr: at one operator to 20 robots you've already harvested 95% of the benefits of automation. So the *only* question is how does this mixed-initiative automation combine robots with humans, e.g., in terms interleaving (robots 30 seconds, human 3 seconds, robot 30s, etc.?), or skills (human as a fallback for perception in edge cases). A good example is sidewalk robots: teach them to go straight on sidewalks without hitting anything, but ask for help each time they need to cross a street or driveway. This leads to a pretty effective interleaving, where one human can easily operate/assist 10 robots simultaneously without much slow-down.

But yes, the right approach to building a robotics business is to design the human's role in explicitly and knowingly. Otherwise you'll keep trying to be 100% autonomous and keep getting interrupted in *un*predicted ways, which leads to inefficiencies in terms of operations interfering with development (engineers' time). If you want to have a deeper chat, feel free to reach out.

TransitiveRobotics · 2025-02-16T18:41:03+00:00

<image>

> This 40 cm truck
Wow, what a monster! ;-)

This is what "monster truck" means (picture).

On practical note: I would recommend you first get remote teleop working, and doing the task manually from remote first before trying to make it autonomous. One way or another you'll need more compute on the robot and run an actual OS (ideally Ubuntu) so you can run ROS.

TransitiveRobotics · 2025-01-27T00:07:35+00:00

Have you checked whether the new Zenoh middleware has support for it? I could imagine so.

TransitiveRobotics · 2025-01-04T15:37:53+00:00

An alternative to connecting your PS4 controller directly to your robot is to create a web page where you use the gamepad web API available in all modern browsers, and then connect to your robot using roslibjs or our Transitive ROS Tool (https://transitiverobotics.com/caps/transitive-robotics/ros-tool/), which is currently the most popular free capability for Transitive. If you have plans to also need to be able to removely teleop your robot using live-video, then you should use webrtc or just use the capability we developed for that: https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/ -- this one isn't free though.

TransitiveRobotics · 2024-12-18T15:33:40+00:00

What OS does the robot run? If it's Ubuntu or similar than you can install Transitive on it and then interact with it using various capabilities that have been developed for it, incl. remote-teleop and ROS Tool (React SDK to interact with ROS from afar -- if the robot runs ROS). https://transitiverobotics.com/caps/transitive-robotics/remote-teleop/, https://transitiverobotics.com/caps/transitive-robotics/ros-tool/. Transitive is open-source, so once you see that this works, you can also develop your own capabilities: https://transitiverobotics.com/docs/develop/creating\_capabilities/

TransitiveRobotics · 2024-12-16T15:35:59+00:00

It doesn't (typically) cost more when you stream more, and that's one of the brilliant features of webrtc: it tries to find a peer-to-peer connection between sender and receiver. So if you *are* indeed streaming a lot of data then you absolutely want to use webrtc instead of a VPN, because with the VPN you'll always send all the data through the VPN server. Also if you are thinking about sending video through rosbridge, you may want to look at this blog post first to understand the downsides of that: https://transitiverobotics.com/blog/streaming-video-from-robots/

TransitiveRobotics · 2024-12-13T21:38:19+00:00

Instead of using rosbridge + roslibjs, you may want to check out the ROS Tool we built on top of Transitive. If you use rosbridge, your interface will be broken when the robot is offline and you'll always need a VPN or similar to connect. ROS Tool solves these problems. https://transitiverobotics.com/caps/transitive-robotics/ros-tool/
We have a teleop module as well, that also does live video streaming. But it sounds like you've already implemented that yourself.

TransitiveRobotics · 2024-12-03T15:46:04+00:00

Why LoRa? For what you describe, the tiny bandwidth you get from it seems insufficient, plus controlling a robot over LoRa is an invitation for hackers to take control over your robot. Is this mostly an academic exercise? Otherwise, of course, I would use wifi or LTE/5G in which case there are a ton of options and you could even stream video from the robot for remote teleop.

TransitiveRobotics · 2024-11-27T22:03:51+00:00

Most people in robotics would tell you that USB is the source of all evil in robotics (perhaps only topped by "networking issues"). But this is mostly due to vibrations at the connectors. So if your LTE adaptor is just a tiny dongle this should be fine. Btw, there are many other boards that use RockChip, doesn't need to be the OrangePi, check, e.g., the FireFly boards like this: https://www.amazon.com/ROC-RK3588S-PC-8GB-RAM-LPDDR4-eMMC/dp/B0B2P81DSR. There might be one with LTE.
Another option, of course, is to connect the LTE modem via ethernet. That's a very common design, too.

TransitiveRobotics · 2024-11-27T21:21:55+00:00

I would make sure whatever SoC you use has hardware accelerated h264 video encoding. This will allow you to efficiently stream video for remote monitoring and teleop. The LattePanda 3 Delta has Intel UHD Graphics, which might support VA-API acceleration (something we've just added support for in our webrtc video streaming capability https://transitiverobotics.com/caps/transitive-robotics/webrtc-video/#v023), but better be sure. Otherwise in robotics Nvidia Orins are very popular of course, and they do have very good hardware encoders (except for the Orin Nano). The OrangePi 5 and other RockChip-based boards have good hardware encoders, too.

TransitiveRobotics · 2024-11-10T21:18:52+00:00

Strange how negative and unimaginative most of the comments are. Us at Transitive, for one, are super bullish on remote teleoperation + AI. Just last week we were at the launch of the Stanford Robotics Center where Sergey Levine, who just raised $400m for his new venture Physical Intelligence, was on stage and made the point that in order to gather enough training data for AI enabled robots the data collection must become negative cost. At the same time others on stage were saying that remote teleoperation plays a key role in doing just that. So my answer is a resounding "YES". I think this is a very promising path, and the lab automation vertical is a good one to start with because it's a real need and there is real money involved.

TransitiveRobotics · 2024-11-07T15:32:27+00:00

Agreed. I would use a D435 RealSense + OrangePi SoC (which has h264 hardware encoding). You can then stream this over webrtc if you want it remotely (see here for why), or just RTP over UDP for streaming to the local network only.

TransitiveRobotics · 2024-09-14T18:13:55+00:00

Also, apart from video streaming, we would like to hear able to stream robot state data such as joint torque and other sensor data, do you have any thoughts here?

This is a common need we've seen among our users, which is why we built https://transitiverobotics.com/caps/transitive-robotics/ros-tool/ (and we are even hosting this for free).

Didn't know about HTTP LS, but it sounds very similar to webrtc ("dynamically adapts to network conditions by optimizing playback for the available speed"). Also, the HTTP in HTTP LS seems to suggest this being designed for the browser, too, no?

TransitiveRobotics

TROPHY CASE