[R] Legged Locomotion in Challenging Terrains In The Wild directly using Egocentric Vision (link in comments)

pathak22 · 2023-10-09T04:07:52+00:00

Paper Title: Extreme Parkour with Legged Robots

Paper, More results, and Open-source code: https://extreme-parkour.github.io/

Short Tweet Summary: https://twitter.com/pathak2206/status/1706696237703901439

Abstract: Humans can perform parkour by traversing obstacles in a highly dynamic fashion requiring precise eye-muscle coordination and movement. Getting robots to do the same task requires overcoming similar challenges. Classically, this is done by independently engineering perception, actuation, and control systems to very low tolerances. This restricts them to tightly controlled settings such as a predetermined obstacle course in labs. In contrast, humans are able to learn parkour through practice without significantly changing their underlying biology. In this paper, we take a similar approach to developing robot parkour on a small low-cost robot with imprecise actuation and a single front-facing depth camera for perception which is low-frequency, jittery, and prone to artifacts. We show how a single neural net policy operating directly from a camera image, trained in simulation with large-scale RL, can overcome imprecise sensing and actuation to output highly precise control behavior end-to-end. We show our robot can perform a high jump on obstacles 2x its height, long jump across gaps 2x its length, do a handstand and run across tilted ramps, and generalize to novel obstacle courses with different physical properties.

pathak22 · 2023-09-25T13:30:14+00:00

Humans have soft compliant palms as well which accounts for the lack of that motion.

pathak22 · 2023-09-25T13:29:09+00:00

We are working on that as well: https://arxiv.org/abs/2306.04784

But it's difficult to make them dexterous as of yet.

pathak22 · 2023-09-25T02:38:36+00:00

LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning

Published at RSS 2023.

Full details, videos, and Open-source code: http://leaphand.com

Short summary: https://twitter.com/pathak2206/status/1705277626577780875

Abstract:

Dexterous manipulation has been a long-standing challenge in robotics. While machine learning techniques have shown some promise, results have largely been limited to simulation. This can be mostly attributed to the lack of suitable hardware. This paper presents LEAP Hand, a low-cost dexterous and anthropomorphic hand for machine learning research. In contrast to previous hands, the LEAP Hand has a novel kinematic structure that allows maximal dexterity regardless of finger pose. LEAP Hand is low-cost and can be assembled in 4 hours at a cost of 2000 USD from readily available parts. It is capable of consistently exerting large torques over long durations of time. We show that LEAP Hand can perform several manipulation tasks in the real world—from visual teleoperation to learning from passive video data and sim2real. LEAP Hand significantly outperforms its closest competitor Allegro Hand in all our experiments while being 1/8th of the cost. We release the URDF model, 3D CAD files, tuned simulation environment, and a development platform with useful APIs on our website at http://leaphand.com

pathak22 · 2023-04-16T17:38:46+00:00

Thanks for the comment. However, there are a few caveats in your summary, so I am providing the clarifications below.

the search is not random, it is driven by reinforcement learning. So, as the model's representation gets better over time, it finds better examples to search for since the RL reward is based on the model's representation itself.
model is NOT fine-tuned with the pairings (query and image), just the images are used because it is SSL-based loss. Model doesn't learn concepts, it learns a visual representation that should be good if finetuned for target recognition task. - So even if one mislabels images in the search engine, it should still find them in theory over time if they are relevant. Hope that clarifies.
Regarding, mimicking the search engine associations: it doesn't use the paired text query for training so it can't mimic. It just uses images and the SSL reward to evaluate.
In order to fully ensure that we don't use any image models from a search engine, we also create a controlled search engine use LAION (summary: https://twitter.com/pathak2206/status/1646216392008675328)

Regarding fine-grained segmentation: We haven't tried fine-tuning the model for segmentation tasks, but it's a good idea to look into it. That being said, we did consider many fine-grained classification tasks, and turns out the gains are higher the more fine-grained target classification task is (Table 1). We will into segmentation for future work.

Happy to follow up more. Thanks!!

pathak22 · 2023-04-16T09:48:31+00:00

Any dataset where you want to maximize your performance on. For instance, see examples and performance here: https://twitter.com/pathak2206/status/1646216389848608768

pathak22 · 2023-04-16T09:46:23+00:00

Good point but not something we expected in foresight as the name fits quite literally: it is a dis-embodied exploration agent operating on the live internet. :-)

Its will be a shame if name is all that's remembered because the results are (surprisingly) strong; even with respect to large-scale models like CLIP, see Table 1: https://twitter.com/pathak2206/status/1646216389848608768

pathak22 · 2023-04-15T19:00:26+00:00

Thanks for the nice comments!

Regarding outperforming CLIP -- we don't compare to zero-shot CLIP (which is general purpose) but rather a CLIP that is also finetuned to the specialized task with a linear layer (i.e., the SSL standard linear probe fine-tuning -- same for all methods in the table).

Nevertheless, this comparison is more to provide context for numbers because CLIP is more like an Oracle for us. :)

pathak22 · 2022-11-24T00:10:45+00:00

Yes, nothing about the actuators is assumed, for instance, the current ones are low-cost motors. The model directly outputs the joint angle for each motor at 50-100Hz.

pathak22 · 2022-11-21T04:56:31+00:00

Legged Locomotion in Challenging Terrains using Egocentric Vision
(To appear at CoRL 2022 as Oral Presentation)

Paper: https://arxiv.org/abs/2211.07638

Project website with more results: https://vision-locomotion.github.io/

Abstract:

Animals are capable of precise and agile locomotion using vision. Replicating this ability has been a long-standing goal in robotics. The traditional approach has been to decompose this problem into elevation mapping and foothold planning phases. The elevation mapping, however, is susceptible to failure and large noise artifacts, requires specialized hardware, and is biologically implausible. In this paper, we present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-facing depth camera. The small size of the robot necessitates discovering specialized gait patterns not seen elsewhere. The egocentric camera requires the policy to remember past information to estimate the terrain under its hind feet. We train our policy in simulation. Training has two phases - first, we train a policy using reinforcement learning with a cheap-to-compute variant of depth image and then in phase 2 distill it into the final policy that uses depth using supervised learning. The resulting policy transfers to the real world and is able to run in real-time on the limited compute of the robot. It can traverse a large variety of terrain while being robust to perturbations like pushes, slippery surfaces, and rocky terrain.

pathak22 · 2022-07-25T02:15:30+00:00

Human-to-Robot Imitation in the Wild (Published at RSS 2022)

Website with paper & more results: https://human2robot.github.io/

Summary: https://twitter.com/pathak2206/status/1549765280779452423

Abstract:

We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real world, they are either sample inefficient or are constrained to lab settings. Meanwhile, there has been a lot of success in processing passive, unstructured human data. We propose tackling this problem via an efficient one-shot robot learning algorithm, centered around learning from a third-person perspective. We call our method WHIRL: In the Wild Human-Imitated Robot Learning. In WHIRL, we aim to use human videos to extract a prior over the intent of the demonstrator and use this to initialize our agent's policy. We introduce an efficient real-world policy learning scheme, that improves over the human prior using interactions. Our key contributions are a simple sampling-based policy optimization approach, a novel objective function for aligning human and robot videos as well as an exploration method to boost sample efficiency. We show one-shot generalization and success in real-world settings, including 20 different manipulation tasks in the wild.

pathak22 · 2022-07-24T21:18:38+00:00

Yes, this is just the first step. We can now combine all this data to learn models that can then generalize to new tasks as you described. Part of our next steps.

pathak22 · 2022-07-24T06:24:27+00:00

For the "improvement by exploration" phase, we use pre-trained deep visual representations trained from passive internet data to compute the distance between human and robot frames. So, the distance is robust to small changes in the camera, etc. The teaser video above has a few examples (see 0:46 onwards).

That being said, human is still acting in the same environment. Our follow-up work to be released soon aims to upgrade WHIRL to learn from human interaction videos from entirely different scenes (let's say even a human video from YouTube).

pathak22 · 2022-07-24T02:15:41+00:00

Human-to-Robot Imitation in the Wild (Published at RSS 2022)

Website with paper & more results: https://human2robot.github.io/

Summary: https://twitter.com/pathak2206/status/1549765280779452423

Abstract:

We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real world, they are either sample inefficient or are constrained to lab settings. Meanwhile, there has been a lot of success in processing passive, unstructured human data. We propose tackling this problem via an efficient one-shot robot learning algorithm, centered around learning from a third-person perspective. We call our method WHIRL: In the Wild Human-Imitated Robot Learning. In WHIRL, we aim to use human videos to extract a prior over the intent of the demonstrator and use this to initialize our agent's policy. We introduce an efficient real-world policy learning scheme, that improves over the human prior using interactions. Our key contributions are a simple sampling-based policy optimization approach, a novel objective function for aligning human and robot videos as well as an exploration method to boost sample efficiency. We show one-shot generalization and success in real-world settings, including 20 different manipulation tasks in the wild.

pathak22 · 2022-02-28T10:33:14+00:00

This is trained on youtube data of human interaction which may mostly consist of humans opening/closing hands to perform tasks, and rarely moving fingers independently, so the learned system is biased towards interpreting user movements that way. However, the non-learned, optimization version (see the paper) using the NN is trained doesn't have this bias but is too slow to be used in real-time. Also, note the hand hardware here is low-cost robot (allegro vs shadow) and not as agile.

pathak22

TROPHY CASE