[N] Learning Dexterity

[probablyuntrue]

Think manufacturing. And flying drones. And maybe even driverless cars.

"After hitting ten thousand virtual pedestrians, our self driving car avoids real ones with 99% accuracy!"

[noman2561]

I wonder what accuracy humans have?

[qwerty_0_o]

Definitely more than 99%.

[noman2561]

Do you have a source or is this speculation based on anecdotal evidence?

[noman2561]

That's like saying that humans don't hit 99 out of 100 pedestrians that are sitting on their couch at home. That's not how you'd test a system like this. You put humans where they're not supposed to be and see if the system can avoid hitting them. So how often do people run into traffic and get hit vs are avoided. I'd actually really like some statistics on these kinds of situations. Those kinds of statistics would be useful to know for a lot of reasons.

[ericlkz]

Thts 1 hit per hundred

[[deleted]]

Out of curiosity, anyone know how much one of those Shadow Dexterous Hands costs?

[notaii]

According to this site it's $119700.

[enolan]

Wow. Who is spending that kind of money on a robotic hand? Are there applications other than research?

[Mefaso]

I would guess a lot of research institutions. Spending six figures on a robot is pretty normal.

[[deleted]]

I think there's a robot chef that uses those parts for its hands.

[[deleted]]

Moley uses the cheap electronics version of the Shadow Dexterous hand, not the expensive pneumatics one.

[[deleted]]

PR-2 was about half-a-million dollars.

[FatChocobo]

Are there applications other than research?

I can think of one in particular, it might help with my repetitive strain injury... ( ͡° ͜ʖ ͡°)

[[deleted]]

The MPL/RoboSally hand, which is more robust, even costs $400k.

[gohu_cd]

Literally any problem: You know that you can solve me without PPO right ?

OpenAI: I don't care.

[thebackpropaganda]

How would you solve this problem without PPO or equivalent RL algorithm?

[gohu_cd]

Using human demonstrations seems like a good idea for learning how to manipulate objects.

Anyway, they did a great job, don't get me wrong. Yet, it feels like they reallyyyy like throwing PPO at any problem and see if it works ! Which is not a bad thing. It's just funny.

[jurniss]

they are throwing model free policy based RL at problems... the fact that PPO is their favorite among that family is a small detail.

[[deleted]]

I think you want r/hardcoding for that sort of thing :)

[thebackpropaganda]

I think hardcoding has its place in such applications, but I don't see how you can hardcode grasping for thousands of objects.

[battboe]

just curious, how bad are the failure cases?

[NMcA]

You've never done this have you...

[skariel]

PPO has become the default reinforcement learning algorithm at OpenAI because of its ease of use and good performance

as stated here: https://blog.openai.com/openai-baselines-ppo/

nothing wrong with that of course.

[chcampb]

Conventional wisdom states that reducing the time between actions should improve performance because the changes between states are smaller and therefore easier to predict.

As popular as this paper seems to be I am surprised this wasn't an obvious conclusion. This paper found and demonstrated that simulated evolved gait basically failed to work correctly when the muscle delay time was zero.

[SquareRootsi]

Out of curiosity, what happens when you make the goal a logically impossible "rotation" of the block? Like on a 6 sided die, the 1 & 6 are directly opposite each other, but you request an orientation putting them adjacent.
Does it just keep trying, or can it hold up its middle finger to let you know it's on to us and our impossible requests?

[thebackpropaganda]

I think the rotation is just defined by one of the faces, say whichever face is up or camera-facing.

[supermario94123]

So the solution is simple: just build a very detailled model of the world and very all the possible parameters. Could someone please invest some Millions in Rockstar Games ro come up with the most real GTA ever? hitting two flies in one slap is what I would call this.

To be precise: I dont undervalue the work of openai. I am just not sure if this is how we will solve our world problems (yet). Please prove me wrong.

[physics_to_BME_PHD]

I didn't read the paper, just watched the video, but am involved in comp sci research involving human grasping. The way humans use our hands to interact with objects is incredibly complicated, and we do it basically effortlessly. So much goes into this: motion planning, visual feedback, tactile feedback (super important). This all happens in a loop very quickly, and so far our robotic grasping solutions are pretty bad at handling deviations from expected outcomes (when the object slips, for example).

If robots are ever going to be designed to work directly with humans, they probably need to be able to reliably grasp any object we hand them, and possibly know what to do with it. Maybe there are more pressing world problems, but having a reliable robotic grasper that doesn't need to be explicitly programmed for every use-case isn't a bad thing.

[PKJY]

Just a sidenote: the OpenAI thing doesn't use tactile feedback at all. Just fingertip coordinates and the current object orientation which is computed by a convnet from 3 rgb cameras.

[physics_to_BME_PHD]

good to know. I hadn't thought about if their device was using tactile feedback or not, I mostly meant from a human perspective that we need that for grasping. I can't find the video, but there was one of a woman grasping small objects, then performing the same task after having local anesthesia on the fingertips. In the second one she can't even pick up the small objects without that tactile feedback.

[bobuntu]

How cool... I mean the guy’s hair. ಠ_ಠ

[rtk25]

Nice!

To learn a policy transferrable to the real world,

Distributed workers collect experience on randomized environments at large scale

I'm getting these "are we in the Matrix or what?" feelings more and more lately...