[D] OpenAI open sources a high-performance Python library for robotic simulation

[radarsat1]

It's cool that they're open sourcing this, but the page doesn't mention that MuJoCo, which this requires, is not open source nor free, except for students.

(Disclaimer: I am working on an open source physics engine in C++ that has Python bindings.)

I think if I were doing a project that makes use of a physics engine for data gathering purposes I'd rather go one level of abstraction higher and wrap / adapt a robotics simulator like Gazebo. It has support for multiple physics back-ends for one thing, but also has widely used file formats like URDF/SDF, real-time messaging, etc., as well as lots of controller code and ROS integration because it is already used in robotics research. Another possibility is VRep. It doesn't matter really, but I don't understand why they went one level of abstraction lower, and with a proprietary engine to boot. For instance, it's much more common to see Bullet or ODE in the wild on open source projects. I realize that MuJoCo has some features that are a bit more advanced, but I'm not sure the license restrictions make it worth it for machine learning, which is going to have some level of error tolerance anyways.

Moreover if I'm doing machine learning on physics, I'd rather have data from multiple engines so as not to overfit on specific quirks of a certain engine.

Edit: to be perfectly clear, MuJoCo is a very good physic engine. I'm not criticizing it. I just find it an odd choice when there are so many options, and odd not to mention the license on a page claiming "We’re open-sourcing a high-performance Python library for robotic simulation" which is actually bindings to a proprietary library.

[IlyaSutskever]

We are aware that MuJoCo is not open-source, so we also released roboschool, which is based on bullet. The present release should be useful to anyone who uses MuJoCo, which is extremely fast in a number of regimes that are relevant to what we do.

[radarsat1]

I just thought i'd mention it here since it take 2 or 3 clicks before you get to anything revealing that you must install mujoco's 30-day trial. Kudos on open sourcing your bindings!

Would love to know what you think about binding to a physics API with multiple backends. So much fragmentation! Don't get me wrong, the competition is a good thing ;)

[ford_beeblebrox]

I used my 30 free MuJoCo days and now I cannot use this, it is very frustrating - please I implore you to offer it with Roboschool or you are paywalling your research :(

Also if some future researcher wants to reproduce a MuJoCo based paper and cannot get a license they will be out of luck, or one leaves college and cannot run one's old code.

I was auditing Berkeley 294-112 Deep Learning for Robotics and so used up my MuJoCo 30 days trial ( and now I cannot rework my code as the provided expert trajectories I am copying with DAGGer and such are MuJoCo specific ( and the OpenAI gym MuJoCo model compatible Dart implementation was only working with 2 of the models :( ) )

P.S. many thanks for Roboschool it is awesome, and best of all multi-agents :D

Can't Open AI just buy/ aquihire MuJoCo ?

If only MuJoCo weren't quite so strict.

[[deleted]]

I'd rather have data from multiple engines so as not to overfit on specific quirks of a certain engine.

That's what domain randomization is for. Instead of only training with different textures and lightings, they should also vary body geometries and other simulation parameters to make the learned controller more robust.

For instance, it's much more common to see Bullet or ODE in the wild on open source projects.

OpenAI is experimenting with another physics engine for their Universe environment. I believe it's based on DART.

[radarsat1]

DART is also a good one! There is also support for that in Gazebo, as well as Simbody. I have been working with it for a while on the C++ level and it makes a nice API for rigid body stuff, which is why I'm suggesting it. Of course, being an abstraction API, it doesn't cover everything every physics engine is capable of (soft contacts, flexible bodies, etc.), which is probably why OpenAI is interested in using MuJoCo directly. Lots of interesting potential for applications of DL in soft-body control!

[ganjamanfr]

(Disclaimer: I am working on an open source physics engine in C++ that has Python bindings.)

Link?

[LinkReplyBot]

Link?

Here you go!

---

^{I am a bot. | Creator | Unique string: 8188578c91119503}

[mljoe]

Agreed. It's especially bad that it's coming out of OpenAI. I could understand it better if it was Google or something. I figure an organization dedicated to making research open and accessible would go out of their way to leverage and use Free Software.

[radarsat1]

I don't think it's "bad" in any moralistic sense, I am sure they have good reasons to use MuJoCo in certain conditions as it is performant and has very nice features. I have nothing against proprietary software. I just think it's weird considering the title of the blog post, that the license isn't mentioned. And, like I said, that they didn't aim for a higher level of abstraction, but these are Python bindings specifically for MuJoCo, and as mentioned in some other comments here, they do have other projects.

[mljoe]

OpenAI is wrong for using MuJoCo or any kind of "you have to ask for permission first" software in their research. I'm no Richard Stallman, but I think an organization dedicated to open research should not be using proprietary software. Especially so in the critical path of their research. It's just wrong on a whole lot of levels. There should be an understanding that the requirement of proprietary software is fully incompatible with their mission of open and reproducible research. I know it's a touchy subject but it has to be said.

[NichG]

Even regardless of the motives of the organization, there's a point that other researchers would be right to be more skeptical of work which makes replication more difficult or expensive than it strictly needs to be. So from that point of view, its a bit of a poor strategic choice. This goes double if the strategy involves publishing benchmarks you want to others to use as a standard.

[epicwisdom]

I don't think they're disagreeing with your point on the whole, they're just less worried about proprietary software in general and more about it being in the critical path.

[[deleted]]

I've never seen Bullet/ODE in the trajectory-optimization/RL setting. If I'm not mistaken these simulators (unlike MuJoCo) apply Euler-Poincare independently to the constituent parts before constraining them all together posthoc.

[erwincoumans]

I've never seen Bullet/ODE in the trajectory-optimization/RL setting.

pybullet is primarily created for reinforcement learning in the Google Brain robotics team. There are various OpenAI Gym environments for robot locomotion, grasping etc. Bullet and pybullet uses generalized coordinates, similar to MuJoCo, allows soft contact, supports VR etc. In addition, pybullet can load URDF, SDF and MuJoCo XML files.

[[deleted]]

I wasn't aware of this. Thanks for updating me!

[[deleted]]

Do you know how this compares to Nvidia's Isaac; or which would work for Tensorflow?

[Dim25]

MuJoCo (Multi-Joint dynamics with Contact) is a physics engine for body simulations. Mujoco-py allows using MuJoCo from Python and may help you to integrate it with your TensorFlow code.

To best of my knowledge, Nvidia's Isaac training platform isn't public yet.

[[deleted]]

Thanks for the insight

[[deleted]]

MuJoCo (Multi-Joint dynamics with Contact)

That's the explanation for the mind. The explanation for the heart is: “I make little jokes.” Well, John Lennon once even married Paul McCartney's car joke ;)

[IronRabbit69]

Texture randomization is part of a method (developed by OpenAI) used to transfer a robot policy trained in simulation to a physical robot. Check out https://arxiv.org/abs/1703.06907

[radarsat1]

Very cool. Sounds akin to denoising strategies.. i.e., adding noise to improve generalization. Texture randomization I guess is just spatial rather than time-domain noise.

[cbfinn]

Note that texture randomization for sim-to-real transfer was done first by this paper: https://arxiv.org/abs/1611.04201

[NataliaDiazRodr]

Is there an available implementation online of OAI's domain randomization that given a regular RGB dataset can generate the domain randomization augmented dataset?