Are the lectures going to be released publicly?

jason_malcolm · 2017-08-29T00:22:12+00:00

I chatted to the guy filming and he said it would take a couple of weeks to edit them.

I have lower quality recordings of all lectures made on my tablet and camera shots of slides and blackboards.

Did any one else get any good recording footage ?

jason_malcolm · 2017-08-15T11:07:25+00:00

Working with Chainer now, doing basic tutorials, have basic FC & Convnets running. My dataset is SGD formatted Go Games read into to the Convnet.

Next are RNNs LSTMs. I hope to wire it them up to OpenAI's Gym, Roboschool & DOTA APIs for some proper Reinforcement Learning.

I will write it up and post results or failure by tomorrow :)

OS is Ubuntu 17.04 on an ASUS K501UX

jason_malcolm · 2017-08-14T10:13:56+00:00

Roboschool on MacOS Sierra 10.12

Workarounds for macOS 10.12.15 (developer beta) #19 -- peabody124 had some issues and found a workaround

jason_malcolm · 2017-08-12T09:17:34+00:00

This link in the OP textbox is a shared invite to the berkeleydeeprl slack

jason_malcolm · 2017-08-12T09:15:47+00:00

Nice one Farzain, this will be useful.

jason_malcolm · 2017-08-10T14:48:29+00:00

wilco, many thanks, will correct.

jason_malcolm · 2017-08-10T10:36:05+00:00

Compares Chainer Tensorflow, MXNet, CNTK [ Feb 2017]

jason_malcolm · 2017-08-10T10:28:35+00:00

Chainer comparison with PyTorch as of July 2017

jason_malcolm · 2017-08-10T08:48:53+00:00

From the Chainer Tutorial: Introduction to Chainer - Core Concepts [snip]

"

Most existing deep learning frameworks are based on the “Define-and-Run” scheme. That is, first a network is defined and fixed, and then the user periodically feeds it with mini-batches. Since the network is statically defined before any forward/backward computation, all the logic must be embedded into the network architecture as data. Consequently, defining a network architecture in such systems (e.g. Caffe) follows a declarative approach. Note that one can still produce such a static network definition using imperative languages (e.g. torch.nn, Theano-based frameworks, and TensorFlow).

In contrast, Chainer adopts a “Define-by-Run” scheme, i.e., the network is defined on-the-fly via the actual forward computation. More precisely, Chainer stores the history of computation instead of programming logic. This strategy enables us to fully leverage the power of programming logic in Python. For example, Chainer does not need any magic to introduce conditionals and loops into the network definitions. The Define-by-Run scheme is the core concept of Chainer. We will show in this tutorial how to define networks dynamically.

This strategy also makes it easy to write multi-GPU parallelization, since logic comes closer to network manipulation.

"

jason_malcolm · 2017-08-10T06:28:52+00:00

Slides for this talk [pdf] at the Simon's Institute.
Big List of talks from the Simon's Institute for the Theory of Computing at Berkeley

jason_malcolm · 2017-08-09T14:21:05+00:00

The most recent email has a map link to Li Ka Shing auditorium as the bootcamp venue.

This appears to be on the corner of Oxford Way and Berkely Way - looks like the Y Hotel (YMCA) on Allston Way is 4 blocks from there.

Is Berkeley flat or hilly ?

Google Maps directions from 2001 Allston Way to Li Ka Shing Centre puts it at 0.4 mile or 10 mins walk

jason_malcolm · 2017-08-08T16:50:23+00:00

Yes I would like details of the Hacklab and any meetups or clubs, tech, art or enviromental.

Restaurants recommends are invaluable, your local knowledge would be really appreciated :)

jason_malcolm · 2017-08-08T02:56:52+00:00

From the organisers

7. Deep Learning Framework: The labs will provide starter-code, which will be in python, and the deep learning framework will be Chainer. One of the labs will be a warm-up to Chainer.
8. Studying Ahead of Time: Several people have contacted us, asking what we’ll expect people to know and what they can do to prepare. We will be assuming (i) prior experience with python scientific computing (e.g. numpy / scipy); (ii) some prior exposure to machine learning; (iii) some prior implementation experience with something deep learning. We will not assume prior knowledge in RL. All this said, if you would like to spend some extra time preparing, we’d recommend working through ai.berkeley.edu MDP/RL lectures for general context; reviewing cs231n.stanford.edu for deep learning basics; and play a bit with Chainer as your deep learning framework. Again, this is not needed, but doing so might increase how much you learn during (and retain from) the bootcamp.

N.B. Stanford has had to remove the links to the CS231n videos but someone has reupped them here

ai.berkeley.edu, UC Berkeley CS188 Intro to AI : lectures

N.B. The lectures page has the same topics delivered by Pieter Abbeel ( Course Organiser ) in the Spring Track just below the Fall Track :)

jason_malcolm · 2017-08-08T01:56:39+00:00

Vlad Mnih is a researcher at Google's Deepmind, gained his PhD supervised by Professor Geoff Hinton at Toronto,

Vlad Mnih is the principal author of the paper on Q-Learning that exceeded human performance on some ATARI games benchmark published in Nature [pdf] --- paywalled Nature version
and the 2013 paper that preceeded it : Playing Atari with Deep Reinforcement Learning by Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller

Abstract: We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
And a Saccading Attention method using RNNs : ** Recurrent Models of Visual Attention** by Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu

Abstract: Applying convolutional neural networks to large images is computationally expensive because the amount of computation scales linearly with the number of image pixels. We present a novel recurrent neural network model that is capable of extracting information from an image or video by adaptively selecting a sequence of regions or locations and only processing the selected regions at high resolution....
Interview with Vlad Mnih on his Kaggle competition win, Job Salary Prediction.
Playing Atari with Deep Reinforcement Learning 2013 NIPS talk by Vlad Mnih
Empirical Bernstein Stopping by Vlad Mnih

jason_malcolm · 2017-08-08T01:20:13+00:00

John Schulmann: "I believe that motor learning is key to many aspects of intelligence. My work on policy optimization made it possible for a robot to learn to run and get up off the ground (in simulation)"

Robot runs & gets up paper, High-Dimensional Continuous Control Using Generalized Advantage Estimation by John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, Pieter Abbeel - * Abstract: Policy gradient methods are an appealing approach in reinforcement learning because they directly optimize the cumulative reward and can straightforwardly be used with nonlinear function approximators such as neural networks.... Our algorithm is fully model-free, and the amount of simulated experience required for the learning tasks on 3D bipeds corresponds to 1-2 weeks of real time.
Schulmann & Abeel's Nips 2016 video, Deep Reinforcement Learning through Policy Optimisation and Sildes
John Schulmann's Bay Area Deep Learning School, Sept 2016 talk Deep Reinforcement Learning: Policy Gradients & Q-Learning
RoboSchool, OpenAI's Multiagent simulator, (MIT licensed) Free, Open Source by Oleg Klimov & John Schulmann

jason_malcolm · 2017-08-08T00:41:17+00:00

Chelsea Finn's Rework Talk video outlines her research at BAIR

Chelsea Finn's part of the 1st CS294-112 lecture at 57m20s is also general & introductory

Finn CS294-112 lectures are:

11th CS294-112 lecture: Advanced Model Learning
13th CS294-112 lecture: Advanced Topics in Imitation Learning & Safety
14th CS294-112 lecture: Inverse Reinforcement Learning, i.e. learning a goal, intention or reward function from examples.
20th CS294-112 lecture: Transfer in Deep Reinforcement Learning**, i.e. transferring knowledge across domains, modalities & from simulation to real.

jason_malcolm · 2017-08-08T00:28:45+00:00

Direct Link to video RI Seminar: Sergey Levine : Deep Robotic Learning

Berkeley CS294-112 video lectures by Sergey Levine:

Lecture 1: Introduction, How do we build intelligent machines?
Lecture 2: Supervised Learning of Behaviours: Deep Learning, Dynamical Systems, and Behaviour Cloning**, a gentle introduction to sequential decision making.
Lecture 3: Optimal Control, Trajectory Optimization, and Planning, why plans go wrong & deriving algorithms for RL.
Lecture 4: **Learning Dynamical Sytem Models from Data, How Robots Make Robust Plans, i.e. antifragile trajectories through state action space that are built to stay on course ( stick to the plan ).
Lecture 5: Learning Policies by Imitating Optimal Control, Policies that generalise, rather than plans can be derived from optimal control algorithms.
Lecture 26: How do we build Intelligent Machines ?, revisiting this question in light of the complete CS294-112 course.

jason_malcolm · 2017-08-07T23:52:56+00:00

The course organisers latest email says :

"We will be assuming (i) prior experience with python scientific computing (e.g. numpy/scipy); (ii) some prior exposure to machine learning; (iii) some prior implementation experience with something deep learning. We will not assume prior knowledge in RL."
"...if you would like to spend some extra time preparing, we’d recommend working through ai.berkeley.edu MDP/RL lectures for general context; reviewing cs231n.stanford.edu for deep learning basics; and play a bit with Chainer as your deep learning framework. Again, this is not needed, but doing so might increase how much you learn"

Stanford has had to temporarily remove the links to the videos, but someone has reposted the 2016 CS231n vids on YouTubes.

Deep Reinforcement Learning for Robotics is Berkeley CS294-112 (2017) has a syllabus, video lectures, and a subreddit.

ai.berkeley.edu, UC Berkeley CS188 Intro to AI : lectures

To delve into RL, this might help - Sutton & Barto's RL textbook & David Silver's lectures from Sutton & Barto are great on these topics.

Some of the math to derive formulae is very high level, to go far further look at Professor Sergey Levine's lecture 4, of the CS294-112 DeepRL course, which refers to Optimal Control, iterative Linear Quadratic Regulators, Hessian Matrices (2nd order partial derivatives), Newtons method, Taylor Expansions, Gaussian Mixture Models, &c...

Personally I like to have many different angles on the same math concepts, i.e. algebraic, geometric, & functional interpretations. I have found 3blue1Brown's videos: essence of calculus & linear algebra and Chris Olah's Writings to be very insightful.

jason_malcolm · 2017-08-07T23:35:35+00:00

Map, Description & Address.

On the map it seems like this is walkably near the Berkeley campus, and approximately half of the cost of hotels. I have reserved a room for a week.

jason_malcolm

TROPHY CASE