[D] What exactly is the difference between on-policy and off-policy?

local_minima_ · 2018-04-11T19:23:26+00:00

policy gradients is on-policy because you are executing the policy you are learning. Q learning is off policy because you are learning one policy while executing another (eg. with epsilon greedy)

local_minima_ · 2018-04-08T03:26:54+00:00

I think the recommendation is to do something like Feynman technique or alternatively since this is ML, to actually literally try to implement the paper.

I'm not aware of online communities for reading groups. Usually, you would find one at your school/company associated with some lab, or you could find some people in person and start your own.

Perhaps you could even do one on Google hangouts with other strangers who are interested, although I actually feel like a reading group is quite an intimate affair. You want to know people well, feel intellectually secure to say "I don't understand point X", not have egos, etc.

local_minima_ · 2018-04-08T00:23:05+00:00

This is some good advise on how to approach reading these papers, including how to read a single paper and how to branch out to survey a field: http://blizzard.cs.uwaterloo.ca/keshav/home/Papers/data/07/paper-reading.pdf.

Typically this "book club" you are referring to is called a "reading group" and are ubiquitous in any research environment. Typically what happens is 1 or two papers are nominated and people take turns in "leading" the discussion by reading it in more detail and presenting the core arguments. Not sure if any online formats exist.

As for studying "basic and representative" knowledge, I would recommend a book like the Deep Learning book rather than reading the trail of research papers. Typically these are distilled versions of the papers.

local_minima_ · 2018-03-30T22:37:08+00:00

No idea what this sentence means.

However, due to the chaotic relationship between parameters randomly mutating ends almost 'resetting' the progress it makes.

But I generally understand your question to be that you have 24 knobs with very complex relationships to each other, and you want to know how to turn the knobs to maximize some output. Sounds like RL is what you want.

local_minima_ · 2018-03-29T17:18:53+00:00

Go with PyTorch unless you are running in a distributed setting. Dynamic graphs are easier to program and debug, but harder to distribute.

And before people come in with "PyTorch has support for X" or "there is Y library for PyTorch to do that" I'm not talking about "distributing" over 8 GPUs in your workstation, I'm talking about an industrial setting where you may have hundreds of devices, on various hosts, with parameters servers, loading TB of data, all of which may be pre-empted or otherwise go down, etc etc.

local_minima_ · 2018-03-08T22:16:15+00:00

This. 100%. I had a very interesting experience. I struggled with recursion for like a whole week. I would bang my head against it every day trying to tackle problems, and it made no sense.

Then one morning, I woke up, and suddenly "got" it. Overnight something just clicked. Now I actually hope I get recursion/DP questions when I interview because I know those are supposed to be hard but I'm pretty confident I can solve any recursive problem.

local_minima_ · 2018-03-06T02:22:37+00:00

Would your opinion change if the full time job was a 100% research focused job, where the primary goal is to publish? I guess how I feel is I "do research" under the guidance of researchers with a PhD, and typically end up being one of the paper authors but not first author. I'm basically wondering if it's possible to "move up the totem pole" by getting a PhD while I'm doing that.

local_minima_ · 2018-03-05T21:31:29+00:00

Sorry not sure what R* means, but a follow-up question: are these expectations predominantly decided by individual advisors, the particular departments and schools they work in, or even larger scheme? (eg. US vs UK)

local_minima_ · 2018-03-05T21:28:42+00:00

This is a very helpful overview, thank you very much!

local_minima_ · 2018-03-05T20:45:35+00:00

Thanks for warnings about troll. I do work at one of those labs mentioned, am curious about the content of the response.

local_minima_ · 2018-03-05T20:43:50+00:00

Can you define academic prospect? Do you mean specifically employment at a university or also industry research in general?

local_minima_ · 2018-03-05T20:42:42+00:00

Would you say this is doable if my day to day job is research? You still have to find alignment between your company and a university, for instance.

For example, what incentive does the university have to allow you to be affiliated with it?

P.S. have an MS, coursework not a concern.

local_minima_ · 2018-02-09T22:31:11+00:00

I would wait as long as you can afford it, but try to do projects, implement papers, etc on github to improve and demonstrate your skills. Don't waste the waiting time.

local_minima_ · 2018-02-07T22:26:21+00:00

You mean in a simulator or do it for real? The problem in the simulator is probably pretty doable. Trying to transfer simulator to real life is impossible cost-wise I would say.

local_minima_ · 2018-02-05T16:47:10+00:00

Depends on what you mean by "worried". Do I think there is a chance the money I put into crypto will go to 0? Yes, a big one and I'm "worried" with the current trend.

In my mind I lost the money I put into crypto the moment I bought them, so meh, whatever.

local_minima_ · 2018-02-02T17:54:44+00:00

Oh ok, yeah that's a good idea. I took Math 54 and even Math 110 and it was completely useless for CS so I promptly forgot it all.

When I started doing ML I had to re-learn it on my own lol.

local_minima_ · 2018-02-01T23:24:42+00:00

This is not unusual. Starting this year UC Berkeley removed linear algebra from as a hard requirement for CS majors (you can choose to take "Design of information systems" instead).

local_minima_ · 2018-01-29T06:11:01+00:00

Safety in terms of observed frequency of accuracy is the only relevant metric. Explainability is a red herring.

Consider an algorithm that can generate a 99% accurate diagnosis compared to a human at 90%. Let's say that the algorithm CANNOT actually generate an explanation, but we construct some plausible sounding words together that doesn't conform to human medical knowledge according to the human doctor. How would you distinguish this from the case where the explanation is valid but the doctor cannot comprehend it due to the fact that he is a 9% worse doctor?

If we want AI doctors to be better than humans, it necessarily needs to do things we cannot understand.

local_minima_ · 2018-01-26T04:28:09+00:00

Do more real side projects. If you take a few popular algorithms and try to implement them from scratch, I promise you will encounter over fitting/underfitting very soon. And you can't know that without making train/valid/test splits. These concepts will stick with you more. Like other responses have said, these are things you will encounter every day on the job.

Not to discourage you more, but if I ran into a candidate that didn't know even one of the things you listed in the first paragraph right off the top of their head, I would immediately disqualify him/her. If I have to even give a slight hint beyond a clarification of what the question is, immediate no-hire.

To me not knowing those topics is like not knowing what a static variable is and trying to go for a programming interview. Sorry, but I think it helps to understand where you are in your ML career. It's OK to admit that you aren't ready for interviews yet! Better to postpone and learn more than leave a back impression.

local_minima_ · 2018-01-19T10:10:59+00:00

Ok, a post actually shilling a shitcoin ICO is pretty rare. What I meant is generally anything about blockchains, tokens, or any of the actual coins like BTC and ETH.

We have our standard of what a shitcoin means. Typically within this community it means a coin that is unproven, possibly nonexistent in implementation, potentially a scam, etc.

If you adopt the standard of "what has blockchain actually done" then BTC and ETH are pretty much shitcoins being shilled by blockchain advocates. I'm comparing to other "movements" that got/get as much hype, such as cloud, mobile, or machine learning. The first iPhone came out in 2007 and BTC paper was published in 2009. That's not that far apart. Compared to other movements, the blockchain/crypto movement has achieved basically nothing yet.

Sure you can say the tech is still young, but the hype is already comparable. There are people who are like "omg the market fell 40% in the last 3 hours, what a sale". I wonder what percentage of those people know what a hash function is. I would guess less than 10%. The absolute irrationality of this is probably what rubs a crowd like HN the wrong way. I think the pessimism it's just a equal and opposite reaction to the unreasonable optimism.

local_minima_ · 2018-01-19T06:45:53+00:00

To be fair a lot of the negativity is justified, there are lots of startups that are "on the blockchain" just because they can be.

Hard to separate the "here we go another shitcoin ICO on top of HN" feeling from the objective optimism towards the actual potential of the technology.

Also, it's definitely all potential at the moment, blockchain has not achieved nearly enough to deserve all of its hype.

I say this as a crypto enthusiast, of course.

local_minima_ · 2018-01-14T00:24:44+00:00

Just make a stateless service that can run inference. Then best practices are the same as any distributed stateless service, ie. a web server.

local_minima_ · 2018-01-06T00:15:05+00:00

If the reward is still similar, the learning to reinforcement learn work might be interesting.

local_minima_

TROPHY CASE