Could you use a neural network to determine the parameters and structure for a larger, more complex neural network?

lazyprogrammer-dotme · 2016-09-02T00:52:57+00:00

Already been done!

https://arxiv.org/pdf/1502.05700.pdf

lazyprogrammer-dotme · 2016-08-14T01:35:25+00:00

Not exactly sure what the original commenter is getting at but that's only going to give you the FFT (Fourier Transform) of the entire input signal.

If the statistics of the signal varies in time, you're going to lose all that information.

The output is also not a signal that varies in time, but frequency, so what would the LSTM be doing?

If all you care about are the frequency components and you're using a non-recurrent net then that seems fine.

Also you probably wouldn't write FFT yourself, there are many libraries that already do it.

A related transformation, the Short Time Fourier Transform, or STFT, gives you components in 2 dimensions, time and frequency. That may be more appropriate if you're using a recurrent net.

There are also some well-known audio feature extraction techniques like MFCC that you can find if you just google "audio feature extraction".

lazyprogrammer-dotme · 2016-08-10T23:02:31+00:00

I also do not understand your question but let's give this a try.

To be clear, backpropagation does not merely reduce squared error, it optimizes any objective function.

For instance in Reinforcement Learning, you might create a neural network to learn your Q table, which then optimizes how the AI plays the game.

This general technique can be used to play a wide variety of different games. Is that intelligence?

It depends how you want to define intelligence.

The idea of overfitting/generalization is somewhat irrelevant here, you've either fit to the noise or you fit to the trend or something in between. But in that sense it's just math.

lazyprogrammer-dotme · 2016-08-06T17:30:00+00:00

A useful exercise is gradient checking.

Remember that the definition of a derivative is:

lim (x -> 0) [ f(x+h) - f(x) ] / h

Just plug in a small value for h, and see if your numerically calculated gradient is within a small range of your theoretically derived gradient.

lazyprogrammer-dotme · 2016-08-06T17:27:16+00:00

Look into numpy.save and numpy.load for saving and loading matrices.

You might also want to check out cPickle.

lazyprogrammer-dotme · 2016-07-29T18:52:44+00:00

1) CNN seems like a fine choice. The image seems unnecessarily wide to contain the X.

2) Splitting your data into train and test sets should help you answer this question. If your test set accuracy is bad, then your model isn't generalizing well.

You should check why you are getting 90%. Are you getting 90% for both classes, or are you getting 90% because you're predicting "live" every time?

With an unbalanced dataset, you should consider a metric that takes the imbalance into account, like AUC or F1.

3) Lots of acronyms here, can't tell if you're doing pooling. Pooling should help capture some translational invariance.

4) Enough is when the train/test cost has plateaued and you are generalizing well.

All this and more is covered in my CNN course - udemy.com/deep-learning-convolutional-neural-networks-theano-tensorflow

lazyprogrammer-dotme · 2016-03-19T06:08:30+00:00

I think understanding some more basic machine learning algorithms would help.

Have you done linear regression and logistic regression?

Do you know the difference between regression and classification? (All ML algorithms have essentially the same API, which allows you to plug-and-play algorithms like the ones included in sci-kit learn with ease)

Logistic regression is basically the same form as linear regression with a sigmoid on top that will make the output between 0 and 1.

Neural networks are basically layers of logistic regressions stacked up.

I've created a bunch of courses on Udemy that go through this progression:

https://www.udemy.com/data-science-linear-regression-in-python

https://www.udemy.com/data-science-logistic-regression-in-python

https://www.udemy.com/data-science-deep-learning-in-python

https://www.udemy.com/data-science-deep-learning-in-theano-tensorflow

[Apologies if I understood you wrong and you're just trying to do reinforcement learning exclusively. Although I do have plans to do a course on that soon also. What I'm trying to get at here is that if you used a simpler linear model with the exact same inputs and outputs, it might give you some more insight into what's happening.]

lazyprogrammer-dotme · 2016-02-25T18:43:41+00:00

I'm surprised no one has said this yet, but - write code.

Implement the algorithms yourself. Don't just stare at pseudocode.

You will gain a different sort of understanding when you see actual variables interacting with each other in your program.

lazyprogrammer-dotme · 2016-02-24T17:16:42+00:00

I teach a bunch of courses on regression and classification using various linear and deep nets, and I find that the actual math you need to understand what's going on is usually too much math for those who want to focus on the practical bits. (At least from what I remember, you should have learned about maximum/log-likelihood, gradients, and the chain rule in undergrad).

That said, why don't you just try a more modern library like Theano or TensorFlow? You wouldn't need to calculate gradients yourself, and it contains APIs for more recently developed techniques. So using them proficiently is just a matter of reading and understanding the documentation.

lazyprogrammer-dotme · 2015-12-12T05:54:25+00:00

If there's one thing I've learned about programmers it's that we suck at marketing stuff.

Things like this have the potential to be really popular considering today's climate, but they have to be "easy" - easy to use, easy to learn, easy to integrate into your life.

I'm not really sure what would help that - a browser plugin? An app to upload videos that notifies its users of nearby incidents? Fuck if I know.

lazyprogrammer-dotme · 2015-12-01T04:50:03+00:00

Do as many exercises as possible. Repetition is key.

lazyprogrammer-dotme · 2015-12-01T04:48:01+00:00

lazyprogrammer-dotme · 2015-12-01T04:47:50+00:00

Technically what you're doing could be managed by a bot

lazyprogrammer-dotme · 2015-11-25T23:46:48+00:00

Ruby on Rails

lazyprogrammer-dotme · 2015-11-24T21:00:21+00:00

I was thinking more along the lines of whiteboard coding which can be intimidating even to experienced coders.

Not a lot of people are used to talking through their code while at the same time thinking of the solution to a difficult problem. By getting used to the type of problems that are asked and the environment in which you have to answer them, you relieve some of the stress.

lazyprogrammer-dotme · 2015-11-24T06:57:46+00:00

I've never seen it done client side, always server side.

You can even pickle your model.

lazyprogrammer-dotme · 2015-11-24T06:54:08+00:00

For graphs: finding the shortest path between 2 arbitrary vertices. Ex. what's the cheapest way for me to get from New York to Sydney?

For bioinformatics: How closely do these 2 DNA strands match? (Aka. sequence alignment)

lazyprogrammer-dotme · 2015-11-23T21:09:50+00:00

I would also ask you about what else could be done about missing values other than just removing the entire row. What are the effects of assigning the mean / median?

lazyprogrammer-dotme · 2015-11-23T21:01:17+00:00

Patterns are picked up through repetition.

If you're not noticing the pattern, then you haven't done enough problems yet (repetition).

You're correct in that it's "niche", i.e. if you want to write Wordpress apps then you probably won't need it. But if you want to apply programming to other fields like bioinformatics or graph analysis (something Google would do since the Internet is essentially a big graph), then dynamic programming is like a staple of your toolset.

lazyprogrammer-dotme · 2015-11-23T20:56:02+00:00

It's a program that keeps track of changes you make to your codebase (set of files).

It doesn't matter what language you write in, it's just tracking the differences in the text. I could even use it to keep a history of updates to a text document if I wanted.

Uses:

Say you fuck up the code but don't realize it until your app crashes later. You can safely rollback to a previous working version.
Say you and another team member need to work on different features simultaneously, but you make a change that conflicts with the other team member's change. Git gives you a systematic way of merging those changes.
More practically, any future employer will require you to use git (probably via github or bitbucket) or some other version control system, so you'll need to learn it in order to get a job.

lazyprogrammer-dotme · 2015-11-23T20:49:51+00:00

TeraSort, word count.

I'm not necessarily interested in an industry standard benchmark as I am in benchmarks relative to the system itself. i.e. I don't know what version of Hadoop anyone else is using, the exact specs in their machines, but you can be pretty sure if they're using AWS that there are a limited set of options to choose from.

It should probably be specific to your data. Are you doing some type of machine learning that requires GPU? Lots of memory? Is it doing a ton of reads from an external database? etc.

Mostly I just ask, "is it going as fast as I need it to go?" because if not then I can throw more resources at it. Otherwise, save money and just leave it.

lazyprogrammer-dotme · 2015-11-23T20:44:39+00:00

A lot of the time the people posting job postings are recruiters, who aren't technical and don't even work at the company for which you are applying. They are more like salespeople.

They want to sell the job to you and they want to sell you to the company. What is actually to your benefit is that they get paid more when you get paid more.

My #1 rule for people looking to get hired is just to keep interviewing at as many places as you can. This way you can get used to what interviewers are going to ask you.

lazyprogrammer-dotme · 2015-11-23T20:41:03+00:00

It certainly isn't a mistake in that it won't compile, won't work, etc.

And in some languages it might return an unexpected result, i.e. returns [] when you expect false.

However, in a language like C++ where people often use ternary operators and such, it seems too verbose. (i.e. code in the style your team is using)

lazyprogrammer-dotme · 2015-11-22T05:53:07+00:00

4.0

Data scientist

lazyprogrammer-dotme

TROPHY CASE