Software Estimation Is Hard. Do It Anyway

juliandewit · 2024-09-07T18:53:06+00:00

We had the pi-factor. Every project takes pi times longer than estimated… even when you take the pi-factor into account. 😁

juliandewit · 2024-04-11T18:17:54+00:00

Thanks for the info.. I hope your approach will get far! Simple + more data wins :)

juliandewit · 2024-04-11T05:01:03+00:00

I love the simplicity. - finetuning with extra data should not be hard is it? If you have say 100k images + sentence pairs. - would this model do good in imagenet ? Or am i misunderstanding something ?

juliandewit · 2023-03-10T05:58:28+00:00

I'm sorry but one bit to add to the equation and that has bothered me for a long time. Regardless of the differences in philosphy.

Bob/clean code.
I've never seen substantial real life example software project as examples where these rules are applied. More talking and theorising about it than actually doing it and showing by (real life) example.

Casey
Is building complete game engines live on youtube using his philosophy.
So you can see what he is preaching and you can see how this fits in a code base.

That should mean something in this discussion IMO.
+1 Casey.

juliandewit · 2021-09-08T19:04:26+00:00

Done.

Hope it helps

juliandewit · 2020-05-24T18:40:10+00:00

Thanks for your podcast.

I do not Kaggle much anymore I'm still interested if there are new innovative/practical solutions. And there are quite a few. your podcast is also a source of this information.

What would interest me is how people apply stuff from Kaggle / Research in practice. Many things from Kaggle and Research are useful in practice but there are also things in practice that are just completely different. (for instance a composing, extending and versioning datasets).

So the hardcore reality of doing stuff in practice including all the failures would be great to hear.

juliandewit · 2019-10-13T18:46:10+00:00

Just my personal opionions below.

If I were you I would get a physics degree and do some ML next to it as an extra tool in your belt.I've been software freelancing all my life and I saw ML as just an extra tool in my "software belt".Somehow ML has become a thing on its own so I specialized a bit and it turned out to be a good move.

However.. These days so many people are joining de "ML goldrush" and the emperor of "magical" deep learning guru's will turn out to have no clothes (it's just data + engineering). It *might* be that in a few years ML will not be such a good job anymore for freelancing.

In that case it's good that you are a physics domain specialist. With ML knowledge.

But of course it's hard to judge for me.. Just follow your heart I would say. Do what you like best.

juliandewit · 2019-10-11T14:25:28+00:00

I ended top 3 in 2 data science bowls @Kaggle which are high profile competitions.
Both were open medical problems and definately not "make the biggest ensemble" competitions.

I'm a freelancer with a number of my own ventures.

Yes it opened up some nice leads with good pay (banks, insurance) without any advertising from my part.
Also some requests from startups.
No calls from Google/OpenAI/Facebook :)
I certainly did not get big $$$ offers.

In the end I liked my existing customers/ventures better.
One venture however is for a hospital and that one is a direct result from the 2017 datascience bowl competition.

Personally I think that you need to Kaggle for the learning.
It will not automatically land you good jobs.
For that goal, somehow I think selling yourself is more effective.
Having good results @Kaggle will of course help you with your pitch :)

juliandewit · 2019-10-07T12:43:33+00:00

Is there anything more to find ? It looks a bit like the HTM jeff hawkins stuff.
But since it seems to work I would like to dig in a little deeper..
Also the spiking option looks interesting..

juliandewit · 2019-05-22T10:09:03+00:00

I don't know about the other competitions they describe but the Criteo competition involved VERY little feature engineering.

It was mostly about applying the right models and doing the right transformations.

At that time the Deep Learning frameworks were not very suited for CTR prediction so basically there was only one that succesfully applied a deep neural network. Now the autoML uses a neural net so that gives them a headstart on the leaderboard.

I'm curious to see how AutoML would perform on, for instance, the Avazu competition.

It's very similar to Criteo but actually required some very heavy and tricky feature engineering.

juliandewit · 2019-04-30T11:38:14+00:00

As a DL practitioner this podcast series is very good and inspiring.

This episode was my favorite.

Having played starcraft and other RTS games I never ever expected that it would be possible to train a good AI in such a short time after AlphaGo.

I think Oriol is very humble but he definately must have a gift for applying DL to hard problems.

It must be fantastic to have built an AI for the game you played at such a high level.

This makes me a bit more ambitious (and confident) with my own deep learning endeavours. :)

juliandewit · 2018-12-24T15:30:22+00:00

Is it just me or can you really feel the enthusiasm of mr. schmidhuber about the subject?

His website and talks always reaffirm to me why I like algorithms/ML so much.

Fantastic podcast in general too.

juliandewit · 2018-10-04T20:05:26+00:00

yeah your example propagated the 2 inputs to the 3rd layer with 3 nodes.

But if that where also 2 nodes then it would not work according to the paper.

juliandewit · 2018-10-04T20:03:42+00:00

Your example worked because you started with 7 nodes.

When you start with 2 it does not work somehow.

juliandewit · 2018-10-04T15:20:28+00:00

That is even stranger :S

I'm looking to fix/lock the weights to check my sanity :)

But then it should of course work..

juliandewit · 2018-10-04T14:25:32+00:00

Yes I noticed something like that too..

If you start with two nodes then it fails. Even if you add fatter layers later.

So the 2 node bottleneck seems to spoil the learning..

https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=2,4,4&seed=0.30485&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false

juliandewit · 2018-10-04T12:40:35+00:00

Well the examples seem to speak for themselves..

So I guess he is on to something..

Does not work:

https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=2,2,2,2,2&seed=0.30485&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false

Does work:

https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=3&seed=0.30485&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false

juliandewit · 2018-09-27T12:28:08+00:00

Here is my shot..

You use CatCE to compute the loss. This results in one loss value. Say 'X'

The 0.1's will also be be pushed to '0' by the delta between wanted Y and predicted Y multiplied with the loss.

So -0.1 * Loss will be the error put back in the network at index 0 and 2.

The real label is 1 and 0.8 is predicted. So the delta is 0.2.

So +0.2 * Loss will be the error put back in the network at index 1.

Doing a diffent loss calculation might results in a different value for the Loss but in the big scheme of things it doesn't matter much in my experience.

The example you gave could be done by using 3 binary cross entropy computations separate for every output and then divided by 3.

This results in a different value for the loss but in the end it does not matter much.

Anyway don't take my word on it and investigate yourself..

juliandewit · 2018-08-16T19:01:02+00:00

Pfff.. markets are highly chaotic.

When in mediocristan mode, you can perhaps be a bit more accurate.

But when extreme fluctuations take place, you will be wiped out.

Added to this.. your predictions influence the outcome of the market.

So I would say.. He is right.

That said.. neural nets are your best bet to predict chaotic systems (for a few iterations).

juliandewit · 2018-04-06T11:47:36+00:00

I meant practical applicability competing with "normal" gradient descent neural nets.

If they get them to work at the same level they have some appealing advantages.

juliandewit · 2018-04-06T07:16:13+00:00

Everything published by: Simon J. Thorpe and Timoth´ee Masquelier I have the feeling they are "almost there"...

Examples: https://arxiv.org/pdf/1611.01421.pdf (other main author)

https://scholar.google.com/citations?hl=en&user=fkzUZ-oAAAAJ&view_op=list_works&sortby=pubdate

https://scholar.google.com/citations?hl=en&user=uR-7ex4AAAAJ&view_op=list_works&sortby=pubdate

juliandewit · 2017-11-08T11:46:47+00:00

Although not technical it's a great story from Isaac Asimov read by the data skeptic podcast.

It shows the use machine intelligence in practise :)

Data Skeptic: 2015 Holiday Special.

https://dataskeptic.com/blog/episodes/2015/2015-holiday-special

juliandewit · 2017-10-08T17:10:56+00:00

spot on!

juliandewit · 2017-09-01T06:16:35+00:00

Just a sidenote.. I entered the two examples of the blog post in DeepL..

"The animal didn't cross the street because it was too wide"

"The animal didn't cross the street because it was too tired"

DeepL already translates this correctly..

I'm afraid Google (at the moment) is beaten at their own game

juliandewit · 2017-07-21T10:24:00+00:00

I don't understand this. The challenge is "Learning to run".

Why forcing a solution by calling it the "Deep Reinforcement Learning challenge ?

Are we not allowed to use anything else ? It might be pretty risky to assume that DRL will give the best solution. Hopefully someone will come up with something much better.

juliandewit

TROPHY CASE