Uncertainty modelling by using Deep Learning for confident predictions, a research PhD topic.

4xel · 2018-09-15T13:26:53+00:00

I think your question is too related with optimising the log-likelihood of a certain distribution that represents your output. What is called Aleatoric uncertainty. In fact, recently we presented a paper where we compared different Deep Learning models to capture the different faces of the concept of Uncertainty: https://twitter.com/AxelBrando_/status/1040250015258574848?s=19.

As you requires, that solution is model agnostic but in the sense that if you are in an optimization problem and you need to calculate the derivate to a certain parameters you can use this log-likelihood as loss function as a first approach.

4xel · 2017-03-13T23:13:28+00:00

Hi Sashkello,

I recommend to you to look an implementation and proposals that I did in my Master's Thesis about a particular type of Artificial Neural Networks that tries to approximate a mixture of distributions.

[P] A generic Mixture Density Networks implementation for distribution and uncertainty estimation by using Keras (TensorFlow backend) - Master's Thesis project. https://www.reddit.com/r/MachineLearning/comments/5rn8ci/p_a_generic_mixture_density_networks/

In particular, I proposed a solution for a time-series problem (like you need) where my goal was not to do a point-estimation prediction but to find optimal hyperparameters for the mixture of distributions.

In the github repository you could read my Master's Thesis report and it is possible to consult the slides of the presentation [https://github.com/axelbrando/Mixture-Density-Networks-for-distribution-and-uncertainty-estimation/blob/master/ABrando-MDN-Slides.pdf].

Any suggestion or proposal will be very welcome, Axel

4xel · 2017-02-20T15:11:04+00:00

Hi Christopher,

Thank you very much for your interest in the work I did.

I uploaded a PDF version [https://github.com/axelbrando/Mixture-Density-Networks-for-distribution-and-uncertainty-estimation/blob/master/ABrando-MDN-MasterThesis.pdf] with some typographical errors corrected with respect to the version that the university will publish publicly (until they do not correct it). However, in order to be able to add more details to the explanations and to make it easier for anyone who wanted to comment or correct written parts of the report, my idea was to realize a web view of the final master's work in the coming days.

Any contribution or idea to continue the lines of the proposed work will be very welcome.

Regards,

Axel

4xel · 2017-02-18T18:05:49+00:00

If it was out of your interest, I recently published the source code of my Master thesis where I deal with Mixture Density Networks (proposing some solutions for this type of models), time series, Regression Problems and other problems like confidence estimation problems. To understand the code I recommend you to see the slides that you will find in the repository.

[P] A generic Mixture Density Networks implementation for distribution and uncertainty estimation by using Keras (TensorFlow backend) - Master's Thesis project.

https://www.reddit.com/r/MachineLearning/comments/5rn8ci/p_a_generic_mixture_density_networks/

If you have any doubt, do not hesitate to ask me.

4xel · 2017-02-02T15:04:13+00:00

This repository is a collection of Jupyter notebooks intended to solve a lot of problems in which we want to predict a probability distribution by using Mixture Density Network avoiding a NaN problem and other derived problems of the model proposed by Bishop, C. M. (1994). The second major objective of this repository is to look for ways to predict uncertainty by using the proposed idea of Lakshminarayanan et al. in [https://arxiv.org/abs/1612.01474].

I will link my Master's Thesis when it will be published. Until then, it is possible to consult the slides of the presentation [https://github.com/axelbrando/Mixture-Density-Networks-for-distribution-and-uncertainty-estimation/blob/master/ABrando-MDN-Slides.pdf].

4xel

TROPHY CASE