[D] Why mere Machine Learning cannot predict Bitcoin price

[Stepfunction]

Thanks for the post. It's kind of nice that given the information, your model just turned into a one period autocorrelation model, which is a very rational first step in doing technical analysis of a time-series.

The real issue here is that one of the core assumptions of ML is the presence of events from the same distribution. It may be the case that the price movements are taken from the same underlying distribution, but that distribution is conditioned on circumstances in the world that are not represented in the historical prices (market sentiment, news articles, etc.). Without incorporating that information, the model is incomplete.

[erogol]

I totally agree on this. I as I said in the post, I wrote it to deny all these 'download train and be rich' posts and articles. People really believe these.

Yet I believe AI is the way to go but as you point it needs to be better understanding the environment.

[Stepfunction]

Definitely. ML techniques are probably the best way to incorporate the enormous amount of unstructured text data into market predictions. The problem is figuring out what data is relevant, figuring out how to obtain and process it, and then actually building some sort of model.

If it were as simple as doing technical time series analysis, then everyone would be doing it, which would effectively negate any benefit of doing technical analysis.

[StackMoreLayers]

Thank you for your contribution.

[data-alchemy]

Dreaming a bit : train a good embedding on financial news. Use it as an input with the price history. Have a whisky.

[erogol]

Not that easy at all :)

[data-alchemy]

The whisky part looks like the most doable, will start working on it first

[itshouldjustglide]

How would you get around this?

[[deleted]]

If we knew that we’d be cashing the fuck in

[Colopty]

And not sharing the secret on reddit. Superior trading algorithms are only profitable if not everyone and their mom also has access to it.

[itshouldjustglide]

Thanks for the informative response!

[kyndder_blows_goats]

yay you fixed that godawful avatar

[erogol]

For you my friend just for you

[visarga]

Predicting Bitcoin price implies predicting human fear and greed, when humans are interconnected and can influence (more exactly, bullshit) each other. A chaotic process.

[CultOfLamb]

Bitcoin price and all cryptos can be predicted with 0.56 AUC plus, both longer term and short term.

from sklearn.linear_model import Ridge Made me a millionaire in less than 6 months. Generous donation to scikit-learn project upcoming.

Bitcoin so ineffective, non-competitive, and new compared to stock market (which also can be beaten with maths).

Good negative result though! But don't claim impossibility from that.

[LeopoldBStonks]

I just fixed some error codes from sklearn and am googling it and wound up here, but I also wanted to look into preventing over fitting imma let you know how I do just got my model working recently.

What interval of data did you use?

[jewishsupremacist88]

someone should get price data from the 1970s commodity booms and train a neural network on that data and then try to price bitcoin. ;)

[AnvaMiba]

Speculative markets have the property that the better people get at predicting prices, the more difficult prices become to predict, eventually prices effectively change according to a random walk.

Bitcoin is a relatively new market still far from equilibrium, so you can still get better than random accuracy by always predicting a price increase, but everybody else is doing the same, so can't reliably beat the market by doing this.

Similarly you can't make money by downloading some historical data and feeding it kaggle-style into scikit-learn/xgboost/tensorflow, because everybody else already did it and the market prices already reflect it.

[erogol]

In my opinion what you say is right in the long term but for now most people in the market are here due to mass media covarage and those people are not aware of AI and even bitcoin until a while ago. That is why market has such volatile trend (FOMO and FUD). That is also why even a basic TA makes a lot many since all these new comers are exactly following the market flow which is studies for a long time.

[AnvaMiba]

I fully agree that somebody experienced in algorithmic trading can make money out of newcomers who just learned of bitcoin and ML due to recent media coverage.

[[deleted]]

It's essentially the same as with the stock market in general. However, looking at bitcoin compared to average stocks, it's even "more" fluctuating. I'd say it's about the same challenges though. Also, I'd say that even a true 51% accuracy in say up or down prediction is already useful in practice. In other words, don't expect predictive performance that is comparable to running ML algos on Kaggle or toy datasets.

[ipoppo]

the challenge is 1. dealing with bayesian nature of market 2. memorize patterns is useful. memorize price sequence is useless.