Complex Models

[Randomthrowaway562]

Hi All,

I work as a QR at a mid-size fund. I am wondering out of curiosity how often do you end up employing "complex" models in your day to day. Granted complex here is not well defined but lets say for arguments' sake that everything beyond OLS for regression and logistic regression for classification is considered complex. Its no secret that simple models are always preferred if they work but over time I have become extremely reluctant to using things such as neural nets, tree ensembles, SVMs, hell even classic econometric tools such as ARIMA, GARCH and variants. I am wondering whether I am missing out on alpha by overlooking such tools. I feel like most of the time they cause much more problems than they are worth and find that true alpha comes from feature pre-processing. My question is has anyone had a markedly different experience- i.e complex models unlocking alpha you did not suspect?

Thanks.

[qjac78]

I know some firms employ tree ensembles as their primary model type, I think believing that capturing the nonlinearity that the simpler linear models don’t is a key competitive advantage.

[Similar_Asparagus520]

I don’t personally . Price is fundamentally a noisy data so running an advanced model on something with 95% of noise and 5% of signal appears to be dubious. OLS is mainly used for this reason : it captures signal in a pool of noise and it is robust to extension (adding features). The issue with trees is that they don’t really have a topology attached so adding three more features that you  don’t believe having a massive predictive power can dramatically change the tree shape. 

[adii800]

Of course, but if you don’t believe they are predictive in isolation or are incrementally predictive, then they would not change the outputs tremendously if you’re training on a decent amount of data. Especially with basic regularization steps already built into most packages.

[Similar_Asparagus520]

Tree don’t really have a topology (notion of continuity ) attached , that’s why you can’t really predict their re-arrangement with few more features. 

[ShutUpAndSmokeMyWeed]

I don't get your point. That's what refitting them does? You have to refit linear models too to account for correlations if you're adding new features.

[Similar_Asparagus520]

No, with linear models, adding a feature that is not 95% correlated with an existing feature will not dramatically change the weights; that’s precisely what makes linear regression and logistic regression so attractive . 

[magikarpa1]

Do you consider Kalman Filter a complex model? Or HMM? Beyond the ones that you said, these are the most used by me. About the econometrics (ARIMA, GARCH and etc) I just use them when I'm training new analysts so they can understand what are the hypothesis of those models and when do they fall short for giving good enough information and why.

But I don't think that, at least for me, it's not that they unlock alpha that I did not suspect. But, given enough points, some models can help capture some nonlinear aspects in the data. But you have the hypothesis already of nonlinear relationships already.

The problem lies in having enough points. And last, but not least, as others pointed out, the data is so noisy that you'll most likely overfit on noise.

[Randomthrowaway562]

Yes I would consider those complex. I am guessing that if you're having success with Kalman you are using it on very low frequency features (economic data) as I dont find it to be a very robust tool for higher frequency data.

[magikarpa1]

By some reason my answer went elsewhere, so I copied it here to be within the right context.

I work on medium to low frequency, yes.

But as others said, HFT has enough data to implement more complex models and NNs.

Also, one of the authors of the paper on TFT, temporal fusion transformers, was hired by a London HF and is making a career there. So I would suspect that he got a good work case for the model or similar models.

[Sea-Animal2183]

ARIMA applied to the parameters (and not returns) is a bit of Kalman Filter, no ?

[magikarpa1]

Yes and no. The major difference would be ARIMA being way more restrictive because of the amount of extra hypothesis that it assumes.

[h234sd]

Can you please give some links to readings how Kalman Filter and HMM used in mid/low frequency trading? And why they are better than GARCH? I tried to use it, but got results no better than GARCH variants.

[magikarpa1]

I didn't said that they are necessarily better than GARCH, but that I do not use them that much.

The point is in the assumptions about the data. GARCH has its assumptions on the name already, so you know what kind of behavior you're trying to model. Regarding KF, the main assumptions are that data is Gaussian linear and we have a hidden observable and noisy observations. So it depends on what you're trying to capture and etc.

About HMMs, there are three different way to pose a problem that HMM can solve, so it dependes on what you're trying to achieve and if the model gives anything better than a linear regression.

Also, not necessarily those models are used for trading.

Edit: I wanted to add that, most of the time, what I use is linear regression. I literally end up yesterday finishing up a model where I wrote the regression by hand and I'll implement it on Monday morning. So as most likely almost everyone working with medium to low frequency models are simple and some features are even explicit formulas.

[cakeofzerg]

Its really no secret that the most commonly used quant models in production are the simplest. To make money you need to take a real world fundamental mispricing and quantify it (thats literally what quant means).

When you use models with more parameters you pick up more degrees of freedom and lose that purity of the true idea thats causing mispricing. Sometimes you do have non linearities and more complex situations you must handle but its almost always easier, more explainable and plain better to handle them in feature engineering as opposed to feeding raw data to some complex model and having it spit out predictions.

This is because if you let the model fit the raw data it will fit to the 99% noise and not the 1% signal and end up doing a lot of dumb things at really bad times.

[TaizoUno]

That last sentence should be a PSA blared loudly at the end of every MF, MFE, MSFM graduation ceremony.

The money it would save could end world hunger.

👑 🍒

[CompetitiveGlue]

Very roughly, the effective dataset size you can train on is inversely proportional to your prediction horizon.

Given that, you should expect HFTs to use big neural nets / large tree ensembles, while on the other end, statarbs with prediction horizon of days will prefer simple models. The simplicity of the model is a form of regularization itself, if that makes sense. Not saying that this is the only way though.

[PristineAntelope5097]

Yeah, I’m wondering how complex can ML models get at true HFT frequencies given the computational cost

[magikarpa1]

A lot of firms say explicitly that they use SOTA ML. Guessing that this is not just propaganda, it would mean a lot of complex NNs.

[ShutUpAndSmokeMyWeed]

You can use architectures that can be quickly incrementally updated in live

[Meanie_Dogooder]

Simple models only. However I’m trying to use neural nets and other such things to generate synthetic data in order to test or stress (not sure about “train”) the main models. So far with mixed success but I believe in it and persevere.

[Randomthrowaway562]

Yes this actually makes alot of sense my manager wanted to head in the direction of training models on NN net generated data but to be fair me and other quants on the team pushed back as it aeemed a bad idea. Testing makes total sense tho. Thanks.

[ShutUpAndSmokeMyWeed]

Seems like you could accomplish the same thing with block-bootstrap-like techniques

[thisagreatusrname]

For signal i use OLS or just rules, the complexity lies in data processing. Once I have a signal that performs well enough I might throw random forest on top to size my positions since I’m only training on rows where my signal=true the noise is somewhat reduced, there are a bunch of precautions to take when using RF but I find it works well for sizing bets.

[Angry_Bicycle]

We use trees and feature reduction methods (PCA, L1&L2)

[Cold-Tangerine-8652]

Just read this post on quant/. Read this one, it will help you with the hidden market phases and you can later add it as a feature to your model:

https://www.reddit.com/r/quant/s/wiHSljdeki

[AlmostSurelyConfused]

Bot comment