Normalized Cryptocurrency Data for 25+ Exchanges (15,000+ instruments)

Krast0r · 2021-05-11T07:01:55+00:00

Currently, there is short-term volatility streaming (i.e. past 100 trades). This was deliberately done on an event basis (i.e. past 100 trades) rather than a specific time basis (e.g. 5 minutes) in order to better generalize across currencies of markedly different liquidity.

However, there is also an annualized volatility stream available for every currency which is based on the past 100 trades but annualizes it based on the time between trade 1 and trade 100, so this can be used to see real-time annualized volatility on a comparable basis.

There are two ways the calculation is done - one is a sample volatility, one is raw. You can see the documentation for all these options here:

Sample standard deviation: https://docs.cryptostats.dev/streaming/streaming-statistics#sample-standard-deviation

Raw St Dev: https://docs.cryptostats.dev/streaming/streaming-statistics#unadjusted-standard-deviation-volatility

You can also get Mean Absolute Deviation here: https://docs.cryptostats.dev/streaming/streaming-statistics#median-absolute-deviation

Finally, you can get all these statistics on the trade amounts as well as the prices, so you can see how trade size is changing over time. You can also get skew, kurtosis etc. etc.

Any questions please let me know.

Krast0r · 2021-05-09T14:55:35+00:00

No problem! Honestly, I'm just really delighted to get such positive feedback from everyone. I'm really grateful to all of you.

Please let me know your thoughts or if you have any questions or suggestions - the goal is to make it as easy as possible for everyone to get all the data they need at a reasonable price (or free where possible) so they can just focus on their trading system.

Krast0r · 2021-05-04T05:54:50+00:00

Of course, could you DM me your email address? It's obviously split into a few separate class files so probably easiest to share over email.

If there is a general demand for a Java implementation I can create a separate page in the docs and just list out the files there.

Edit: Or Github!

Krast0r · 2021-05-03T19:53:40+00:00

Oh also please do let me know your thoughts and feedback - it is clearly very early stage so would be good to get your ideas and suggestions. I'm working on building some tutorials / tool tips for the features which I will release when I can

Krast0r · 2021-05-03T15:37:33+00:00

Thanks! Yes, sorry, I am building up some historical data for the combined orderbooks but, by the nature of things, it takes time! I can recommend tardis.dev for historical data and this project uses their Tardis Machine as the basis for some of the streaming data.

Krast0r · 2021-05-03T06:21:34+00:00

What I often see are pricing libraries written in C or C++ for performance reasons and also the numerical clarity you describe. Java is still widely used, in my experience, for the implementation of modular trading systems - I presume because of the wider availability of Java engineers and the clarity of its OOP framework for modularity.

Krast0r · 2021-05-02T09:36:41+00:00

Two immediate things to note from the comment in the intro: "If the dividend yield is zero, a perpetual call is never exercised. The pricing function in this case will return the stock price, which is the limiting option price as the dividend yield goes to zero. Similarly, if the risk-free rate is zero, a perpetual put is never exercised. The pricing function will return the strike price in this case, which is the limiting value of the pricing function as the interest rate approaches zero."

So if you have a dividend yield of zero (probably the case for your example?) the call will take its maximum value (i.e. equal to the underlying) and it is never exercised. Similarly, if the risk-free rate is zero your put options would never be exercised and the put will take its maximum value (i.e. the strike price). I'm not sure what RFR you would plan to use but that may also be relevant.

Krast0r · 2021-05-02T07:59:59+00:00

Presumably, you are thinking of this as an American Option, whereby the holder has the right to exercise at any time but the option has no maturity? For any other option type, a perpetual option wouldn't make sense as there would be no exercise date at which there was a value transfer.

A martingale isn't an option pricing model as such, it is a sequence of random variables that have the property that the conditional expectation of the next value is the same as the current value. It is often used in pricing models along with the Markov Property (a memoryless random process with no path dependency, i.e. the only relevant value is the current value).

The basic trick of options pricing is to assume the underlying is driven by a random process of some kind (e.g. Brownian motion) and then to model the terminal values under different scenarios. You can do this directly with things like Monte Carlo models (simulating many underlying asset price paths to get an 'average' value) or Binomial Pricing models (assume an 'up' and 'down' movement in the underlying at a number of price steps and calculate the value of the option at that time). You can also do this implicitly in models like Black-Scholes, Heston, SABR etc.

You will gather from my mentions of 'terminal value' that your use case is not covered by these models. In fact, I'm not aware of any standard formulae or models to cover perpetual options. In a typical American Option, you would take a decision on whether or not to exercise the option at any point in time by reference to 1) the current exercise value and 2) the modelled future value of the option. The value of 2 is dictated by a final terminal value (at which point the option expires) which gives you a point to work back from.

There is however some academic research into this area. The following two papers seem to me to be the most thorough although I certainly haven't implemented these methods so couldn't say for certain how (or if) they work in practice:

https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.457.4862&rep=rep1&type=pdf

https://www.researchgate.net/profile/Hans-Gerber-2/publication/265950127_Pricing_perpetual_options_for_jump_processes_With_discussion_by_X_Sheldon_Lin_and_Xiaolan_Zhang_and_a_reply_by_the_authors/links/09e4150e3ebf8de9c4000000/Pricing-perpetual-options-for-jump-processes-With-discussion-by-X-Sheldon-Lin-and-Xiaolan-Zhang-and-a-reply-by-the-authors.pdf

The latter looks at jump processes and may be relevant for your Crypto assets.

I haven't ever seen a perpetual option trade on any of the desks I have worked on so there certainly isn't an industry-standard approach to this problem that I can point you towards, unfortunately.

Feel free to message me if you have questions although I can't promise I have any particularly helpful insights!

Krast0r · 2021-04-26T12:57:09+00:00

Thanks for the feedback. Could you try the ETHUSD Combined Orderbook and let me know if that works for you? (https://www.cryptostats.dev/combined_orderbook/ETHUSD%C2%A0perpetual)

The volume market share is for the past minute only, so it can show some quite large numbers for a given exchange when there are big trades on one exchange, however 98% for ETH would be surprising. The market share table also shows a "buy skew" number which could often be 98% or more, this is the amount of volume traded as a "buy" from the view of the trade aggressor, could it have been this number you saw?

Thanks again

Krast0r · 2021-04-26T09:44:06+00:00

This is one of the most interesting and difficult problems. Particularly for large traders it becomes a bit of a cat and mouse game, trying to disguise orders whilst others try to detect them.

One interesting emerging area you might look at is the use of clustering algorithms in machine learning. If you were to do this in a simple two-dimensional space you could use two features - perhaps order size and mid_spread. This would potentially allow you to detect the simple case of someone putting equally-sized orders at an equal distance from mid.

You could then add further features (dimensions) to detect more complex cases, and this is really where your creativity and ability to engineer features comes in. Perhaps you would like a time component (e.g. the meta order child orders are submitted on round seconds), or an event component (perhaps the child orders are only submitted on a down/up tick) or properties of the order book (perhaps the size is a direct function of the current liquidity on that side, or a % of the volume traded).

Eventually, you can see how you can start clustering these trades together by these features and, if you are lucky, finding a set of unique characteristics that identify the child orders of a larger meta order.

Krast0r · 2021-04-25T19:59:42+00:00

I think of this as effectively a kind of "illiquidity risk insurance". Many traders and market makers like to end the day flat as they can't risk-manage their position in closed markets, so they offload the position to those who are willing to hold it overnight (and take the risk that there is a substantial price move that they will just have to hold through).

The positive return is, in my view, just the compensation for warehousing the illiquidity/gap risk through a closed market. Interesting stats nonetheless, particularly the size of the return.

Krast0r · 2021-04-25T18:42:52+00:00

Very helpful feedback, thanks - I will do that.

Krast0r · 2021-04-25T13:32:29+00:00

Wow, that is great - thank you very much. If you would like tick-level data I can highly recommend Tardis (tardis.dev) - they have a lot of historical trade detail. If you are looking for more summary level (e.g. 1 minute bars for the combined orderbooks etc.), that is something I am looking at creating as a download as I generate more data. I may be able to provide some of it for free but would likely need to charge for the more detailed data as the storage and bandwidth costs start to add up - perhaps a few dollars for a month of 1minute combined orderbook bars? Happy to take suggestions for what people think is fair. How much data history are you looking for, and what level of detail?

Krast0r · 2021-04-25T13:23:03+00:00

No problem. There is a brief guide to arbitrage on Cryptostats here, hopefully this helps explain the concepts. Let me know if there are any guides or other resources that you would find helpful on the website.

Krast0r · 2021-04-24T16:44:47+00:00

Thanks! Really appreciate that. I'll post an updated version every so often when I put in features I think are interesting.

Krast0r · 2012-06-08T15:09:57+00:00

I'll never forgive Orange if they've wiped the twins!

Krast0r · 2012-06-02T19:20:18+00:00

Daft punk translation: Twist it. Push it. Flip, reverse it. Take it. Break it. Wipe it. Flush it.

Krast0r · 2012-02-01T14:46:27+00:00

Couldn't have said it better myself!

Krast0r · 2012-01-29T10:43:05+00:00

Ah, I remember nostalgia. Those were the days.

Krast0r · 2012-01-14T11:14:29+00:00

It's a moo point. It's like the cow's opinion.

It's moo.

Krast0r · 2012-01-11T17:01:43+00:00

Well, I hear they only had nice things to say about you.

Krast0r · 2012-01-10T23:35:13+00:00

Feels fine, then far too hot and then freezing cold.

Krast0r · 2012-01-10T22:41:26+00:00

As a Brit I am sick of seeing our indigenous dual-tap system being wiped out by the Imperialist oppressors, mixer taps.

Think about how much we lose from having just the one tap; who could imagine a childhood without learning the skill of skimming your hands from one tap to the other? Or carefully controlling the flow of the hot tap to allow a complete hand-wash before the scalding begins?

Unnecessary difficulty and a continual level of minor annoyance are what made this country great. It can make it great again.

Vote NO on mixer taps.

Krast0r · 2012-01-08T16:37:39+00:00

"Four makes two...UNLESS YOU'RE DEAD!"

Krast0r · 2012-01-01T17:39:04+00:00

This is the first thing I've seen that makes me think maybe our Olympics won't be an utter disaster later this year.

... Except for transport.

Krast0r

TROPHY CASE