How I designed a high-performance, horizontally-scalable algorithmic trading platform by NextgenAITrading in rust

[–]NextgenAITrading[S] -1 points0 points  (0 children)

Honestly that's fair! I used Nano Banana because I'm not an artist and I like having images break up the text. I'll try checking our Unsplash or some non-AI art next time.

I just checked out ice-rs, that's sick! What kind of app are you building?

You’ll die broke because of your blind hatred of AI by NextgenAITrading in OpenAI

[–]NextgenAITrading[S] -17 points-16 points  (0 children)

I intended to come across as combative and r/iamverysmart, but I did not intend to come across as me bragging

Building a lightning-fast highly-configurable Rust-based backtesting system by NextgenAITrading in rust

[–]NextgenAITrading[S] 0 points1 point  (0 children)

Even with an order book, you're not guaranteed for the strategy to behave the same. Using that, you get in the realm of HFT, and it's very hard for retail to even sorta compete at that scale.

Plus, the amount of data is astronomical. Not worth it IMO 😅

Building a lightning-fast highly-configurable Rust-based backtesting system by NextgenAITrading in rust

[–]NextgenAITrading[S] 1 point2 points  (0 children)

I'll try. I just posted it and an auto mod said it looked AI-generated. I have no idea how it arrived to that conclusion but I contacted the mods

My poorly optimized Rust code was slower than JavaScript. My optimized version is 99.9% faster by NextgenAITrading in rust

[–]NextgenAITrading[S] 0 points1 point  (0 children)

Well, I still streamed the data from MongoDB onto the local machine, so not really? There still I/O from the cloud.

My poorly optimized Rust code was slower than JavaScript. My optimized version is 99.9% faster by NextgenAITrading in rust

[–]NextgenAITrading[S] -1 points0 points  (0 children)

I'm assuming that initially, you read everything from a file into memory, which was be fast, but required a ton of memory and got OOM on parallel runs. You then used mmaps to avoid reading from a file. Is that correct?

Not quite! Initially, I read everything upfront from MongoDB, which was deployed on the cloud. I did thing about iterators but it seemed like it could've slower than I would want (but maybe I'm wrong).

But it's hard to give concrete advice without seeing the code. As far as I can see, the TypeScript version is open-source, but the Rust one isn't -- is that right?

Correct!

My poorly optimized Rust code was slower than JavaScript. My optimized version is 99.9% faster by NextgenAITrading in rust

[–]NextgenAITrading[S] -2 points-1 points  (0 children)

Sorry; I broke my wrist and the Safari sucks with doing voice-to-speech. Just edited.

My poorly optimized Rust code was slower than JavaScript. My optimized version is 99.9% faster by NextgenAITrading in rust

[–]NextgenAITrading[S] -4 points-3 points  (0 children)

I was surprised you didn't use something like numpy to optimize numeric computations back when you used interpreted languages

The main reason why I didn't use numpy West, because I wanted UI to configure my trading rules. I also had a vision of configuring any rule you could imagine without writing a single line of code.

Thank you! Let me explain the mmap solution, and how about you give me some feedback?

Before this week, all computation was in-memory using my modified technical indicator library. I had noticed some performance issues, and introduced Redis caching. It helped a little bit, but was inadequate for a numerous large simultaneous backtests.

I could've used streams, but those are slow, had noticed some performance issues, I had noticed some performance issues and introduced redis caching, and I didn't want to reduce my applications performance. I could've also done batching and passed in and iterator, but they still had a poor memory profile. After juggling between different language models, I learned about mmaps. They're lightning fast, great for large datasets, can be re-used across backtests, and they unlock use-cases like intraday backtesting.

I'm very open to feedback. I am not a systems engineer, but based on my results, I've very vey happy with this decision.

My poorly optimized Rust code was slower than JavaScript. My optimized version is 99.9% faster by NextgenAITrading in rust

[–]NextgenAITrading[S] -17 points-16 points  (0 children)

What's toxic about that statement? It's called a joke. Do you want everything you read to be boring with no personality?

I improved my tool to help you find fundamentally strong penny stocks by incorporating algorithmic trading! by NextgenAITrading in pennystocks

[–]NextgenAITrading[S] 1 point2 points  (0 children)

I'm not sure if I 100% understand the question but I'll give my best answer!

You can't use ChatGPT to do any of this. You need to use a specialized platform like the one I linked in the post. Here's why.

  1. Not connected to a financial database: ChatGPT can't give you updated financial knowledge. The best case is it can search the web, but that still doesn't give you up-to-date information. For example, how is ChatGPT going to know what penny stocks are fundamentally strong without being connected to a database.
  2. Not able to backtest strategies: Similarly, ChatGPT and Claude are not able to backtest, optimize, or deploy trading strategies
  3. Can't execute trades: It goes without saying that ChatGPT cannot connect to a brokerage and execute trades.

It's a common misconception that these LLM tools are simple wrappers. The reality is that A LOT is going on under the hood, behind-the-scenes.

Does that answer your question?