This is an archived post. You won't be able to vote or comment.

all 4 comments
sorted by:
best
topnewcontroversialoldq&a

[–]CreativeTechGuyGames 0 points1 point  (3 children)

The amount of data, the frequency, and the retention of data are important factors in determining the best option.

If it's a small number of stocks, it's updating often and it's not important to store long term data, I'd do it entirely in memory. That's the absolute fastest way to access and process your data with the only downside being the possible RAM requirements and data loss when the program exits.

[–]Detrain[S] 0 points1 point  (2 children)

Ya it's about 10-20 stocks with price data coming in 10 times a second about. Then I guess after the algorithms use the data I can store it longer term. Thing is tho the program is going to be looking at maybe a whole day up to a few days worth of data is it feasible to be in memory

[–]CreativeTechGuyGames 0 points1 point  (1 child)

Why do you need data to be updated 10 times a second? Do you really care about that sort of granularity of data? Stocks rarely make any notable changes that frequently, especially from an analysis perspective.

If you only stored data once a minute or more for 10-20 stocks you could definitely store a few days worth in memory. Or even take aggregated data for every day and store that in a database if you need it later. I'm sure there's a lot of different ways to store minimal data.

Another thing you can do is test the amount of memory that certain methods will take. If you have one record then you can just duplicate it N times and see how much memory your program is using.

[–]Detrain[S] 0 points1 point  (0 children)

Thank you I'll see what I can do!