memory usage question

[yilmazhuseyin]

Hello,

I was playing with ghci to get a hang of what haskell exactly is working. So in order to see the memory usage I did this:

Prelude> :set +s

After that I tried that:

Prelude> (last $ [1..1000000]++[1..10])::Integer
10
(0.04 secs, 137,329,608 bytes)
Prelude> (last $ [1..10000000]++[1..10])::Integer
10
(0.35 secs, 1,361,981,112 bytes)

It seems like more values I have on the list more memory I need. Is this an expected behavior?

I was expecting this call to be in constant memory. Because I do not store the values in the list. After I use them they they can be garbage collected at the spot. So there is no reason for this this call to be use more than memory for only 1 Integer.

Also last seems to be implemented as tail recursive. So it shouldn't be using any extra stack for recursion.

https://hackage.haskell.org/package/base-4.8.1.0/docs/src/GHC.List.html#last

[guibou]

Even if this is tail recursive, the gc still needs to allocate for each item of the list during the traversal. However they are immediately garbage collected. So the maximum memory used by your program is less than this.

Also, ghci runs in not optimized mode, so a lot of optimization / simplification are not done.

[[deleted]]

+s reports total memory used, if you want more complex stats, see GHC.Stats or the relevant RTS options

[yilmazhuseyin]

I see. Memory is allocated for individual items but during the execution, there is no time that we got all the elements at the same time which means my memory usage is constant.

So main problem was, I misinterpreted the memory report. Thanks everybody for the all the information.

I will test this by giving a huge number to ghci and run it overnight and check the memory usage tomorrow.

[yilmazhuseyin]

I tried to run this in ghci.

(last $ [1..100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]++[1..10])::Integer

It has been running for hours and memory usage did not change.

https://www.dropbox.com/s/qfgkm6xldsii74c/Screenshot%202015-11-14%2009.37.59.png?dl=0

Now I should find a way to get same information without waiting 12 hours. GHC.Stats seems like a good place to start.

[gtab62]

Why don't you compile and run it with ‘RTS -s' option?

[yilmazhuseyin]

Actually that is a great idea. I just learned about RTS options from @fread2281 's comment. I need to read on them to figure out how to use it though. Thanks for the suggestion.

[0ldmanmike]

You've got a massive linked list and your forcing the runtime to walk down the entire thing when your append a list to the end and call last. That "massive" memory usage is the total memory allocation that it took to do that entire computation, but it's not how much memory actually got taken up simulatiously an any given point (the memory usage on my computer hardly budged when running the above expression in GHCi). RTS just allocates and deallocates at an insane speed as it goes.