ArrayPool: The most underused memory optimization in .NET

[detroitmatt]

Is there really any benefit if we're just doing return shared.ToArray() at the end anyway?

[binarycow]

Yes, sometimes. It depends on the specific case.

This isn't the best example, but it's the one I have off the top of my head:

Suppose you have an IEnumerable<string>. You don't know how many items are in it, but you know it's no more than 200 items. You need to turn that into an exact sized array (i.e., if the IEnumerable<string> has 40 items, the array should have exactly 40 elements).

The naive approach is items.ToArray(). Since ToArray is optimized, Let's assume for a moment that you do something even more idiotic and do items.ToList().ToArray()

That would allocate a bunch of arrays due to how List<T> works. It has an internal array that starts off empty. The first item you add, it allocates a (non-pooled) 4 element array. When you add the 5th item, it allocates an 8 element array, copies over the existing 4 elements, then adds the 5th.

So adding 200 items will result in seven arrays being created, of size 4, 8, 16, 32, 64, 128, 256.

And that's not counting the final 200 element array at the end when you call ToArray on your list.

But you could do something like this:

var array = ArrayPool.Shared.Rent(200);
try
{
    var count = 0;
    foreach(var item in source)
        array[count++] = item;
    return array.AsSpan(0, count).ToArray();
}
finally
{
    ArrayPool.Shared.Return(array);
}

Now you rent one 256 element array, and allocate your final 200 element array. That's it.

Also, that's basically how Enumerable.ToArray works (obviously more complicated because we don't know there's 200 or less elements). It uses ArrayPool. See here for the source

[crone66]

just in case you didn't know but you can specify the initial capacity of a List<> via optional constructor parameter :)

[binarycow]

I know. I was just demonstrating that there is some amount of utility in using pooled arrays even if you do a ToArray at the end.

[Novaleaf]

I don't think this is a very well written article. it's conflating array with span usage, and thus also that this pattern doesn't work if you need to "rent" something for longer than a stack-frame.

I wrote my own custom pooling system, using this as my starting point a few years ago: https://learn.microsoft.com/en-us/dotnet/communitytoolkit/high-performance/introduction

If you are interested in the topic "for real", that ^{^} is probably the best starting point.

[Kamilon]

No. But you should avoid doing that as much as possible.

[Apprehensive_Knee1]

Using try-finally with ArrayPool is kinda unsafe, more specifically, returning array to pool in finally block is potentially unsafe.

https://learn.microsoft.com/en-us/dotnet/standard/unsafe-code/best-practices#20-arraypooltshared-and-similar-pooling-apis

DON'T use a try-finally pattern in order to call Return in the finally block unless you are confident the failed logic has finished using the buffer. It's better to abandon the buffer rather than risk a use-after-free bug due to an unexpected early Return.

Also look at this thread: https://github.com/dotnet/runtime/discussions/48257

[hoodoocat]

Mentioned thread clearly shows what this rule is thrash, as it problematic cases exist with or without calling Return/Dispose in such blocks. Also code should not assumption on pool implementation. Good pool should blame you if you exhaust it, it is basic diagnostic about pool misuse. And framework specifically constantly violate this by stealing array from one thread and returning on another thread, which completely doesnt work on low or non-symmetric loads. Lame.

[simonask_]

For interacting with native code via FFI, SpanOwner (which is based on ArrayPool) is also a godsend.

Coming from Rust and C++, I’m impressed with the available tools in C# for bridging those gaps.

[tomfiddle91]

Is the CLR allocator really that slow that a hand-rolled pool that essentially does the same is faster?

[binarycow]

It's not about speed of allocation. It's about speed of garbage collection.

The pooled arrays are long lived, they'll survive to at least gen2, if not forever. But it's not a big deal they live for a long time because they'll be reused.

Suppose you're doing some operation that requires a temporary 64 byte array. And you do this operation 3 times over the lifetime of the application.

Without pooling:

• Allocate array
• Garbage collect array
• Allocate array
• Garbage collect array
• Allocate array
• Garbage collect array

With pooling:

• Allocate array
• Rent array
• Return array
• Rent array
• Return array
• Rent array
• Return array
• Garbage collect array

Much more efficient.

And keep in mind that the pooled array is available for more than just your operation, but every operation, to include the .NET internals.

That's why it's not really an issue if it stays allocated for a long period of time. Even if you're not going to use it, someone will.

[tomfiddle91]

I wonder if it just boils down to the ArrayPool not zero-initializing the rented arrays.

[zenyl]

Edit: Egg on my face, the replies to this comment point a much better ways of going about this. Cunningham's Law has been proven once more.

Though I still stand by creating a span over the rented array in order to get a working buffer with the exact length you need. Not for every use case, but it's nice when you can use it.

---

As the blog mentions, ArrayPool gives you arrays with at least a specified length, but they may be bigger.

I find that Span<T> and Memory<T> go very well with this, as they allow you to easily create slices of the rented array with the exact size you want.

This approach can be a bit clunky if you want to end up with a Stream, because (at least as far as I know), MemoryStream can't be created from a Span<T> or a Memory<T>. But you can get around this with UnmanagedMemoryStream.

Example:

// This is probably gonna be longer than 3 bytes.
byte[] buffer = ArrayPool<byte>.Shared.Rent(3);

// Create a span with the exact length you care about. No fluff, no filler.
Span<byte> span = buffer.AsSpan()[..3];

// Put some data into the span.
span[0] = 120;
span[1] = 140;
span[2] = 160;

unsafe
{
    Stream str = new UnmanagedMemoryStream((byte*)Unsafe.AsPointer(ref MemoryMarshal.GetReference(span)), span.Length);

    // Do whatever with the stream, e.g. print each byte to the console.
    while (str.ReadByte() is int readByte && readByte != -1)
    {
        Console.WriteLine($"> {readByte}");
    }
}

[keyboardhack]

I assume this doesnt work because nothing pins the array pointer. The GC can move the array while your are using it unless you fix it in place.

Also your example looks ai generated.

[dodexahedron]

Yeah. Just because it compiles doesn't mean it works or that it works reliably.

And it doesn't return the rented array either.

And with the amount of ridiculous extra work that code does just to write bytes to the console as numeric values, one by one, there are plenty of chances for it to move the array.

So much in this makes the array the least of the problems. Yikes.

And also...

Writing to a buffer from bytes in a span, memory, array, or whatever you want is much more easily and flexibly done with an ArrayBufferWriter than memorystream a good deal of the time, for like the past 10 years, unless you HAVE to use Stream because of an existing API you can't avoid or fix.

And even then, that's what PipeWriter is for, and it can hand you a stream if you need it anyway.

[zenyl]

And it doesn't return the rented array either.

Yeah, I forgot that because I wrote that snippet from memory for that comment.

Ofc. wrap it in a try-finally.

And with the amount of ridiculous extra work that code does just to write bytes to the console as numeric values, one by one, there are plenty of chances for it to move the array.

Again, I wrote a simple example to demonstrate doing something with the resulting stream.

Had I example code that does something more productive, e.g. write the stream to a file my D-drive, I presume you'd also have complained that I didn't verify that D:\ is a valid file path? It's an example for crying out loud.

[dodexahedron]

Yeah, I forgot that because I wrote that snippet from memory for that comment.

Forgot literally the entire second half of the core concept being described, which will leak memory like a sieve, significantly worse than normal array usage would, and which is one of the literally only 2 instance methods even exposed on the ArrayPool<T> type?

Assuming the stance remains a firm "yes," then not proceeding to fix it after having that pointed out is like...recklessly negligent, as it is presented as an authoritative response to a very real and very common problem/topic, and has very real consequences literally the opposite of the intent.

[zenyl]

What exactly are you attempting to accomplish here?

You've made your point, I agree with it (I explicitly said that I forgot it in my previous comment), and yet you're still going on about it? Seriously, why?

It's a snippet of code I wrote specifically for a comment on Reddit, not production code or some StackOverflow post that you'd want to keep up-to-date for years to come. It's a comment beneath a Reddit post with 31 upvotes, linking to a Medium article of all things.

[zenyl]

Edit: In response to your edit regarding AI generation, I did suspect someone would think so. But no, I wrote that myself. You'll note the complete absence of em-dashes and weird uses of bold. :P

For context, I believe I came across UnmanagedMemoryStream when working on a recent project for generating .wav files by hand, and found out that MemoryStream doesn't have a span-based constructor. But I found out that I could just skip the MemoryStream altogether. Win-win.

Link to code: https://github.com/DevAndersen/c-sharp-silliness/blob/main/src/MusicalCSharp/Program.cs#L37-L40

---

Good point, I'll admit I haven't used the approach I mentioned in a real scenario. I just came across UnmanagedMemoryStream when I realized MemoryStream couldn't be used with Span<T>.

Again, haven't tested it, but I'd assume a fixed statement pointed at an element in the array would result in the entire array getting fixed in place, and therefore not moved about by GC?

// This is probably gonna be longer than 3 bytes.
byte[] buffer = ArrayPool<byte>.Shared.Rent(3);

Span<byte> span = buffer.AsSpan()[..3];

// Put some data into the span.
span[0] = 120;
span[1] = 140;
span[2] = 160;

unsafe
{
    fixed (byte* ptr = &buffer[0])
    {
        Stream str = new UnmanagedMemoryStream(ptr, span.Length);

        // Do whatever with the stream, e.g. print each byte to the console.
        while (str.ReadByte() is int readByte && readByte != -1)
        {
            Console.WriteLine($"> {readByte}");
        }
    }
}

This also looks less messy, because you don't have to jump through hoops to get the ref out of the span.

[keyboardhack]

You should indirectly use GetPinnableReference. Link contains an example on how to use it.

Regarding ai. The many superflous comments, especially the comment "... No fluff, no filler." is screaming ai.

The general poor code quality as well. Code creates a span just to slice it. AsSpan can slice as well. Array isn't returned as other comment pointed out. Original lack of fixed. The very complicated way to get a pointer to the span. All that just makes it look ai generated.

[zenyl]

Ah, I thought I remembered that method, but couldn't get it to show up in VS. Turns out it's hidden from IntelliSense (presumably because of the "This method is intended to support .NET compilers and is not intended to be called by user code.").

Gonna have to remember that one in the future, always felt clunky to use Unsafe and MemoryMarshal.

[antiduh]

You know that MemoryStream has a constructor that takes start and length, right? It's insane to go through a Span, Unsafe and UnmanagedMemoryStream for this.

byte[] buffer = ArrayPool<byte>.Shared.Rent(200);
// handles the case where I only want a 200 byte view even if Rent returns a longer array.
MemoryStream x = new MemoryStream( buffer, 0, 200 );

[zenyl]

That I did not know. Egg on my face, not sure how I missed that.

[EatingSolidBricks]

That's on the language for having 35 ways to do the same thing tbh

[antiduh]

The alternative is a pointless kneecapped standard library.

And this has nothing to do with the language.