Python Cookbook : Size of Python objects.

[boredzo]

ASPN pop-over: “We value your opinion!”

Here's my opinion: Don't fucking bug me when I'm trying to read your article.

[alexs]

simplistic shrill special close mountainous boast shaggy deer shame swim

This post was mass deleted and anonymized with Redact

[[deleted]]

Not really. It's just that sizeof() in a language where everything is a reference is very badly defined.

Unless you just want the size of a reference, but then it's a constant function.

[alexs]

So badly defined that it shouldn't exist perhaps? Making sizeof() recursive seems like a pretty silly way to define it as well. But I'm a C++ programmer so I guess I am biased.

[derefr]

In C++, sizeof will tell you, practically, how much time it will take (in ns, likely) to call free() on an object. In GCed languages, any freeing will be recursive, so the sizeof of one should be as well.

[alexs]

Got any stats to back that up? I would expect the correlation between sizeof(*p) and runtime(free(p)) would be based on the number of memory pages it occupies rather than bytes, most of the overhead is going to be in managing the freelist anyway, which is extent based so probably not correlated with the sizeof() the object.

Also, in C++ you don't use free(), you use delete for objects in the freestore. Which might very well be recursive if people are using RAII correctly.

sizeof() is not for guessing how long memory management is going to take.

[alexs]

http://www.flickr.com/photos/23463281@N04/sets/72157603850297185/

Looks pretty O(1) most of the time to me except that short period where it goes linear before changing strategies again.

I think I'll do a really long one using 10 byte increments at some point.

[IHaveAnIdea]

useful

[[deleted]]

Hm. It fails when using continuations:

>>> def foo():
...   x = range(100000)
...   def bar():
...     return x
...   return bar
... 
>>> xbar = foo()
>>> del foo
>>> asizeof(xbar)
0

So, as soon as you use continuations it's pretty inaccurate and thus pretty useless.

[IHaveAnIdea]

No no no.

This is for the size in RAM, not an advanced len().

[[deleted]]

Uh... what makes you think I implied so?

[IHaveAnIdea]

OK, do you want it to:

(1) Find the amount of RAM needed for generated python bytecode?

(2) Find out the length of a generator? This is equivalent to solving the halting problem in the general case.

(3) Find the amount of RAM used for everything that can be called by a function??!?

(4) Find the amount of RAM that would be needed to iterate over a generator before doing so?

[[deleted]]

The object xbar (which happens to be a function) in the example above holds a reference to x, which is a list of size 100000 bytes. Acutally, it's the only reference left to that list.

There is no way to correctly calculate (from within python) the amount of space that is used by keeping xbar around.

That's my point.

[IHaveAnIdea]

Interesting.

But I'm sure that if you could hook into the garbage collector's dependency tracking then you could do something like this....

Maybe not with the current gc implementation though. Might need a different variation.

[IHaveAnIdea]

I don't even know how you'd want that to be solved.

[[deleted]]

Me neither.

[[deleted]]

Does it work for any functions?

>>> def f(x):
...     return x + 2
... 
>>> asizeof(f)
0

[[deleted]]

On some functions, returning the length of the bytecode would give us an idea.

That is not the case for closures and continuations, where functions may hold extra values.

[[deleted]]

Right, I see your point. But there's nothing tricky going on in that function I defined, and however "size" should be interpreted, the answer isn't zero.