Roughly estimate the total memory usage of an object

Environment: Python 3.5.0, 64bit

The memory usage of an object can be checked with sys.getsizeof, but it does not count up to the referenced object.

>>> import sys
>>> sys.getsizeof([1, 2 ,3])
88
>>> sys.getsizeof([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
88

If you want to know the approximate total memory usage including the elements in the container, it seems that you need to trace all the referenced objects and integrate the getsizeof value.

reference: http://code.activestate.com/recipes/577504/

total_size.py



import sys

from itertools import chain
from collections import deque


def total_size(obj, verbose=False):
    seen = set()

    def sizeof(o):
        if id(o) in seen:
            return 0
        seen.add(id(o))
        s = sys.getsizeof(o, default=0)
        if verbose:
            print(s, type(o), repr(o))
        if isinstance(o, (tuple, list, set, frozenset, deque)):
            s += sum(map(sizeof, iter(o)))
        elif isinstance(o, dict):
            s += sum(map(sizeof, chain.from_iterable(o.items())))
        elif "__dict__" in dir(o):  #There must be a better way
            s += sum(map(sizeof, chain.from_iterable(o.__dict__.items())))
        return s

    return sizeof(obj)

There seems to be another way to use pickle.

Recommended Posts

Roughly estimate the total memory usage of an object
Get the attributes of an object
Create a shape on the trajectory of an object
Get the id of a GPU with low memory usage
An introduction to object orientation-let's change the internal state of an object
How to know the internal structure of an object in Python
Touch the object of the neural network
The meaning of ".object" in Django
Memory usage of GC target object may increase by 8 bytes from Python 3.7.4
Python Note: When you want to know the attributes of an object
The story of making Python an exe
Scraping the usage history of the community cycle
[Python3] Rewrite the code object of the function
The story of making an immutable mold
The usage of TensorBoard has changed slightly
Reading comprehension of "The Go Memory Model"
I want to store the result of% time, %% time, etc. in an object (variable)
Calculate the memory sharing rate of Linux processes
Calculate the total number of combinations with python
Visualized the usage status of the sink in the company
Organize the super-basic usage of Autotools and pkg-config
[Blender] How to get the selection order of vertices, edges and faces of an object
Command to check the total number of CPU physical cores / logical cores / physical memory on Mac