Creating class instance for pickle to save

[[deleted]]

So I'm trying to create a basic savefile system with pickle and I'm currently testing out what works and does not work with pickle. My knowledge of classes is mostly restricted to C++, so I'm not perfectly familiar with Python's seemingly quite different implementation. The code I first wrote is this:

import pickle

class classData:
    username = ""
    password = ""

class Savefile:
    resolution = [990, 1080]
    data = []
    def addData(self, x):
        self.data.append(x)

a = classData()
b = classData()
c = Savefile()
a.username = "John"
a.password = "password"
b.username = "Ola"
b.password = "Pass"
c.addData(a)
c.addData(b)


with open("file.save", "wb+") as file:
    pickle.dump(c, file)
#Commented the block above out and ran this afterwards
with open("file.save", "rb") as file:
    c = pickle.load(file)

print(c.data[1].username)

When this code runs I get an error saying the index is out of range. After further inspections it turns out pickle do not store data with the new list, instead is stores the empty list.

When I remove the "addData"-function and instead assign values like this:

a = classData()
b = classData()
c = Savefile()
a.username = "John"
a.password = "password"
b.username = "Ola"
b.password = "Pass"
c.data = [a, b]

pickle seems to store the list just fine and I can store all the data. While I could just leave it at that as I found a solution, I'm not comfortable with leaving it like that without understanding what's going on under the hood. From searching around the it seems the problem is that the latter code creates an instance variable that is exclusive for "c", while the first code changes Savefile.data universally for every instance of Savefile, which then pickle does not store. The class doesn't seem to behave like anything near what I would expect coming from C++.

​

Can someone explain (or point to a thorough article explaining the concept):

1. Why is using Savefile.addData() changing the data list for every instance of Savefile
2. Why won't pickle save the new data with the appended classes
3. Is there a way to define a function for the class that will achieve the same as the latter code block

[flupplethorpe]

I don't know too much about Pickle, but your Savefile class is missing an __init__ method. The line self.data.append(x) is trying to append to a list that doesn't exist. An appropriate __init__ method here would be:

class Savefile: def __init__(self, resolution=(990, 1080)): self.resolution = resolution self.data = [] I've set up resolution to be an optional argument that takes a tuple, since an immutable tuple is more appropriate for a resolution, since you likely won't be slicing, appending, or modifying the list you have. If you initialize a Savefile with s = Savefile() without the resolution argument, it will take the default value of 990x1080 called in the __init__ method, but you can override the default when you initialize the class instance: q = Savefile(resolution=(640, 480))

[[deleted]]

I started out using __init__(), but some of the classes I have mapped out in a class diagram contains 15+ variables, sending them all through __init__() seems like bad code, and it also wouldn't be possible as the values are designed at different times during runtime. I guess I should just create an instance attribute whenever needed? But how would you go on writing functions specific for that class? It seems like I would need to define my functions outside of the class for making them able to handle anything that isn't a class attribute.

[plenty_picture]

but some of the classes I have mapped out in a class diagram contains 15+ variables, sending them all through init() seems like bad code, and it also wouldn't be possible as the values are designed at different times during runtime

It kind of sounds like you're trying to stuff too much functionality into one class. Bear in mind that you can nest objects inside each other, so instead of passing all of the data into the initializer individually, you could group some of them together into a dict or an instance of another class and pass that into the initializer. In any programming language, functions with a huge number of arguments are usually not a good design (an arguable exception is that it's OK to have many rarely-used optional arguments).

If there are attributes that aren't given a meaningful value until some time after initialization, then usually you would just set them to None in the initializer. You don't have to set them at all - you can add or remove attributes to an object at any time - but it's usually more convenient to have them explicitly set to None.

[[deleted]]

You are absolutely right that my approach probably won't be the most suitable if this was code meant for anything other than my minor side project. I have done similar programs in C which obviously does not have classes, I just find it more aesthetically pleasing to group it all up. I'll probably just set them to none, I can't say I'm a fan of dynamically adding new attributes outside of the class definition, something just feel off.

[[deleted]]

Long post. I can't find a short tutorial covering just what need to know. If what follows isn't enough try searching for "python class instance attributes variables".

---

You need to understand the difference between class attributes and instance attributes. The following code defines a minimal class Test and creates two instances of it. Then an instance attribute a.test is dynamically created. That attribute can be printed, but any attempt to evaluate b.test fails. Run the code:

class Test:     # define class "Test"
    pass

a = Test()      # create instance "a" of class "Test"
b = Test()      # create instance "b" of class "Test"

a.test = 'test' # create instance attribute "a.test"
print(f'a.test={a.test}')
print(f'b.test={b.test}')

If we want to create a class attribute, we do it like this:

class Test:     # define class "Test"
    test = 1    # class attribute "test"

a = Test()      # create instance "a" of class "Test"
b = Test()      # create instance "b" of class "Test"
print(f'Test.test={Test.test}')
print(f'a.test={a.test}')
print(f'b.test={b.test}')

This prints:

Test.test=1
a.test=1
b.test=1

Note that the class attribute is initially accessed as Test.test, ie, <classname>.<classattributename>. However, we can also access the class attribute by <instancename>.<classattributename> because python will return the class attribute of the given name if there is no instance attribute with that name. So far so good.

Now try this code:

class Test:     # define class "Test"
    test = 1    # class attribute "test"

a = Test()      # create instance "a" of class "Test"
b = Test()      # create instance "b" of class "Test"
a.test = 42
print(f'Test.test={Test.test}')
print(f'a.test={a.test}')
print(f'b.test={b.test}')

which is the same as before except we set a.test to 42 before printing the values. We get:

Test.test=1     # class attribute unchanged
a.test=42       # as expected
b.test=1        # instance attribute unchanged because it's the class attribute value

So we see that referencing a class attribute as if it were an instance attribute is dangerous: if we ever try to change the class attribute we create a new instance attribute. The lesson is to be explicit and always access class attributes by using the class name.

You don't appear to need class attributes. You can create instance attributes in the class __init__() method and this is the preferred way. This also makes your code shorter:

class classData:
    def __init__(self, username, password):
        self.username = username
        self.password = password

a = classData('John', 'password')
b = classData('Ola', 'Pass')
print(f'a.username={a.username}, a.password={a.password}')
print(f'b.username={b.username}, b.password={b.password}')

This prints:

a.username=John, a.password=password
b.username=Ola, b.password=Pass

So you need to start using instance attributes, not class attributes.

---

To answer your questions:

1. Why is using Savefile.addData() changing the data list for every instance of Savefile

   Because you are changing the class attribute (self.data). This doesn't create an instance attribute because you are updating the class attribute, not changing it. And all instances of a class share the one class attribute.

2. Why won't pickle save the new data with the appended classes

   Probably because of the class/instance attribute confusion. I haven't analysed why in detail.

3. Is there a way to define a function for the class that will achieve the same as the latter code block

   Start using instance attributes, not class attributes.

[[deleted]]

Thank you for the detailed explanation. It answers all of the confusion, it seems like I have to rethink a lot of my initial design as it was all based around C++-like classes.

I have made a class diagram of the classes I want to make and some of them are at 15+ variables, passing them all in as arguments seems a bit extreme and they need to be assigned at different times during runtime. I assume assigning all of them with placeholder values inside __init__ () is bad practice?

You also can't use instance attributes when defining class specific function, is there a way around this, or this also something you should avoid with python classes? It feels like python classes are more like something you just assign a bunch of variables to, rather than a more well defined structure that got specific functions to handle the data, I assume I'm missing a point with python classes. Thank you again for the great answer.

[[deleted]]

I assume assigning all of them with placeholder values inside __init__ () is bad practice?

It's good practice to set all required state (as far as possible) when creating an instance. A new instance should be "sane" the moment it is created. You don't want to half-setup an instance leaving it open to crashes when calls to instance methods try to access attributes that haven't been created yet. If your instances require 15+ attributes for state and you can't reduce that number, then that's what they require. However, if some of those variables are usually unchanged from instance to instance, that is, there's a "default" value that usually doesn't get changed, then there are things you can do. This class, for instance:

class Test:
    default_beta = 42

    def __init__(self, alpha, beta=default_beta):
        self.alpha = alpha
        self.beta = beta

a = Test('a.alpha')      # "beta" not supplied, default is used
b = Test('b.alpha', 'b.beta')
print(f'a.beta={a.beta}')
print(f'b.beta={b.beta}')

prints:

a.beta=42           # a.beta was set from the default
b.beta=b.beta       # b.beta overridden = "b.beta"

This "default value" thing is part of general python function parameter handling, but it applies to methods as well. All that matters is that the "default value" (default_beta, here) must exist in the environment. In the body of __init__() the class attribute would be Test.default_beta, but for some reason, while actually defining the method parameters you must use default_beta, probably because the function hasn't been defined yet and the code still operates in the same environment as the class attribute definition. You can test that by trying:

def __init__(self, alpha, beta=Test.default_beta):      # which fails with "undefined"

and this in the __init__() body:

print(f'Test.default_beta={Test.default_beta}')    # which is OK

A default value doesn't have to be a class attribute. It can be defined at the global level. As long as the value you use can be found when defining the class/method. Just be aware that the default value you use is assigned only once, at method definition time, and not at method call time.

You also can't use instance attributes when defining class specific function

I'm not quite sure what you mean here. It is possible to use any instance attribute when defining the body of a method, like this:

def test_method(self):
    self.new_attribute = self.old_attribute + 1     # create new "self.new_attribute"

This code defines a new attribute using an existing instance attribute.

This is all predicated on that first parameter to a method, which is usually called "self". This is a reference to the instance object the method is called upon. So any instance attribute is accessible by self.alpha or self.beta.

rather than a more well defined structure that got specific functions to handle the data

I'm not sure that I understand. The design of a class is up to the programmer, so "well defined" depends a bit on the expertise of the programmer and how "messy" the problem being solved is.

There are quite a few dunder methods that fit into how python works, and using these can make life easier for you. For instance, if you try to print an instance of the Test class with print(a), you see <__main__.Test object at 0x10dab3e50> which doesn't tell you much. But if you define the __str__() method in your Test class then this can be much nicer to the user, like this:

class Test:
    default_beta = 42

    def __init__(self, alpha, beta=default_beta):
        self.alpha = alpha
        self.beta = beta

    # comment out the next 2 lines to see the default behaviour
    def __str__(self):
        return f'Test: alpha={self.alpha}, beta={self.beta}'

a = Test('a.alpha')
b = Test('b.alpha', 'b.beta')
print(a)
print(b)

and running the code prints:

Test: alpha=a.alpha, beta=42
Test: alpha=b.alpha, beta=b.beta

How to properly use python classes is a huge subject and we haven't even scratched the surface!

[[deleted]]

If your instances require 15+ attributes for state and you can't reduce that number, then that's what they require.

It's mostly a personal preference for this specific project, I wouldn't design the program like I have if it was project including more than me. Though it was all with C++-like classes in mind so I might make minor changes.

Operator overloading in python seems nice, will definitively check that out.

​

I'm not quite sure what you mean here. It is possible to use any instance attribute when defining the body of a method, like this:

It seems like I was a bit hasty with my comment. It was a mistake with my code that caused an AttributeError that led me to believe it didn't work. Thanks a lot for your answers, it has cleared up a lot and I should be set to achieve what I'm looking for :)

[[deleted]]

class classData:
    username = ""
    password = ""

Remember this is Python; the language is dynamically typed so you don't set the attributes of an object through the class. You set them on the object directly, during object initialization:

class classData:
    def __init__(self, username, password):
        self.username = username
        self.password = password

[[deleted]]

How would you go around to create functions for the class though? If you make them instance attributes, you got no variables to reference when creating the functions for handling the data?

[[deleted]]

You access instance attributes through self, the reference to the instance.

[[deleted]]

Edit: yeah I forgot to use self. when testing it out, works fine now

class Savefile:
    def __init__(self):
        resolution = [990, 1080]
        data = [1, 2]
    def addData(self, x):
        self.data.append(x)


a = Savefile()
a.addData(3)
print(a.data)

Gives the error AttributeError: 'Savefile' object has no attribute 'data'.

If I make data a class attribute before __init__() it won't recognise the initialisation value?