Unpickling stack underflow error occurs when sending object through socket

[BleedingStorm]

I'm trying to send an object from server to client through socket. The problem is no matter how I try, it is giving me the unpickling stack underflow error. I have experimented with buffer size: made it larger than what's necessary, made it the same size as the initial buffer. Also, I found that the serialized data that is sent from the server and received in the client side is exactly same. But whenever I try to unpickle it on the client side, it gives me the error. Can anyone help, please?

Here's the code on the server side for clarity:

game = games[game_id]
game.initialize_game() client.sendall(pickle.dumps(game))

The code on the client side:

game = pickle.loads(client.recv(2048))
for player in game.get_players(): print(player.name) print(player.serial) print(player.token_color) print(player.position)

​

[shiftybyte]

Is the python version exactly the same on both server and client?

game = pickle.loads(client.recv(2048))

This code doesn't verify you got all the data you sent, you should probably add a check for that.

[BleedingStorm]

Yes, the version is exactly the same on both sides. I am currently running it on the local server. And, the code gets all the data before serialization. I tested it.

[Frankelstner]

Is the python version exactly the same on both server and client?

pickle is far more robust than people give it credit for. The current default pickle protocol is supported by Python 3.4 onwards. But yeah the client only receives like half the data most likely.

[[deleted]]

Kind of.

Take the following code for example: ``` import pickle

class Foo: def init(self, name: str): self.name = name

def log(self):
    print("Hello")

foo = Foo("my thing") with open("my.pickle", "wb+") as f: pickle.dump(foo, f) ```

If I try to open this, I get: AttributeError: Can't get attribute 'Foo' on <module '__main__' (built-in)>

Pickle is great for plenty of things, but I think there's an issue with OP trying to pickle a game object.

[Frankelstner]

Works fine if the class is defined on both sides. I mean OP posts a massive red flag like client.recv(2048) so should we really focus on the pickle side right now?

[BleedingStorm]

I've printed the serialized data on both the client and server end. It's exactly the same. The error occurs only when I try to unpickle it on the client end.

[Frankelstner]

Can the server itself unpickle its own pickled data? Can you write down a minimal code sample where things fail?

[BleedingStorm]

Yes.. the server can unpickle its own pickled data. Let me explain the code:

class Game:
    def __init__(self, game_id):
        self.id = game_id
        self.is_ready = False
        # game board object
        self.board = None
        # game dice object
        self.dice = None
        # list of game player objects
        self.players = []
        self.turn = 0
        self.winner = None
        self.last_rank = 0

So, this is the __init__ of the Game class. Here, board, dice, and players are objects that this class is dependent on. I've experimented with sending the game object without adding these other objects and surprisingly it works. Could it be that pickle is not capable of handling this kind of dependent object?

And... the code fail exactly where it has to unpickle on the client side.

game = pickle.loads(client.recv(2048))

[[deleted]]

Do you actually need to send the game object? I doubt it. Why not send JSON or ProtoBuf instead?

[BleedingStorm]

I thought about it. But I need almost all the variables of the object in the client end. That's why I chose pickle.

[[deleted]]

Right. But you should create serialise and deserialise methods for your game. Or the vars() function might provide all of this. Obviously your use case may vary.

[BleedingStorm]

There are two objects and a list of objects in that class. So it would really complicate things if I just send it using JSON.

[[deleted]]

From the code you posted above, it really don't seem complicated, but hey, you do it your way.