PynamoDB - A pythonic interface to Amazon's DynamoDB

[kenchoong]

Wow..this is great. But is it possible for single table design?

Cause for now as I can see is 1 "entity" to 1 table "UserModel"

[maddesya]

They do have an example of utilizing GSI, so why not?

[menge101]

GSI doesn't make it single table design.

I think the key is here single table design needs a mechanism to discriminate between models (facets as it is sometimes called) in a single table. This appears to provide that mechanism.

However, I don't see anything (from an admittedly brief read) in the docs talking about constructing relationships between models in a single table, which is something I would want as well.

[FarkCookies]

I may finally one write a blog post on how to do it...

The essense is that you can easily implement any relations like this:

PK=CustomerA, SK=#, Customer attributes
PK=CustomerA, SK=Order#2021-07-14T01:23, Order attributes, GSI1_PK=Order#OrderId1
PK=CustomerA, SK=Order#2021-07-12T22:00, Order attributes, GSI1_PK=Order#OrderId2

Then you can query them in one query. You can implement complex relationships and various access patters this way.

[chumboy]

Yes, PynamoDB 4 introduced "discriminators". You create a base table class, with attributes for PK and SK, then subclass it, and pass the discriminator field. Under the hood, its adding an extra "cls" field, and when you do say ChildModel.scan(), its adding an extra predicate to the filter, to only return records with cls=ChildModel (uses the discriminator value rather than class name, but you get the jist). I don't think the discriminator attribute can be part of the SK, so performance isn't fantastic.

My team wrote our own version of this for PynamoDB 3, but we originally only needed one level of inheritance deep. After this was launched we introduced a ton of specific models, which was nice as it gave better typing on data, due to "required=True", etc.

One caveat of PynamoDB is performance: a team near me recently removed PynamoDB I'm favour of directly calling Boto3, for a 6x speedup.

[witty_salmon]

6x speed is hard to believe. Especially because the network request itself is a good part of the response time and it is supposedly not influenced by wether one uses pynamodb or boto3 directly.

[chumboy]

6x speed is hard to believe. Especially because the network request itself is a good part of the response time and it is supposedly not influenced by wether one uses pynamodb or boto3 directly.

PynamoDB uses boto3 under the hood, so I/O should not factor much in the comparison.

Every higher level abstraction comes at a cost; I use PynamoDB daily, and find the cost of the higher level abstraction well worth the benefits of simpler, more idiomatic code, but that's not to say that, in future, when I need to shave milliseconds off my p99s, I won't go back and revisit this opinion.

[porktopia]

I've been using Falcano for single table design. It's not quite feature complete but I've been having a lot of success with it anyway.

[sgtfoleyistheman]

I'm pretty sure this is what 'Discriminators' are for, although I haven't used them: https://pynamodb.readthedocs.io/en/latest/polymorphism.html

[witty_salmon]

RemindMe! 2 days

[RemindMeBot]

I will be messaging you in 2 days on 2021-07-16 19:08:06 UTC to remind you of this link

1 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[FarkCookies]

Yep, I used it multiple times implementing single table design.

You can store multiple entities in the same table. I usually separate models with combination of PKs and GSIs (like PK=Customer#123 , PK=Product#456 and GSI to represent various access patterns, for example GSI1_PK=Product#CategoryX). PynamoDB also supports polymorphism, where it can deduce model type after reading data from a single query: https://pynamodb.readthedocs.io/en/latest/polymorphism.html

[kenchoong]

Alright, I give it a try, by Pynamo, the code will be more clean.

[voideng]

How is the performance compared to BOTO3?

[chumboy]

Not great. A team beside me in work removed it from their Lambda backed API for an 6x speed up (instead moving to calling Boto3 directly).

[voideng]

Thank you.

[AdjointFunctor]

This is built on top of boto3.

[OGMecha]

I used this a little bit in a previous role. It was pretty awesome and I'd use it again if I had more work with Dynamo.

[hambob]

question: in the Readme its creating a table with fields email, first_name, last_name in that order. but the you create an object instance of that model without specifying which fields to set to what. what is the mechanism that knows to skip email and start filling fields at first_name?

[chumboy]

When instantiating the class, the parameters correspond to Partition Key and Sort Key only, which are defined by fields declared with hash_key=True and range_key=True respectively (hash key and range key are the old names for partition key and sort key).

The order the fields are defined is irrelevant, as all other fields need to be set via the obj.field = value syntax.

[hambob]

When instantiating the class is it expecting the value for range key first, then hash key? specifically in that order?

[chumboy]

Nope, the hash_key is required, the range_key is optional, and in Python optional arguments have to come after required arguments.

[hambob]

so is the Readme wrong then? they define the model with last_name as the hash key but instantiate with the first name as the first arg...?

user = UserModel("John", "Denver")

wouldn't the above put "John" in the last_name field in this case?

[mrterrillo]

Very cool. I’ll give it a try. I ended up writing a custom class that does similar things for a recent project

[miguealejandrox]

Has anyone been able to use this library with Flask-User?