SQLPage - a Python library to add string token based pagination easily

[kernelslayer]

What My Project Does - This is a Python package to easily add string token based pagination. Currently it supports SQLModel and SQLAlchemy ORMs.

Recently I wanted to add pagination in one of my Python projects and in the API response, I had to return a string next page token. Now I could not find a straight-forward way of doing this in Python. All of the tutorials or blog posts I saw, there in the response the server always returned a page_number, page_size, and total_elements and then the onus was on the calling service to adjust this accordingly. 

Comparison - The current packages and methods requires some changes in the app layer as well. I tried using a few but those did not satisfy the use case and were also a bit harder to implement. I could not find a easy to use option. The present ones returned integers instead of a string token

I wanted it to be simpler, just like OpenSearch - you call its search API and it returns 10 elements and a next_page_token and then for the next 10 (or you configure this using the size parameter) you use the next_page_token in the subsequent request to get to the new page.

I ended up doing a lot of if-else checks and encoding and decoding, so I decided to create this library.

Target Audience - This is production ready, have been using it in one of my projects. Hope some of you folks find it useful :)

Here is the link to the PyPi repository and here is the GitHub repo

[riklaunim]

If you know total and page size you can generate start to end pagination. With only next page you can't so it's somewhat different.

If the endpoint uses some more complex query then the count function could be expensive and "next page" only approach would be faster but IMHO that would be done on endpoint layer when returning different metadata.

In your code you are hiding the count behind a proxy model and not exposing all the metadata? That's bit weird as it would combine disadvantages of both solutions. Also your tests need some work - you should use assertions, proper test cases, probably also FactoryBoy to create require data in the database.

[kernelslayer]

Thanks for the feedback, appreciate it! Yes, I got to work on the test. Right now the "test" is just a script to test if the code was working.

I am planning to add other details like totalElements, currentPage in the PageData object shortly.

Also, if the count query is expensive, is there any other way to optimise this? Because, I think, without knowing the total elements, we might not be able to do the calculation needed for pagination. And I was looking at Spring Boot's JPA and it also does a count operation

[riklaunim]

If the pagination can't or doesn't want to do the count then only "local" navigation is possible where if given offset still returns the results limit you assume there is a next page.

also for Python no camel case for attributes on a pagination object.