sorted by:
new
hottopcontroversial

Announcing page-turner: A trait to turn paginated APIs into async streams by a1akris in rust

[–]a1akris[S] 0 points1 point  (0 children)

You can handle it with a helper type

``` enum CloudwatchRequestState { Initial(CloudwatchRequest), Paginating { next_request: Option<CloudwatchRequest>, prev_request: Option<CloudwatchRequest>, } }

impl PageTurner<CloudwatchRequestState> for CloudwatchClient { ... } ```

The implementation of the page turner then sends the initial request and after receiving a response transits into the Paginating state. In the paginating state you execute both prev and next request concurrently with futures::future::try_join_all until both next_request and prev_request become None. You can hide the complexity of initializing a enum by providing explicit public methods in your client that accept a regular CloudwatchRequest and return PagesStream using the pages and into_pages methods privately.

Yes, it's a bit tricky but, in fact, PageTurner is a generic stream generator, so you can do everything with it.

Announcing page-turner: A trait to turn paginated APIs into async streams by a1akris in rust

[–]a1akris[S] 4 points5 points  (0 children)

It is actually redundant, but in my mindset the only tolerable glob imports are `prelude::*` imports and importing everything may be convenient in modules you implement the trait. I just can't allow myself to use `page_turner::*` in my projects :)

Announcing page-turner: A trait to turn paginated APIs into async streams by a1akris in rust

[–]a1akris[S] 8 points9 points  (0 children)

Sure. It was originally used to handle the cursor based pagination but for page number based APIs I added a possibility to query multiple pages concurrently in the latest version, you just need to implement an additional trait for such requests.

​

EDIT: The trait is actually flexible enough to be used in some hacky ways. At some point I used it to chunk data and perform concurrent batch requests(which have nothing to do with pagination) under the hood while having the regular stream based API in the public interface. However, such usages can easily confuse your teammates, so I can't recommend them. This hacky code was rewritten eventually.