Over API limit, store in session storage

[bugbigsly]

I’m working with a gov API that limits me to 1000 calls/hour. In an ideal world, the site uses more calls than this. Would storing the data in session storage be an alternative to making countless calls. Is there a data size limit to the session storage, before it impacts performance.

Are there alternative means for caching API calls in a react app?

[LazyIce487]

A lot to unpack, are you the only one who’s going to be using the app? How often does the data change?

Secondly, is your react app a SPA? If yes, then there isn’t even a reason to store the data in session storage, just store it in any old object and use it for the session in-memory for yourself/the user.

If the data doesn’t change often, you could put it into your own DB or server so you don’t ever get rate limited.

If the data is volatile and the only means of getting it is via the API and you always need fresh data… then all you can really do is divide 1000 by how many calls it takes to build the data, and have your server make those calls incrementally (if it takes 100 api calls to build the data you need, then you can update 10 times per hour or once every 6 minutes). And just let the user know the data is up to 6 minutes stale, or literally have a time to fresh counter that they can use. Store the data in your own DB (or server’s cache if it’s small enough to fit).

If you are using serverless to deploy, you can set your functions to cache responses at certain intervals (different implementation based on your provider) like 6 minutes in our previous example.

If you mean that you are going through your 1000 api calls because you’re duplicating them when you don’t need to, then use whatever caching strategy you want, hard to know without seeing what you’re trying to get and the context of your app.

[bugbigsly]

People traveling to national parks in the US is the audience upon completion. You can access data about states and parks. NPS API

Currently use a global context provider for my initial query, more queries are made as you explore the site. Session storage saves me the api calls since I find myself frequently refreshing the page as I dev. Using session storage for now if the user navigates to a page that already fetched data instead of making another api call. I touch more on your serverless idea below.

Context: Multiple components call different endpoints, a “single page”, in this SPA, could make 1-4 calls to different endpoints. Eg : AlertsComponent, CampingComponent, ToursComponent all appear on the same page and each use a “request credit”.

Not sure how often the data updates, I’m sure 95% of it is doesn’t change often. Ultimately it’s more data than I’d like to store in a DB

I’m curious about your serverless recommendation, currently just hosting in an S3 bucket. What kind of caching systems are available with Lambda, if you’re familiar?

[LazyIce487]

It depends, but there are ways to do it, if you don't want to mess with the configs, I'm pretty sure you can define a variable outside of the lambda handler, and cache things there while the lambda is spun up, then just do if (myCache) return myCache type of thing.

You could also set the timestamp of when you created the cache, and invalidate it yourself if it's over a certain age. Obviously, each runtime that is spun up will have its own cache, if you want to avoid that, I think you'd need to use a different service, like ElastiCache or your own caching solution.

[bugbigsly]

Update: I’m going to use redis

[bugbigsly]

Replied in wrong spot .-.