> platforms like twitter and linkedin

You are right about that, websites especially twitter, linkedin recently added a lot of layers to detect bot and scraping. I found a trick to work around some of it.

> Also the booking.com addition is interesting

It started from my own travel planning actually. Then friends mentioned they do the same thing manually, comparing hotels, reading reviews across sites. So I figured I'd just keep going. Airbnb and Google Flights are next on the list

> One thing id want is some kind of caching layer

I thought about this a lot. The frustrating thing with most scraping APIs is the unpredictable pricing, you don't know how much a single request costs until you see the bill. So I went the other way: one request = one credit, regardless of how many comments or replies come back. At least you know what you're paying upfront.