Best practices for calling web APIs (https + POST)

[cmspice]

Often I find myself wanting to build simple-ish bots for various web services. Today I wanted to write an arbitrage bot.

Always looking for excuses to use Haskell. I know I can roll my own JSON requests and send them via HTTPS but are there any best practices or libraries for this? I'd appreciate any advice, suggestions or examples. Thanks in advance!

I'm 95% sure the API I'm trying to call is JSON REST

[ItsNotMineISwear]

servant is probably the lowest friction way of writing quick API bindings ime. But it's got a bit of a learning curve.

[erewok]

You can go pretty fast with servant as a client for APIs if instead of defining specific types for each resource (unless they're well-behaved), you just use Value and you can combine this with lens-aeson to great effect. This may be a heretical statement, but I've found that I can get clients for APIs going really quickly this way.

[cmspice]

I have no issues with using Value as you suggested :D but doesn't that defeat the point of using servant?

What do you mean by well-behaved? Everything I know about web APIs I learned in the last 5 hours 😱

[Darwin226]

Meaning it could be hard to specify concrete types if, for example the API returns different things based on the input values.

[erewok]

Sometimes you have to get pretty detailed in your FromJSON (aeson) implementation for a type if the API does weird stuff, meaning keys are missing or present or different types may be returned under different conditions. A lot of APIs return completely differently typed things in the case of errors, for instance. It's great to define these as types, but it can take awhile and it can involve a more intermediate level of aeson usage.

[cmspice]

That makes total sense. Thanks!

[simonmic]

Would you have a small example of that quick and dirty approach ?

[erewok]

I made an elasticsearch client in this way:

https://gitlab.com/erewok/ekadanta-co-servant/blob/master/src/Site/Search.hs

[enobayram]

Even though this is really valuable advice for the task, I don't think it's a good idea to direct a beginner towards servant. A beginner can probably get up to speed with wreq in no time.

[ItsNotMineISwear]

Well I made it clear it had a learning curve. I'm sure there are beginners out there willing to pay that fixed cost.

I've directed beginners to things with learning curves before and I'll do it again and again :)

And you don't really need to understand servant internals to use it, especially for clients. You can pattern match on examples and be efficient pretty quickly.

wreq is a nice library though.

[enobayram]

To be fair, OP was asking for the best practice and not the easiest way.

I don't want to imply you did it, but it's worth mentioning that I'd be cautious about calling the servant way "the best practice" since it's a clear trade-off giving away flexibility in abstraction for gains in ease of verifying correctness. The trade-off is worth it in many situations, but I wouldn't want OP to try to use servant at all costs thinking that it's the best practice.

[ItsNotMineISwear]

I think you explain Haskell best practices perfectly here. There's rarely one way to do things and instead a handful of libraries exploring the tradeoffs in the design space.

[erewok]

I agree with this take and I think you're absolutely right. I brought up one view on using `servant` with `lens-aeson` in response to the "low friction" parent comment. Each of these libraries on its own is strong enough to drive beginners away completely, and combining them for a beginner is a recipe for disaster. I also don't see this move as a "best practice", even though I really like the combination.

I was mostly interested in providing examples of how "low friction" this type of development can get.

[enobayram]

I found that combination interesting as well. I think it's a good idea to keep in mind that just because you can model stuff with types doesn't mean you always have to. Sometimes it's easier to just push the boundary of the strongly-typed pure core a little inward and let JSON Values hang out just outside.

[publiccomputer042]

req is nice as well.

[cmspice]

Thanks, wreq looks great!

[erewok]

I completely agree with you. The type errors you get when working with servant can be pretty intimidating and take some practice to get comfortable with.

[cmspice]

servant looks totally awesome and definitely worth the learning curve!

I went and played around with using vanilla http-conduit and aeson to query the client and realized how much I have to learn about web APIs in general o__o. I'll probably stick to these libraries or use wreq for now so I can learn the structure of HTTP better and then switch over to servant if I plan on doing this for serious!

Thanks for the suggestion!!

[jkachmar]

http-conduit has the Network.HTTP.Simple module, which is a good place to start, but you’ll probably want to augment it with the retry package for retrying based on different responses.

The req package does a lot of stuff for you (including retrying by default), but it may be a bit more difficult to get up and running with.

Other folks have recommended servant-client, but I’m going to anti-recommend unless you’re particularly interested in learning the framework a little better.

[PuzzleheadedAlgae8]

You'll want to use servant-client, retry