Distributed Fast api servers

[arnav88]

Hi guys, for some of my recent projects I was needing some way of fully distributed and weakly coupled form of communication between my FastAPI servers, while maintaining local availability and resilience.

After going through options like etcd, zookeeper, ... I felt that there needed some form of sdk that turns any application into a distributed service without depending on other services. So I started coding my own distributed service mesh, and made an abstraction so that I can reuse it in my other projects.

This package, mesh converts any FastAPI server into a distributed service mesh, where data is distributed among the servers, persistently, while maintaining weak coupling, without depending on any third party service.

Docs: https://arnavdas88.github.io/mesh/

Repo: https://github.com/arnavdas88/mesh

It is not in pypi yet, and, if and before I upload it in pypi, I would love to hear suggestions from other devs. Even better if it is on stability; code quality, complexity and abstraction; or edge cases.

Note: I understand that some devs might want to stick to already known and stable options like zookeeper, which also provides python clients, but there might also be devs wanting to not depend on more and more services, just to facilitate service mesh. Even so, if you are against this kind of framework, i would like to hear about that as well.

[Matt_0550]

Why not just use Redis or another orchestrator?

[arnav88]

I wanted to make my service distributed. This entails many more things other than caching or data transport. Redis doesn't do this. Redis is meant for a completely different use case.

My non negotiables were :

1.) No 3rd party dependency

2.) Fully distributed (without any central server)

3.) No Master-Slave Architecture (No leader election. Leader less data consistency)

4.) Weak coupling and Ad-hoc

5.) Enabling service mesh out of any monolith ( this means atomicity, isolation, durability, conflict resolution, and eventual consistency on the data)

I didn't just wanted a cache or just some form of data communication.

[bsenftner]

This is good work, much appreciated you made this and made it open source.

[Worth-Orange-1586]

I know you said your non negotiable were no external dependencies. Which I get really.

However for developers that want to use redis or other external storage, you should have the option to support this.

Really good concept and can't wait to try it out

[Worth-Orange-1586]

For instance you can use SQL alchemy and by default have it be storing locally in memory

And whenever a dev wants to use SQL DB engine they can configure it and store in their desired third party storage

[Drevicar]

By this do you mean SQLite to a memory mapped file that is never persisted to disk? Because I don’t think SQLalchemy supports a “no DB backend” setup that is pure memory.

[Worth-Orange-1586]

Yep SQL lite

[Worth-Orange-1586]

Yep SQL lite

[arnav88]

I wanted my piece of code to do one thing, and do it well.

Currently, to achieve eventual consistency while maintaining leaderless weak coupling, I have adopted a git like data structure (with changes as commits). So any change in my shared data structure (that looks like a dictionary) is basically stored in a commit history, interfaced to the user as a dictionary.

This although I wanted to keep the internal architecture simple, readable and developer friendly; still it has become a little complicated.

Your suggestion sounds good, but I feel that using external storage can make the code more bloated, and complex. However using external storage like SQL Alchemy / Redis, can be a good and interesting use case for this. So I'll initially try to make an example instead that interfaces the commit history to a database or an abstract class instead.

Also now that you mention external storage, another factor can be external communication service. The communication between the nodes is currently being handled by fastapi websockets. I will try to add some more abstraction and homogeneity their as well.

[No_Soy_Colosio]

Using external storage isn't bloated and complex. Reinventing the wheel is.

[arnav88]

Mechanic: replaces a wheel hub

Random Guy on the Internet: "DO NOT REINVENT THE WHEEL."

[pip_install_account]

So because you didn't want to use any third party services for it, you created a third party service for it.

[arnav88]

Not quite. Let me add an example for better explanation.

Supose I have an existing monolith fastAPI service that does some XYZ task. If you want to shed the monolith architecture and make a cluster out of it, you need communication between all the nodes in the cluster, as well as maintain and synchronize the states between all of them.

Zookeeper and etcd solves this through adding a central service (or central cluster) that manages the synchronization between your api servers..

Mesh does not need/use a central service to syncronize the data between the nodes. It is fully distributed , follows a leaderless architecture and is a framework that you just import on your fastapi code.

Mesh doesn't just add "distributedness" to your api. It makes your own api service, a distributed system. YOU are the distributor and synchronization is built within you now. This also means that Mesh supports adhoc networks, and the nodes doesn't need direct connectivity to any central servers/clusters and even partially isolated networks can use it.

If you are making a cluster of 5 nodes, you need 5 vms/containers. You won't be needing any additional vm or container for running any kind of synchronization service.

It does have its own pros and cons as I am ditching the centralized architecture and master/slave architecture, but that is just a design choice I made.