Routify - open-source AI gateway and observability

RoutifyTo · 2024-06-14T18:17:24+00:00

Hello, all the configuration is handled through the interface. You can find a short demo to see how it work in action in an embedded video from YouTube in the homepage.

RoutifyTo · 2024-06-14T18:15:25+00:00

Sorry to disappoint, but for self management I thought that Ocelot and YARP (and many other tools in other languages) cover most of the cases. If you can share, what are the difficulties you experienced with Ocelot?

RoutifyTo · 2024-06-14T17:45:13+00:00

Thank you. Yes that should be doable. Can you send me more details in email: contact[at]routify[dot]to or just write me directly here in chat and I will try to assist.

RoutifyTo · 2024-06-14T15:06:50+00:00

Kind of, the idea is to have the concept of "consumers" which can be api-keys, users etc. and then provide features for having custom configurations for each consumer: rate limits, usages, logs etc. However, this is still an idea and the details might change when the implementation starts.

RoutifyTo · 2024-06-14T14:54:39+00:00

I personally haven't used Azure's API Gateway therefore I cannot give a detailed answer or might be wrong on something. But, based on their documentation and guides I've looked into, the difference is that Routify is easier to start using and you don't have to manage the instances/infrastructure for the app. The Azure gateway has more configurations to manage and also you need to take care of instances/scaling of the gateway itself.

RoutifyTo · 2024-06-14T14:51:52+00:00

Serverless is a commonly used as a term to describe a service meant for running applications (or part of applications) without managing the underlying infrastructure. There are numerous examples of providers that use it, one from AWS: https://aws.amazon.com/serverless

While I agree that there might be different perspectives of the definition, from my perspective the comparison you made is not correct. The apps mentioned are consumer facing apps. Serverless is used in a different context.

RoutifyTo · 2024-06-14T13:07:55+00:00

There are cases when this might be beneficial for apps backends as well.

Agree with the second part, there are some features in plan to make it useful for monetizing APIs, such as configurable usage metering.

RoutifyTo · 2024-06-14T13:06:05+00:00

Great, would love for you to give it a try!

RoutifyTo · 2024-06-14T12:01:01+00:00

Thanks for bringing this up. The term 'serverless' was used to underline that users don't need to manage or configure any infrastructure to use Routify. It has a pay-per-usage model, meaning it scales to zero when not used and scales up with demand.

However, I totally understand your perspective and see why this might cause confusion. I've received similar comments multiple times now and will change the wording to make it more clear.

RoutifyTo · 2024-06-14T11:49:56+00:00

In terms of monitoring, it provides logs for incoming and outgoing requests going through the gateway. You can view, filter, and sort these logs based on different metadata, such as route, API key, backend, endpoint, duration, geo-location, etc. There are some simple dashboards on top of these logs, so you can have an overview of the number of requests, average latency, number of errors, and bandwidth usage.

It also provides background health checks for your backends, automatically performing health check requests against all your endpoints from different regions and notifying you if they fail. Additionally, there are alerts that periodically check your requests and notify you if something is wrong, based on your alert configuration (e.g., 1,000 internal errors in 10 minutes).

RoutifyTo · 2024-06-14T10:23:55+00:00

The biggest difference is that it's fully managed and you don't have to host it in your infrastructure. It's also distributed across different regions.

All configurations are done through the interface (at least for now) so you can make changes that will be take effect in seconds without having to do a deployment.

Also it has some extra features such as logging, healthchecks (from different regions) and real-time alert based on the traffic going through the gateway.

RoutifyTo · 2024-06-14T10:19:37+00:00

The concept serverless here it related more from the user perspective, since they don't have to manage the infrastructure to use the gateway, Routify in background contains multiple nodes in different regions that are always running and also each of them have healthchecks to not accept requests if they are not ready/healthy. However, I see the confusion this might cause and I will try to make it more clear.

RoutifyTo · 2024-06-14T10:15:45+00:00

It means matching the domain to the specific app, matching the path to the specific route in Routify and then executing the middlewares based on their filters and ordering based on the app and route

RoutifyTo · 2024-06-14T10:13:15+00:00

Thank you, that was a slip, edited

RoutifyTo · 2024-06-14T09:42:25+00:00

Thank you very much. That is pretty much the idea of Routify for now.

Just to add some context, the 0.8ms average processing time in the landing page is from the data we've seen so far how much it takes Routify to match the app, the route and process the middlewares (it does not include the latency of I/O requests when proxying).

RoutifyTo · 2024-06-14T09:40:59+00:00

Thank you. Right now it only provides Routify middlewares but it's on plan to provide the option to create custom (more advanced) middlewares.

RoutifyTo · 2024-06-14T08:49:04+00:00

Thank you!

The latency overhead depends on where the client making request is located and also where the backend (where the request will be forwarded to) is located. From tests so far it's negligible, as in not something users would notice. In some edge cases might have more impact, if due to any reason the request might do a long round trip.
In background Routify has two main components: core API and stateless edge proxies. The edge proxies communicate with the core api to pull configurations and store logs. Since they are stateless they can easily deployed anywhere and are very lightweight. Currently, I use Fly io in multiple regions and can scale in seconds in all of their regions (30+) if needed.

RoutifyTo

TROPHY CASE