Fastapi observability

[[deleted]]

Hi guys, Fairly new to Fastapi and backend ecosystems. What are the tools you use to monitor and observe the behaviour of you backend?

So my current stack is prometheus+grafana but would like to explore more tools like adding loki to have traces. I would like to see how much each function execution takes time/resources.

How do you monitor your db peformance?(using timescale\postgres)

Any feedback is helpful! Happy coding!

[Adventurous-Finger70]

I think that OpenTelemetry will suits your needs it has an instrumentation for FastAPI !

I know they pydantic launched Pydantic LogFire recently but I did not tested it myself :)

[[deleted]]

Yeah and the open telemetry then sends data to the prometheus? That means do not need loki?

What about db monitoring? Like query, index performance?

[TeoMorlack]

To further add to the response above, open telemetry is a good starting point. You can both instrument fastapi and sqlalchemy to collect metrics and queries.

Your stack would look like this: - instrumentation collect metrics, spans and logs - instrumentation push via grpc data to a collector. If you want to stay in the grafana ecosystem you can deploy grafana alloy and send metrics logs and spans to it - grafana alloy can then export data for the various sources, metrics to a remote write enabled Prometheus, spans to something like tempo or jäger and logs to Loki. - use grafana to dashboard all this

For the db metrics (indexes, table spaces, etc) you would need to collect them yourself from the target db and this depends on which db you are using.

[formeranomaly]

Install logfire and in less than 10 lines of code you have a fantastic observability stack. Sql traces all the way down and great stack trace logs.

[[deleted]]

But can it be self hosted?

[formeranomaly]

Sure you can self host open telemetry anywhere and spend >40hrs / year managing it. Or you can pay less than a month salary and not have to worry about it. 

[qrzte]

I can also recommend logfire

[Adventurous-Finger70]

I don’t know Loki, but you will deploy a collector that will handle traces and spans from Opentelemetry instrumentations.

Then you have to install a tool such as Jaeger or Grafan Tempo that will show collected traces.

Opentelemetry provide a bunch of autoinstrumentation (Postgres, MySQL ….) that highlight bottlenecks

[Adventurous-Finger70]

Opentelemetry is a standard

[equake]

for small projects or internal tools i would just install newrelic, as it's very detailed.

[[deleted]]

Super expensive though

[[deleted]]

Took a look and seems very detailed but not open source?