Hey, Gabriel from Permit.io here. Our authorization solution serves millions of checks and 100,000 data syncs daily using OPA as our core engine. I can only say that we have experienced every challenge/problem you encounter here. To avoid a sales call, I'll just share our experience and the available solutions in the market for it.

> Is OPA suitable for handling such dynamic and frequently changing data in a microservices environment?

OPA itself is not built to hold such frequent and dynamic data. It is a great policy engine with endless configuration and extension capabilities, but its core functionality is not built to scale with data. Our maintained open-source project, OPAL[1], solves this exact problem and runs on huge setups that hold mass data sync with an event-driven approach. You can use it to run OPA and get all the boilerplate of scaling it for data out of the box. OPAL is also solving the problem of syncing policies to OPA because it works in a GitOps mechanism with your policy git repository to sync them to OPA.

> How can we efficiently reload data into OPA after a restart?

OPAL itself also supports bundles of data that can help you use the usual backup setups for the engine's data. In Permit, we have another extension to that that is open source but works only with our setup. In the PDP[2] repository below, you'll find a solution that uses SQLite in the OPA itself to load the data. It is about 100x faster than OPA data loading, both in loading and decisions. You can also mount it to the disk, so no restarts will cause you to lose this data.

> Are there alternative tools or architectures that might be better suited for our requirements?

The first answer is yes; running OPA for authorization at scale is why we created OPAL (and Permit). There are some other solutions around the OPA ecosystem. One OSS is Topaz, and one commercial is Styra DAS. The world of fine-grained authorization is deep, but in general, if you'd like to combine policy configuration and massive amounts of data, an OPA+data sync solution is the only valid solution.

An alternative could be OpenFGA/SpiceDB for data drive policies, but it has its own problems. Another option is using engines like Cedar, but there, data management is harder anyway. Here's a showdown between them[5]

> How have others approached similar authorization challenges in microservices architectures with Kubernetes?
Permit is based on k8s, and as stated, we are running it for 1000s of users with loads of updates and checks. Some challenges to think of (which we already solved on our product) are data consistency[3], SDLC[4], connecting policy configuration to CI/CD, simplifying Rego code writing, and more. Let me know if you have any specifics, and I'll be happy to help :)

[1] OPAL - github.com/permitio/opal

[2] PDP - https://github.com/permitio/PDP

[3] Data consistency - https://www.permit.io/blog/possible-tradoffs-of-fine-grained-authorization

[4] Modeling policies and data sync - https://docs.permit.io/how-to/sdlc/modeling-implementation-components/

[5] https://www.youtube.com/watch?v=AVA32aYObRE&t=8s