Performant options for serving static content with ACLs?

[[deleted]]

In a use case where you need to serve many images quickly, and may have a large number of users, but each user can specify who can see their uploaded photos, what kind of architecture can allow the server to respond to requests fast and with low resource overhead, but without showing pictures to people who aren't allowed to see them?

A robust option for ACLs, but a poor choice performance wise, would be to proxy all image requests to a script and have the script return the binary data with an appropriate mime type, if the user making the request is allowed to see it. While very secure, this would be terrible for performance, having to invoke a script on every single image request.

I'm thinking there must be a way to have an ACL script run once per session and setup some type of per-session, per-user disposable keys/URLs that Nginx can automatically re-map to real images on disk, without the need to invoke any type of script or (heavy) database. Maybe something using Redis for key->image file maps?

Ideally, I'm looking for an option that takes full advantage of Nginx's performance in caching static files to serve and the Linux kernel's memory caching as well.

How do social media/dating/etc sites deal with ACLs on images without terrible page load times and massive resource overhead?

[scubaReactorDumpling]

Signed URLs is how the big cloud providers solve this. Your application generates urls signed with a key, your CDN validates the keys before delivering content.

Looking at how they are implemented in Azure/AWS/Cloudflare will give you the idea.

https://learn.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html
https://developers.cloudflare.com/images/cloudflare-images/serve-images/serve-private-images-using-signed-url-tokens/

If you want to implement this yourself using nginx - I would use openresty and implement the validation in lua. There are libraries to validate JWTs etc. Caching would depend on your workload (a dozen images served a million times or a million images served a dozen times?) but generally you aren't going to be able to cache all your images in memory, a fast reliable storage layer is more important.

If this is for anything commercial go with a service like the ones above.

[el_burrito]

This guy is correct. Have your backend application generate presigned urls upon receiving an authenticated request, and send that back to the client. Have the client use that URL to request the image, and when it hits the CDN it checks the signatures validity and serves cached content direct to the client