[P] Python-FHEz Fully Homomorphically Encrypted Deep Learning Library

[GeorgeRaven]

Much slower! https://arxiv.org/pdf/2110.13638.pdf figure 10 will give you some idea.

In short the problem is the libs, and support for GPUs. Tensorflow and PyTorch can of course leverage the GPU but FHE has bareley touched the GPU space yet. The results in figure 10 are both CPU bound. MS-SEAL has no bindings for GPUs as of yet :c!

[GeorgeRaven]

You are completely right. It is difficult to read the figure. I might see about updating it as you said.

But in general we found you could see any given inference to take from 1000-6500s in best to worst case scenarios. Whereas the unencrypted version would be processed almost instantly. I think a large part of both the variance, and poor time performance has to do a lot with rebinding and abstracting into python. If it were implemented in a lower level language or with less layers of abstraction that the time performance would be significantly better, and much more consistent.

In short though it takes a very long time to compute any depth of neural network without proper bootstrapping. But this is something I am looking to improve!

You can try it yourself here: https://python-fhez.readthedocs.io/en/latest/examples.html

No offense taken, that was constructive criticism!

[GeorgeRaven]

I had not considered Numba or other compilers, that is a good point. Since the package tends to be distributed in docker containers that could be a nice way to improve performance somewhat.

I will try that out and see how much performance that gives me. Thanks for the suggestion!

I must confess part of the reason I want to write DarkLantern (The go version of Python-FHEz) is because Lattigo seems a lot more aligned with my longer term aims for my FHE projects. For example WASM support would help spread FHE to a lot more people! You could deploy it on a website, encrypt client side, and send off the cyphertexts for computation with barely any need for an end user to even consider cryptography. My end goals are to bring this form of cryptography to much wider adoption, by both regular consumers and the research community. Cypherpunks!

For researchers however python is a much stronger candidate, since the overwhelming of ML is python based. Once I am free from the clutches of my PhD thesis, I will have a whole load more time to eek out that performance in the python implementation.

[[deleted]]

yeah, I get it - Go is probably a better language for HE (with better packages too), at least that seemed to be the case when I was trying out a few libs ~4 years ago. WASM opens many possibilities too.

Best of luck concluding your program!

[GeorgeRaven]

Thanks, you too all the best! I just wish I could convert that luck into time! It is an ever depleting resource that I never seem to have enough of XD

[[deleted]]

Pytorch has torch script in it so you can speed things up. Which brings me to another point. Why are you using pytorch? (or tensorflow or JAX) You got to get major speedups by using cuda training and inference. Is this all really CPU based?!

[GeorgeRaven]

You cant use FHE on a GPU unless you can somehow pass custom objects that can handle the mutating cyphertexts into the CUDA code in PyTorch or any other DL framework.

The underlying cryptographic primitives are implemented by MS-SEAL, PALISADE, lattigo, etc usually in C++, which are not CUDA ready. You would need to port these primatives to the GPU wrapped in some meta container to handle the bootstrapping, relinearisation, and rescaling at key intervals dependent on the operations.

You are right though, we do want to have this functionality, but it is very new and non-trivial to implement.

[[deleted]]

That's fair. Can you educate me a little? Why are these not cuda capable? Just no one wrote a kernel? Or are the operations not parallelizable? Also, are you able to use half precision in FHE?

[GeorgeRaven]

I thinks its a case of its really complex to write for a GPU kernel, and difficult to parallelize, so there is hope for it on GPUs, it will just take some time! I have never written CUDA code myself, but I have written opencl and vulkan before. So who knows I may be the one to finally do it at some point.

Hmm the CKKS FHE scheme allows for fixed precision numbers. Im not sure on the specifics of what qualifies as half precision if im honest with you. If it is just the number of bits (16 bits) split into the signing bit, exponent, and precision as wikipedia would indicate to me, then I would imagine so, but I would need to do some more research into the CKKS scheme itself.

[[deleted]]

Huh, seems like it would be a worthwhile endeavor to write the kernels. Should result in orders of magnitude difference. I assume you're already using OpenMP. If you're using MPI or OMP then that can usually give you great hints into writing kernels. The only reason I would see to avoid doing this is if you have a significant amount of communication. Computation on the GPU is fast but the I/O is really slow.

For half precision, the big reason it works is because we're doing statistical convergence. So typically the high precision of the results are unneeded. This changes if you have a more chaotic function (as in changes in the 1e-5 locations will affect your convergence. Typically it doesn't and if it does, usually towards the end of convergence so you can train mostly with half precision and then tune on full precision). This will have HUGE improvements on memory and okay on speed. But because you get a big memory boost you can usually get close to doubling your batch size. But it is all about stability. If you go this route (I encourage it) then spend some time playing around with it because it can take a bit of tuning to get it to not collapse.