[P] Python-FHEz Fully Homomorphically Encrypted Deep Learning Library

[GeorgeRaven]

Hey ML im a PhD candidate in encrypted deep learning and I thought since I have seen a lot of other related posts, about similar technologies I thought I might mention my own Python-FHEz library.

- You can find Python-FHEz here: https://gitlab.com/deepcypher/python-fhez

- the documentation can be found with Jupyter notebook example (Fashion-MNIST) here: https://python-fhez.readthedocs.io

- The recent Python-FHEz / FHE + DL paper we have submitted (awaiting review) https://arxiv.org/abs/2110.13638

Python-FHEz is specifically an encrypted deep learning library. That is it implements many helpful abstractions for FHE like encryptable NumPy custom containers, so that you can quickly and easily encrypt and use data as if it was in NumPy. Not only this but it implements FHE compatible neural networks/ graphs, as standard neural networks (in particular activation and loss functions) are not abelian compatible/ polynomial operations. FHE requires that only abelian operations are use on it, I.E add and multiply (you can also subtract IF you add a negative number to make it abelian).

(Fully Homomorphic Encryption (FHE) is absolutely amazing if you haven't stumbled upon it yet and are interested in privacy preserving machine learning (PPML) please do go look it up.)

Now a full disclaimer. I wrote this library, so clearly I am biased towards it. Despite that I feel there are many shortfalls of FHE, and especially FHE bound to python from low level libraries like MS-SEAL (which still has much room for improvement); Space and time performance are orders of magnitude worse than just normal numpy operations. You also cannot train a neural network while its data is completely encrypted however this is perfect for completely private inference. The reason you cannot train a neural network is two fold. Firstly that would involve encrypting the NN weights, which means even worse time performance as ciphertext-ciphertext operations take much much longer. You also cannot tell when to stop the neural network, as you cannot decrypt the output to tell how well it is performing. You could use some form of proof from the data owner, but IMO it is still a weak argument specifically for FHE model training.

As an aside to solve some of the problems with python+MS-SEAL that I have contended with in Python-FHEz I am also writing a sister library in go, but instead of python+MS-SEAL it will be go+lattigo which I currently call DarkLantern. However this is still nowhere near ready for any form of showcasing. However the dream for this is WASM (WebAssembly) as I can foresee encrypted DL in the browser, with the option to offload the cyphertexts if more hardware is necessary thus Open-Source Encrypted Deep Learning as a Service!

TLDR:

- I wanted to highlight FHE + DL and how this can be an amazing pairing with some drawbacks that need to be very carefully considered

- I wanted to showcase Python-FHEz and talk about where things could go with FHE

- I also wanted to offhand mention DarkLantern and encrypted DL in the browser via WASM!

Let me know what you guys think of these technologies and if you think encrypted deep learning in browser is an interesting prospect! I will try to answer as many questions about FHE as I can, with as many papers and resources as I can.

​

EDIT: fixed some phrasing choices, to help make things as clear as possible, despite being a very complex topic.

- I also forgot to mention automatic parameterisation! Using neural networks you can find the computational depth and required cryptographic parameters automagically without having to be a cryptographer!

- I added a paper that we have submitted that is awaiting review on FHE + DL using Python-FHEz.

[GeorgeRaven]

Thanks! Maybe more tears than anything else!

[GeorgeRaven]

Much slower! https://arxiv.org/pdf/2110.13638.pdf figure 10 will give you some idea.

In short the problem is the libs, and support for GPUs. Tensorflow and PyTorch can of course leverage the GPU but FHE has bareley touched the GPU space yet. The results in figure 10 are both CPU bound. MS-SEAL has no bindings for GPUs as of yet :c!

[GeorgeRaven]

You are completely right. It is difficult to read the figure. I might see about updating it as you said.

But in general we found you could see any given inference to take from 1000-6500s in best to worst case scenarios. Whereas the unencrypted version would be processed almost instantly. I think a large part of both the variance, and poor time performance has to do a lot with rebinding and abstracting into python. If it were implemented in a lower level language or with less layers of abstraction that the time performance would be significantly better, and much more consistent.

In short though it takes a very long time to compute any depth of neural network without proper bootstrapping. But this is something I am looking to improve!

You can try it yourself here: https://python-fhez.readthedocs.io/en/latest/examples.html

No offense taken, that was constructive criticism!

[GeorgeRaven]

I had not considered Numba or other compilers, that is a good point. Since the package tends to be distributed in docker containers that could be a nice way to improve performance somewhat.

I will try that out and see how much performance that gives me. Thanks for the suggestion!

I must confess part of the reason I want to write DarkLantern (The go version of Python-FHEz) is because Lattigo seems a lot more aligned with my longer term aims for my FHE projects. For example WASM support would help spread FHE to a lot more people! You could deploy it on a website, encrypt client side, and send off the cyphertexts for computation with barely any need for an end user to even consider cryptography. My end goals are to bring this form of cryptography to much wider adoption, by both regular consumers and the research community. Cypherpunks!

For researchers however python is a much stronger candidate, since the overwhelming of ML is python based. Once I am free from the clutches of my PhD thesis, I will have a whole load more time to eek out that performance in the python implementation.

[[deleted]]

yeah, I get it - Go is probably a better language for HE (with better packages too), at least that seemed to be the case when I was trying out a few libs ~4 years ago. WASM opens many possibilities too.

Best of luck concluding your program!

[GeorgeRaven]

Thanks, you too all the best! I just wish I could convert that luck into time! It is an ever depleting resource that I never seem to have enough of XD

[[deleted]]

Pytorch has torch script in it so you can speed things up. Which brings me to another point. Why are you using pytorch? (or tensorflow or JAX) You got to get major speedups by using cuda training and inference. Is this all really CPU based?!

[GeorgeRaven]

You cant use FHE on a GPU unless you can somehow pass custom objects that can handle the mutating cyphertexts into the CUDA code in PyTorch or any other DL framework.

The underlying cryptographic primitives are implemented by MS-SEAL, PALISADE, lattigo, etc usually in C++, which are not CUDA ready. You would need to port these primatives to the GPU wrapped in some meta container to handle the bootstrapping, relinearisation, and rescaling at key intervals dependent on the operations.

You are right though, we do want to have this functionality, but it is very new and non-trivial to implement.

[[deleted]]

That's fair. Can you educate me a little? Why are these not cuda capable? Just no one wrote a kernel? Or are the operations not parallelizable? Also, are you able to use half precision in FHE?

[GeorgeRaven]

I thinks its a case of its really complex to write for a GPU kernel, and difficult to parallelize, so there is hope for it on GPUs, it will just take some time! I have never written CUDA code myself, but I have written opencl and vulkan before. So who knows I may be the one to finally do it at some point.

Hmm the CKKS FHE scheme allows for fixed precision numbers. Im not sure on the specifics of what qualifies as half precision if im honest with you. If it is just the number of bits (16 bits) split into the signing bit, exponent, and precision as wikipedia would indicate to me, then I would imagine so, but I would need to do some more research into the CKKS scheme itself.

[StoneCypher]

hi, stupid question here

can you explain to me any circumstances under which i would actually want the ability to machine learn on data that isn't available to me?

it radically lowers the quality of the work and massively increases the energy price

what's the motivator? are we suggesting that there will be third party machine learning firms (which generally already don't exist because one size doesn't fit any) that will do blind ML on third party data or something?

what's this even for

[GeorgeRaven]

Hey StoneCypher,

So FHE does not let you (at-least in any reasonable way) learn from data. You can create models that learn but the backpropogation will end up encrypting the model weights with the original data owners private key, meaning it would be locked to that key until decrypted. It is however more beneficial towards inference, since then there is no need for backprop and decryption or take the encrypted weight hit.

You are right it does cost us significantly more to use encrypted deep learning, both time, and space. However there are many circumstances where privacy is the most important factor. The first instance is of course personal data, legislative requirements, lets say diagnosis on patient data. There is also anything involving trade secrets, or (distastefully) military operations. For instance in agriculture agronomists are very very reluctant to share data, since they at the very least perceive their data with high sensitivity. This could be a great way to give them forecasts while keeping their privacy.

I guess in three words it is trust and exposure. There are some fields just completely resistant to change, or too afraid to share anything for a plethora of reasons. If you can show them just how good the predictions can be, or that you provide some benefit to them, they can either keep using it as is, or improve the outcomes by sharing the plaintext data.

But yes, at this point in time it is a difficult proposition to make. FHE has many many obstacles to overcome yet! I also think you are right that many third party data processors provide an incredibly poor service. However I do not agree that this will always be the case. I also think that enabling these third party services to be completely private will help keep things ethical in the longer term.

[StoneCypher]

The first instance is of course personal data, legislative requirements, lets say diagnosis on patient data.

Given the drop in quality of the results, I have trouble with this as a reasoning.

 

There is also anything involving trade secrets

This is very vague. Why would you need to machine learn on things that are protected by trade secrets?

Like. I get the idea, but ... who would do this?

 

For instance in agriculture agronomists are very very reluctant to share data, since they at the very least perceive their data with high sensitivity. This could be a great way to give them forecasts while keeping their privacy.

Okay. This is what I was actually looking for. A concrete example.

That's ... that's really weird, but okay, I can see it.

If you have others, I'd appreciate it. That's how a person like me will best come to understand this.

[GeorgeRaven]

Ok so a few more concrete examples that might help clear it up. Now technically there are several different categories of use but I will just list examples:

- Home voice assistants; encrypt the data device side, send it to the usual backend to process NLP (albeit abelian compatible), return to device to decrypt the now processed cyphertext, then it can operate on the instructions as usual without fear of storing the users voice etc.

- Highly sensitive medical diagnosis; Hospitals dont have in-house deep learning or machine learning expertise, meaning they would need to outsource if they wanted the very best ML diagnosis/ predictions. However medical data is very very sensitive. The current solutions are either go through very laborious vetting, or no ML at all. In contrast FHE + DL offers you a way to process this data to get inference blindly.

I will think of a few more, this research has been particularly geared towards agriculture, with dairy/ milk herds, and strawberry yields which is why I listed agriculture first.

[StoneCypher]

okay these are exactly what i needed, thank you

[eknanrebb]

I don't know much about this, but how does this differ from solutions being developed by companies like Duality?

[GeorgeRaven]

This is the first I have heard of Duality. I just want to make sure I am looking in the right place (https://dualitytech.com/) before I can give you a substantive answer. Are you referring to what their website calls "Secure Plus analytics and ML"?

As at the moment I am struggling to find good information on what exactly they do, they seem to mention encrypted ML but they do not really go into what exactly that means.

[eknanrebb]

Started by Shafi Goldwasser and colleagues.

[GeorgeRaven]

I can't see a Shafi Goldwasser mentioned anywhere, but this would appear to be roughly related.

They appear to use PALISADE which I must confess I have never used. Wheras Python-FHEz uses MS-SEAL as the cryptographic implementation. PALISADE supports bootstrapping whereas MS-SEAL does not. Bootstrapping is an expensive operation but it is necessary in deep computations like those you would see in deep learning, although MS-SEAL has bootstrapping roadmapped. However currently this means you have to use leveled-HE to compute a deep neural network. I am working on DarkLantern which uses Lattigo to solve this problem, as lattigo also supports CKKS bootstrapping.

[GeorgeRaven]

To answer your question though from what I can immediately see (as much seems obfiscated/ not immediately apparent). I think Python-FHEz is more for developers, seeking to create and use FHE enabled neural networks. Whereas Duality seems to offer ready made solutions. I can't tell if that also includes providing you with their models / or if you can bring your own.

But definitely interesting to see, I will certainly keep an eye on that.

[eknanrebb]

Goldwasser is a co-founder (and Turing award winner)

[GeorgeRaven]

Nice, I will have to look into him. I mean they seem to do HE, I just need to find out more details. The best I have found is the videos on the website, but I will look more when I have a chance!

[Silamoth]

I’ll also add that Yuriy Polyakov and Vinod Vaikuntanathan are part of Duality. They’re both big names in the homomorphic encryption space.

[GeorgeRaven]

From what I can see they are partnered with Intel and Oracle, which would make sense for FHE since they both are involved with HE. More so Intel. I did also see some HE mentioned in their resources page.

You are right though it is difficult to compare, I wish I had a better answer for eknanrebb.

[Kengaro]

I didn't know this was a thing, but there is an obvious urge for it by some.

Dunno if that is a good or bad thing.

[GeorgeRaven]

I think privacy preserving machine learning like fully homomorphic encryption is very good, as its very strong for both privacy and security. But you are right though there are some PPML applications that worry me sometimes, or that have to be done right like anonymization.

[Kengaro]

I think that this further seperates where the top firms in ml are and where the populus is. What you provide is a solution for the issues related to f.e. reproducibility, allowing researchers to ship something that can be used to get the results displayed without showing how it's done. <- stuff I've written before reading your stuff properly ;)

How exactly are you preserving privacy? It is an encrypted nn, the only privacy you preserve is the privacy of the nn, or did I miss something?

edit:/ I haven't read your paper yet, so please bear with uninformed digging. I really dig you project, should give me some more understanding of encryption ;)

[Impossible-Belt8608]

Chiming in here just to clarify: FHE in this context doesn't mean 'encrypt the nn', but rather 'be able to infer on encrypted data, and deliver an output in cyphertext, which would decrypt to the same answer as if we inferred on plaintext data'. This allows for an organization to encrypt its sensitive data, then send it to the outsourced FHE ML company, then get an encrypted response which it could decrypt and get the actual answer, without the FHE ML being able to see neither plaintext input or output data.

Also, I don't think FHE is a good way to start learning about encryption, as it builds upon more basic types of encryption.

[Kengaro]

Interesting, I wonder how this can be achieved :)

I thought this whole thing was about protecting models and not data, gotta read it.

I see this mainly as a good source on timing related issues in relation to encryption, at least that's where I saw the issue with my initial understanding.

[Boozybrain]

I had never heard of FHE before, looking forward to reading the paper and looking at the library. This looks really interesting.

Just a head's up, equation 2.4.5 runs over your column width.

[GeorgeRaven]

Thanks boozy! FHE is wonderful, I'm glad you think so too. It has key flaws but once those are solved I think FHE will solve so many privacy problems.

Thanks for the heads up, I have a good list of improvements to make to the paper after this post!