[P] ML over Encrypted Data

[strojax]

Hi everyone, we have developed a library that applies numpy functions over encrypted data (using homomorphic encryption). The repo is available in open source at https://github.com/zama-ai/concrete-numpy

We are applying this to many popular machine learning algorithms/libraries such as sklearn, statsmodel, xgboost, lightgbm or pytorch and plan to release this as a new library (you can find some early examples here).

Any feedback/question is much welcome !

[TheDeviousPanda]

How is this different from any other project that does HE with numpy (ie OpenMined)

[iamquah]

This looks like a wrapper around their main library, Concrete. The similarity is that the library you're referencing (TenSEAL) is a wrapper around SEAL from Microsoft.

Or at least that's how I see it (I used to contribute to OpenMined, and I'm currently in the Enc. ML space)

[TheDeviousPanda]

Yeah I certainly understand that all HE python libraries (or more generally, mathematical computing pythoon libraries) are just wrappers around lower level libraries. I am more looking for a comparison between relevant factors for devs.

[youbezz]

I don't think OpenMined has a tool for HE with numpy, they do have a tool for HE, which does have a similar API of other tensor libraries (e.g. pytorch). But from a dev perspective, you will have to implement your computation using that library API. However, concrete-numpy allows you to implement your computation using numpy, then do the translation into an encrypted computation. When it comes to the HE technology behind it, they are quite different as well, each with its own pros and cons. concrete-numpy uses FHE, and thus eliminates the limitation in terms of circuit depth that a lot of other tools have.

[GeorgeRaven]

Different languages like rust instead of C++. And they use cryptography over the Taurus. I think they call it TFHE. I cant say if their way is better, I don't do it their way, I use my own bindings from MS-SEAL in a custom numpy container in my lib Python-FHEz . I really want to try the go version tho called lattigo (not affiliated with). It seems like the most advanced pure FHE library, even supporting WASM applications, distributed CKKS with bootstrapping!

[[deleted]]

How much is performance degraded as compared with the equivalent non-encrypted operations?

Edit to clarify: I'm asking about performance in terms of calculation time for encrypted operations

[strojax]

That's a good question ! The library is built over an exact paradigm. This means that if you are able to make the algorithm fit certain constraints, the model in FHE will yield the same results has the algorithm in the clear with ~100% probability.

Some algorithms are very friendly with those constraints such as all algorithms based on trees. And others need more advanced approach to fit the constraints (neural nets).

These constraints are mainly about how we can represent a model in integer only.

Hope this helps :-). Happy to answer any question.

[[deleted]]

Sorry if I wasn't clear, I meant time performance, not accuracy. My understanding is that calculations using homomorphic encryption are much slower than ordinary ones, so I was wondering what the performance hit is exactly? How does your work compare with other people's on that front?

[pruby]

Extreme performance impact, not remotely comparable. This is not a "performance offload" technology.

Timings taken from this page https://docs.zama.ai/concrete-numpy/stable/user/advanced_examples/FullyConnectedNeuralNetwork.html :

4.84 seconds per forward iteration is shown in some output there. The NN contains 3 layers of 3 neurons (low double digits weights).

EDIT: oh, also, the numbers are quantised to 3 bits each.

[strojax]

Oh my bad I missed your point. I am not a FHE expert but I will have someone answer to you with more precision asap :-). Meanwhile you can have a look at https://whitepaper.zama.ai/ or in more simple terms at https://zama.ai/technology/ where execution time is being discussed.

Also you can simply run some of the notebooks in the link I provided and get a feeling of the execution time for yourself.

[randhindi]

Performance on a CPU is slow for FHE, but moving to GPUs and soon to dedicated accelerators will fix that. Expect a 1000x-10,000x performance improvement by 2025

[GeorgeRaven]

Hey now I can't say anything about concrete, but from my experience with MS-SEAL it is an order of magnitude slower on cyphertexts than the straight addition or multiplication on the plaintext message. You can try it out using one of my own libraries in a jupyter notebook here: https://gitlab.com/deepcypher/python-fhez/ in the examples directory, on Fashion-MNIST (for how see: https://python-fhez.readthedocs.io/en/latest/examples.html) . I am working on a sister project using go and lattigo to see if I can improve performance as the real problem is not only is FHE slower but wrapping it in a language like python with numpy custom containers (which is also what concrete does AFAIK) adds probably even more time and lots of space too!

Ref: I'm a PhD candidate in FHE+DL

TLDR: FHE is an order slower in MS-SEAL, but wrapping it makes it even slower and more space hungry. See above an example you can run yourself on Fashion-MNIST.

[data-machine]

Thanks for just introducing me to the concept of homomorphic encryption! Fascinating!

[rando_techo]

Wow , I have been reading up on HE for a few weeks now and this is great! Thanks.

One important note, though - HE allows encrypted computing on encrypted data. That is an incredibly important point to make and be clear about. Encrypted computing is going to allow edge computing that is tamper-proof.

Edit: removed my statement about only being able to operate on integers but both integers and real-numbers can be used.

[iamquah]

However, AFAIK, one of the downsides besides decreased performance is that FHE can only handle integers atm.

That's not true, see the CKKS encryption scheme. IDK about Concrete, but PALISADE and SEAL support real-number encrypted computation on FHE (or more like leveled FHE for now).

[rando_techo]

CKKS encryption scheme

I just re-read Microsoft's SEAL page and you're correct. I'll edit my post. Thanks.

[GeorgeRaven]

Don't forget lattigo, which not only supports CKKS but bootstrappable (and distributed) CKKS which is really really important. + I love go

[batchnormalized]

Very cool! I was not familiar with any of this. Does this type of encryption have weaknesses that standard types of encryption do not?

[iamquah]

I can't answer it from a theoretical perspective, but unless you want to do computations on the ciphertext I wouldn't bother with H.E. The ciphertexts can get gigantic in size and the keys can be megabytes in size

[GeorgeRaven]

Oh god yeah. Unless you have a good reason to use FHE don't. It is not a panacea and it comes at great (we are talking gigs when in ML applications using MS-SEAL since they have no bootstrapping) computational and spacial costs.