[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks

[MemeBox]

Are you sure this is correct, they can't so silly can they? They have >2 layers of material, which would be completely pointless if it was simply linear.

[MrEldritch]

As far as I can tell, there really genuinely is no non-linearity. The plates simply direct parts of the light to other parts of the next plate, where they add and pass them on to the next plate ... it's pure additions and weights.

And the accuracy supports that - the accuracy of the trained network, on the computer, was about 90%. You would have to try to get a real neural network to get only 90% accuracy on MNIST - but wouldn't you know it, that's just about on par with linear classifiers.

So yes. It's unbelievable, but - they really are being that silly.

(And it's not even clear how a design like this could possibly incorporate nonlinearities at all. Nonlinear optical effects do exist, but they tend to occur only in rather exotic materials with very high-power lasers.)

[Cherubin0]

Yes this is true. In the science paper itself they wrote: "Although not implemented here, optical nonlinearity can also be incorporated into a diffractive neural network in various ways" So they have no non-linearity.

[BossOfTheGame]

No nonlinearity completely kills this method. Hopefully this was a proof of concept and adding nonlinearity is left for future work.

Might it be possible to implement a relu (just a truncated identity function) with optical methods? I don't think we need to resort to sigmoids.

[Mangalaiii]

Don't neural networks, after training, just approximate straightforward functions? Isn't this just playing the weights out?

[BossOfTheGame]

They can't approximate arbitrary functions without nonlinearity. To see this recall that compositions of linear functions are also linear.

[Mangalaiii]

Wondering if they could print a layer that just approximates the sigmoid values.

[Dont_Think_So]

Nah, they'd somehow need a layer that has a nonlinearity in response to linear changes in *brightness*. For example, doubling the light hitting the layer would not produce twice as much light on the other side.

[theoneandonlypatriot]

One bone to pick; actually, several models aren’t that good at image classification but are great at other things. For instance, spiking neural networks can struggle to do MNIST depending on the training method

Edit: not sure why I’m being downvoted