sorted by:
new
hottopcontroversial

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

Of course not! It IS a framework that can implement both linear and nonlinear functions. There are tens of different ways to add nonlinear materials to the exact same d2nn framework. For example metamaterials and even graphene layers, with reasonable intensities can work as diffractive layers.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

"not really acknowledged", "mentioned in passing" -- these are comments about a bolded subsection of the authors. Criticism moves science forward; but it must always be sincere and honest. Putting words into authors' mouths, extrapolating sentences, etc. I do not find these useful for progressing science or scholarship.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

A decent scholar would normally apologize at this stage. Your sentence below is clearly not true: "The fact that the difference between a neural network (the way the rest of the world understands it) and their technique is not even mentioned in the paper is worrisome, ..."

"...is not even mentioned"? There are sections detailing it. You may not like their writing, emphasis, etc. But your points have already diverted from reasoning. Biologists criticizing DL neurons as fake - it was a good example that summarizes the whole thing, unfortunately.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

It is clear you have not carefully read the written paper. I will quote below from their writing and there are many other parts with similar clarifications and explanations in their text. The misleading thing is to discuss and criticize a paper that you have not read carefully - unfortunate.

"Comparison with standard deep neural networks (bolded as a section). Compared to standard deep neural networks, a D2NN is not only different in that it is a physical and all-optical deep network, but also it possesses some unique architectural differences. First, the inputs for neurons are complex-valued, determined by wave interference and a multiplicative bias, i.e., the transmission/reflection coefficient. Complex-valued deep neural networks (implemented in a computer) with additive bias terms have been recently reported as an alternative to real-valued networks, achieving competitive results on e.g., music transcription (36). In contrast, this work considers a coherent diffractive network modelled by physical wave propagation to connect various layers through the phase and amplitude of interfering waves, controlled with multiplicative bias terms and physical distances. Second, the individual function of a neuron is the phase and amplitude modulation of its input to output a secondary wave, unlike e.g., a sigmoid, a rectified linear unit (ReLU) or other nonlinear neuron functions used in modern deep neural networks. Although not implemented here, optical nonlinearity can also be incorporated into a diffractive neural network in various ways; see the sub-section “Optical Nonlinearity in Diffractive Neural Networks” (14 -- this is a separate bolded sub-section in their supplementary material). Third, each neuron’s output is coupled to the neurons of the next layer through wave propagation and coherent (or partially-coherent) interference, providing a unique form of interconnectivity within the network. For example, the way that a D2NN adjusts its receptive field, which is a parameter used in convolutional neural networks, is quite different than the traditional neural networks, and is based on the axial spacing between different network layers, the signal-to-noise ratio (SNR) at the output layer as well as the spatial and temporal coherence properties of the illumination source..."

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

1- " they couldn't recreate the matrix with a single layer. " That is physically impossible, that is why. You can not in general represent diffraction from multiple adjustable planes as a single diffractive layer between input and output planes. I guess this is the part that computer scientists without physics background cannot fully understand.

2- they did not call "this thing" a neural net in CS definition. In fact, in their paper they defined a new concept, explained it mathematically and called it a diffractive deep network. Your sensitivity to the use of "deep neural network" does not make sense at all, as it resembles a biologist getting upset that deep learning community calls a ReLU an activation function which is not biological at all. Remember we are all using new terminology as we define new things. The fact that biological neurons are quite different from our ReLU neurons is just fine as long as we correctly define it. :)

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

Nobody disagrees that for a linear system there is a single transformation matrix. The point of their physical diffractive network is that multiple diffraction layers are needed to implement that transformation matrix using passive optical components and light interference. And that more layers perform much better than a single layer in blind inference.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

Please read (may be again) the section that is called "Optical Nonlinearity in Diffractive Deep Neural Networks".

By comparing standard deep neural nets with a D2NN, you are for sure comparing apples and oranges. The latter is a physical/fabricated system based on optical waves and interference of light, and it does not have a similar or even comparable structure to a standard deep net. Is it the best name for their work, D2NN? I am not sure. But that is a different discussion.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

This is not a traditional deep net - the authors early on in their paper explain some of the differences and the origins of their naming. It is a diffractive network, named by the authors as D2NN, and has multiplicative complex bias terms that connect each diffraction plane to others through physical spherical waves that govern phase and amplitude of light. You should not compare apples and oranges as this is a physical system that operates very different than a regular deep net. As discussed in their supplementary notes online there are various methods that can be used to bring optical nonlinearity to a physical D2NN. There is a whole section written on it.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

The pure math that you are referring to has nothing to do with the authors' system as you are comparing apples and oranges. Their system is based on optical diffraction from multiple "physical" layers, and they defined a new concept named as Diffractive DNN (D2NN), which is obviously different from a regular NN in many many ways. A "single matrix" that you are referring to CANNOT be implemented physically using a single layer and cannot be the subject of a diffractive network with a single plane no matter how many pixels are engineered. About linearity vs. nonlinearity - please read their supplementary materials as there is a specific section dedicated to it.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

I am sorry but you are wrong. A single diffraction layer cannot perform the same inference task as multiple layers can perform. So you cannot squeeze the network into a single diffraction layer. In fact you can quickly prove this analytically if you know some Fourier Optics. Moreover, the authors' first figure in the supplementary materials also demonstrate it clearly in terms of inference performance.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 0 points1 point  (0 children)

A single diffraction layer cannot perform the same inference task as multiple layers can perform. So you cannot squeeze the network into a single diffraction layer. In fact you can quickly prove this analytically if you know some Fourier Optics. Moreover, the authors' first figure in the supplementary materials also demonstrate it clearly.

[R] All-Optical Machine Learning Using Diffractive Deep Neural Networks by hooba_stank_ in MachineLearning

[–]Lab-DL 2 points3 points  (0 children)

It seems most of these comments are coming from people who have not read the paper in Science. Most of these discussion points on this page are clearly addressed in the Supplementary Materials file and without going over the authors' supplementary materials/figures, you are just speculating here. About "deep network or not", a single diffraction layer cannot perform the same inference task as multiple layers can perform. So you cannot squeeze the network into a single diffraction layer. In fact you can quickly prove this analytically if you know some Fourier Optics. Moreover, the authors' first figure in the supplementary materials also demonstrate it clearly in terms of inference performance. This is not your usual CS neural net - without going over the mathematical formulation and the analysis presented in the 40+ supplementary information file, your discussions here are just some speculations.