All-Optical Machine Learning Using Diffractive Deep Neural Networks

We introduce an all-optical deep learning framework, where the neural network is physically formed by multiple layers of diffractive surfaces that work in collaboration to optically perform an arbitrary function that the network can statistically learn

Abstract:

Deep learning has been transforming our ability to execute advanced inference tasks using computers. We introduce a physical mechanism to perform machine learning by demonstrating an all-optical Diffractive Deep Neural Network (DNN) architecture that can implement various functions following the deep learning-based design of passive diffr...More

Code:

Data:

Insight

Full Text

PPT (Upload PPT)

Similar

Reference

Cited

Introduction

Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts.
As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method
After this numerical training phase implemented in a computer, the D2NN design is fixed and the transmission/reflection coefficients of the neurons of all the layers are determined.
This D2NN design, once physically fabricated using e.g., 3D-printing, lithography, etc., can perform, at the speed of light propagation, the specific task that it is trained for, using only optical diffraction and passive optical components/layers, creating an efficient and fast way of implementing machine learning tasks

Highlights

Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts
We introduce an all-optical deep learning framework, where the neural network is physically formed by multiple layers of diffractive surfaces that work in collaboration to optically perform an arbitrary function that the network can statistically learn
A D2NN can be physically created by using several transmissive and/or reflective layers, where each point on a given layer represents an artificial neuron that is connected to other neurons of the following layers through optical diffraction
Following the Huygens’ Principle, our terminology is based on the fact that each point on a given layer acts as a secondary source of a wave, the amplitude and phase of which are determined by the product of the input wave and the complex-valued transmission or reflection coefficient at that point
An artificial neuron in a diffractive deep neural network is connected to other neurons of the following layer through a secondary wave that is modulated in amplitude and phase by both the input interference pattern created by the earlier layers and the local transmission/reflection coefficient at that point
A comparison of Fig. 3C and Fig. S4 reveals that the diffractive surface reconstruction errors, absorption related losses at different layers and 0.1 mm random misalignment error for each network layer, all combined, reduced the overall performance of the network’s digit classification accuracy from 91.75% (Fig. 3C) to 89.25% (Fig. S4)
As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method

Results

A D2NN can be implemented in transmission or reflection modes by using multiple layers of diffractive surfaces, without loss of generality here the authors focus on coherent transmissive networks with phase-only modulation at each layer, which is approximated as a thin optical element.
In this case, each layer of the D2NN modulates the wavefront of the transmitted field through the phase values of its neurons.

Conclusion

For a D2NN, after all the parameters are trained and the physical diffractive network is fabricated or 3D-printed, the computation of the network function is implemented all-optically using a light source and optical diffraction through passive components.
A phase-only D2NN can be designed by using the correct combination of low-loss materials and appropriately selected illumination wavelengths, such that the energy efficiency of the diffractive network is only limited by the Fresnel reflections that happen at the surfaces of different layers.
Such reflection related losses can be engineered to be negligible by using anti-reflection coatings on the substrates.
The match between the experimental results obtained with our 3D-printed D2NNs and their numerical simulations supports this

Summary

Introduction:
Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts.
As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method
After this numerical training phase implemented in a computer, the D2NN design is fixed and the transmission/reflection coefficients of the neurons of all the layers are determined.
This D2NN design, once physically fabricated using e.g., 3D-printing, lithography, etc., can perform, at the speed of light propagation, the specific task that it is trained for, using only optical diffraction and passive optical components/layers, creating an efficient and fast way of implementing machine learning tasks
Results:
A D2NN can be implemented in transmission or reflection modes by using multiple layers of diffractive surfaces, without loss of generality here the authors focus on coherent transmissive networks with phase-only modulation at each layer, which is approximated as a thin optical element.
In this case, each layer of the D2NN modulates the wavefront of the transmitted field through the phase values of its neurons.
Conclusion:
For a D2NN, after all the parameters are trained and the physical diffractive network is fabricated or 3D-printed, the computation of the network function is implemented all-optically using a light source and optical diffraction through passive components.
A phase-only D2NN can be designed by using the correct combination of low-loss materials and appropriately selected illumination wavelengths, such that the energy efficiency of the diffractive network is only limited by the Fresnel reflections that happen at the surfaces of different layers.
Such reflection related losses can be engineered to be negligible by using anti-reflection coatings on the substrates.
The match between the experimental results obtained with our 3D-printed D2NNs and their numerical simulations supports this

Funding

Funding: The Ozcan Research Group at UCLA acknowledges the support of the National Science Foundation (NSF) and the Howard Hughes Medical Institute (HHMI)

Reference

Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature. 521, 436–444 (2015).
G. Litjens et al., A survey on deep learning in medical image analysis. Medical Image Analysis. 42, 60–88 (2017).
A. Graves, A. Mohamed, G. Hinton, in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on (IEEE, 2013), pp. 6645–6649.
K. Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
A. Krizhevsky, I. Sutskever, G. E. Hinton, in Advances in neural information processing systems (2012), pp. 1097–1105.
D. Silver et al., Mastering the game of Go with deep neural networks and tree search. Nature. 529, 484–489 (2016).
U. S. Kamilov et al., Learning approach to optical tomography. Optica. 2, 517 (2015).
Y. Rivenson et al., Deep learning microscopy. Optica. 4, 1437 (2017).
K. H. Jin, M. T. McCann, E. Froustey, M. Unser, Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing. 26, 4509–4522 (2017).
Y. Rivenson, Y. Zhang, H. Gunaydin, D. Teng, A. Ozcan, Phase recovery and holographic image reconstruction using deep learning in neural networks. Light: Science & Applications. 7, 17141 (2018).
A. Sinha, J. Lee, S. Li, G. Barbastathis, Lensless computational imaging through deep learning. Optica. 4, (2017).
K. Hammernik et al., Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine (2017), doi:10.1002/mrm.26977.
Y. Rivenson et al., Deep learning enhanced mobile-phone microscopy. ACS Photonics (2018), doi:10.1021/acsphotonics.8b00146.
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition. Proceedings of the IEEE. 86, 2278–2324 (1998).
Y. Shen et al., Deep learning with coherent nanophotonic circuits. Nature Photonics. 11, 441–446 (2017).
D. Psaltis, D. Brady, X.-G. Gu, S. Lin, Holography in artificial neural networks. Nature. 343, 325–330 (1990).
K. H. Wagner, in Frontiers in Optics (Optical Society of America, 2017), pp. FW2C–1.
B. J. Shastri et al., Principles of neuromorphic photonics. arXiv preprint arXiv:1801.00016 (2017).
M. Hermans, M. Burm, T. Van Vaerenbergh, J. Dambre, P. Bienstman, Trainable hardware for dynamical computing using error backpropagation through physical media. Nature Communications. 6, 6729 (2015).
D. Brunner, M. C. Soriano, C. R. Mirasso, I. Fischer, Parallel photonic information processing at gigabyte per second data rates using transient states. Nature Communications. 4, 1364 (2013).
R. Girshick, J. Donahue, T. Darrell, J. Malik, in Proceedings of the IEEE conference on computer vision and pattern recognition (2014), pp. 580–587.
Y. Chen, Z. Lin, X. Zhao, G. Wang, Y. Gu, Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 7, 2094–2107 (2014).
J. W. Goodman, Introduction to Fourier optics (Roberts and Company Publishers, 2005). 24. http://www.image-net.org/25. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research. 15, 1929–1958 (2014).
26. A. Greenbaum et al., Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nature Methods. 9, 889–895 (2012).
27. A. Greenbaum et al., Wide-field computational imaging of pathology slides using lens-free on-chip microscopy. Science Translational Medicine. 6, 267ra175 (2014).
28. A. Ozcan, E. McLeod, Lensless imaging and sensing. Annual Review of Biomedical Engineering. 18, 77–
29. A. Greenbaum et al., Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy. Scientific Reports. 3, 1717 (2013). 30. https://www.nanoscribe.de/en/31. D. P. Kingma, J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 15 (2014).
32. M. Kazhdan, H. Hoppe, Screened poisson surface reconstruction. ACM Transactions on Graphics. 32, 1–13 (2013).
33. http://www.meshlab.net 34. P. Cignoni et al., Meshlab:an open-source mesh processing tool. Eurographics Italian Chapter Conference, 129–136 (2008).
35. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 13, 600–612 (2004).
36. D. Grischkowsky, S. Keiding, M. Van Exter, C. Fattinger, Far-infrared time-domain spectroscopy with terahertz beams of dielectrics and semiconductors. JOSA B. 7, 2006–2015 (1990).

Full Text

Your rating :

All-Optical Machine Learning Using Diffractive Deep Neural Networks

Introduction:

Results:

Conclusion: