All-Optical Machine Learning Using Diffractive Deep Neural Networks

Science (New York, N.Y.), Volume abs/1804.08711, 2018.

Cited by: 224|Bibtex|Views46|DOI:https://doi.org/10.1126/science.aat8084
EI WOS
Other Links: dblp.uni-trier.de|pubmed.ncbi.nlm.nih.gov|academic.microsoft.com|arxiv.org
Weibo:
We introduce an all-optical deep learning framework, where the neural network is physically formed by multiple layers of diffractive surfaces that work in collaboration to optically perform an arbitrary function that the network can statistically learn

Abstract:

Deep learning has been transforming our ability to execute advanced inference tasks using computers. We introduce a physical mechanism to perform machine learning by demonstrating an all-optical Diffractive Deep Neural Network (DNN) architecture that can implement various functions following the deep learning-based design of passive diffr...More

Code:

Data:

0
Introduction
  • Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts.
  • As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method
  • After this numerical training phase implemented in a computer, the D2NN design is fixed and the transmission/reflection coefficients of the neurons of all the layers are determined.
  • This D2NN design, once physically fabricated using e.g., 3D-printing, lithography, etc., can perform, at the speed of light propagation, the specific task that it is trained for, using only optical diffraction and passive optical components/layers, creating an efficient and fast way of implementing machine learning tasks
Highlights
  • Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts
  • We introduce an all-optical deep learning framework, where the neural network is physically formed by multiple layers of diffractive surfaces that work in collaboration to optically perform an arbitrary function that the network can statistically learn
  • A D2NN can be physically created by using several transmissive and/or reflective layers, where each point on a given layer represents an artificial neuron that is connected to other neurons of the following layers through optical diffraction
  • Following the Huygens’ Principle, our terminology is based on the fact that each point on a given layer acts as a secondary source of a wave, the amplitude and phase of which are determined by the product of the input wave and the complex-valued transmission or reflection coefficient at that point
  • An artificial neuron in a diffractive deep neural network is connected to other neurons of the following layer through a secondary wave that is modulated in amplitude and phase by both the input interference pattern created by the earlier layers and the local transmission/reflection coefficient at that point
  • A comparison of Fig. 3C and Fig. S4 reveals that the diffractive surface reconstruction errors, absorption related losses at different layers and 0.1 mm random misalignment error for each network layer, all combined, reduced the overall performance of the network’s digit classification accuracy from 91.75% (Fig. 3C) to 89.25% (Fig. S4)
  • As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method
Results
  • A D2NN can be implemented in transmission or reflection modes by using multiple layers of diffractive surfaces, without loss of generality here the authors focus on coherent transmissive networks with phase-only modulation at each layer, which is approximated as a thin optical element.
  • In this case, each layer of the D2NN modulates the wavefront of the transmitted field through the phase values of its neurons.
Conclusion
  • For a D2NN, after all the parameters are trained and the physical diffractive network is fabricated or 3D-printed, the computation of the network function is implemented all-optically using a light source and optical diffraction through passive components.
  • A phase-only D2NN can be designed by using the correct combination of low-loss materials and appropriately selected illumination wavelengths, such that the energy efficiency of the diffractive network is only limited by the Fresnel reflections that happen at the surfaces of different layers.
  • Such reflection related losses can be engineered to be negligible by using anti-reflection coatings on the substrates.
  • The match between the experimental results obtained with our 3D-printed D2NNs and their numerical simulations supports this
Summary
  • Introduction:

    Deep learning is one of the fastest-growing machine learning methods of this decade [1], and it uses multilayered artificial neural networks implemented in a computer to digitally learn data representation and abstraction, and perform advanced tasks, comparable to or even superior than the performance of human experts.
  • As an analogy to standard deep neural networks, one can consider the transmission/reflection coefficient of each point/neuron as a multiplicative “bias” term, which is a learnable network parameter that is iteratively adjusted during the training process of the diffractive network, using an error back-propagation method
  • After this numerical training phase implemented in a computer, the D2NN design is fixed and the transmission/reflection coefficients of the neurons of all the layers are determined.
  • This D2NN design, once physically fabricated using e.g., 3D-printing, lithography, etc., can perform, at the speed of light propagation, the specific task that it is trained for, using only optical diffraction and passive optical components/layers, creating an efficient and fast way of implementing machine learning tasks
  • Results:

    A D2NN can be implemented in transmission or reflection modes by using multiple layers of diffractive surfaces, without loss of generality here the authors focus on coherent transmissive networks with phase-only modulation at each layer, which is approximated as a thin optical element.
  • In this case, each layer of the D2NN modulates the wavefront of the transmitted field through the phase values of its neurons.
  • Conclusion:

    For a D2NN, after all the parameters are trained and the physical diffractive network is fabricated or 3D-printed, the computation of the network function is implemented all-optically using a light source and optical diffraction through passive components.
  • A phase-only D2NN can be designed by using the correct combination of low-loss materials and appropriately selected illumination wavelengths, such that the energy efficiency of the diffractive network is only limited by the Fresnel reflections that happen at the surfaces of different layers.
  • Such reflection related losses can be engineered to be negligible by using anti-reflection coatings on the substrates.
  • The match between the experimental results obtained with our 3D-printed D2NNs and their numerical simulations supports this
Funding
  • Funding: The Ozcan Research Group at UCLA acknowledges the support of the National Science Foundation (NSF) and the Howard Hughes Medical Institute (HHMI)
Reference
  • Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature. 521, 436–444 (2015).
    Google ScholarLocate open access versionFindings
  • G. Litjens et al., A survey on deep learning in medical image analysis. Medical Image Analysis. 42, 60–88 (2017).
    Google ScholarLocate open access versionFindings
  • A. Graves, A. Mohamed, G. Hinton, in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on (IEEE, 2013), pp. 6645–6649.
    Google ScholarFindings
  • K. Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
    Findings
  • A. Krizhevsky, I. Sutskever, G. E. Hinton, in Advances in neural information processing systems (2012), pp. 1097–1105.
    Google ScholarFindings
  • D. Silver et al., Mastering the game of Go with deep neural networks and tree search. Nature. 529, 484–489 (2016).
    Google ScholarLocate open access versionFindings
  • U. S. Kamilov et al., Learning approach to optical tomography. Optica. 2, 517 (2015).
    Google ScholarLocate open access versionFindings
  • Y. Rivenson et al., Deep learning microscopy. Optica. 4, 1437 (2017).
    Google ScholarLocate open access versionFindings
  • K. H. Jin, M. T. McCann, E. Froustey, M. Unser, Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing. 26, 4509–4522 (2017).
    Google ScholarLocate open access versionFindings
  • Y. Rivenson, Y. Zhang, H. Gunaydin, D. Teng, A. Ozcan, Phase recovery and holographic image reconstruction using deep learning in neural networks. Light: Science & Applications. 7, 17141 (2018).
    Google ScholarLocate open access versionFindings
  • A. Sinha, J. Lee, S. Li, G. Barbastathis, Lensless computational imaging through deep learning. Optica. 4, (2017).
    Google ScholarFindings
  • K. Hammernik et al., Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine (2017), doi:10.1002/mrm.26977.
    Locate open access versionFindings
  • Y. Rivenson et al., Deep learning enhanced mobile-phone microscopy. ACS Photonics (2018), doi:10.1021/acsphotonics.8b00146.
    Locate open access versionFindings
  • Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition. Proceedings of the IEEE. 86, 2278–2324 (1998).
    Google ScholarLocate open access versionFindings
  • Y. Shen et al., Deep learning with coherent nanophotonic circuits. Nature Photonics. 11, 441–446 (2017).
    Google ScholarLocate open access versionFindings
  • D. Psaltis, D. Brady, X.-G. Gu, S. Lin, Holography in artificial neural networks. Nature. 343, 325–330 (1990).
    Google ScholarLocate open access versionFindings
  • K. H. Wagner, in Frontiers in Optics (Optical Society of America, 2017), pp. FW2C–1.
    Google ScholarFindings
  • B. J. Shastri et al., Principles of neuromorphic photonics. arXiv preprint arXiv:1801.00016 (2017).
    Findings
  • M. Hermans, M. Burm, T. Van Vaerenbergh, J. Dambre, P. Bienstman, Trainable hardware for dynamical computing using error backpropagation through physical media. Nature Communications. 6, 6729 (2015).
    Google ScholarLocate open access versionFindings
  • D. Brunner, M. C. Soriano, C. R. Mirasso, I. Fischer, Parallel photonic information processing at gigabyte per second data rates using transient states. Nature Communications. 4, 1364 (2013).
    Google ScholarLocate open access versionFindings
  • R. Girshick, J. Donahue, T. Darrell, J. Malik, in Proceedings of the IEEE conference on computer vision and pattern recognition (2014), pp. 580–587.
    Google ScholarLocate open access versionFindings
  • Y. Chen, Z. Lin, X. Zhao, G. Wang, Y. Gu, Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 7, 2094–2107 (2014).
    Google ScholarLocate open access versionFindings
  • J. W. Goodman, Introduction to Fourier optics (Roberts and Company Publishers, 2005). 24. http://www.image-net.org/25. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research. 15, 1929–1958 (2014).
    Locate open access versionFindings
  • 26. A. Greenbaum et al., Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nature Methods. 9, 889–895 (2012).
    Google ScholarLocate open access versionFindings
  • 27. A. Greenbaum et al., Wide-field computational imaging of pathology slides using lens-free on-chip microscopy. Science Translational Medicine. 6, 267ra175 (2014).
    Google ScholarLocate open access versionFindings
  • 28. A. Ozcan, E. McLeod, Lensless imaging and sensing. Annual Review of Biomedical Engineering. 18, 77–
    Google ScholarLocate open access versionFindings
  • 29. A. Greenbaum et al., Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy. Scientific Reports. 3, 1717 (2013). 30. https://www.nanoscribe.de/en/31. D. P. Kingma, J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 15 (2014).
    Findings
  • 32. M. Kazhdan, H. Hoppe, Screened poisson surface reconstruction. ACM Transactions on Graphics. 32, 1–13 (2013).
    Google ScholarLocate open access versionFindings
  • 33. http://www.meshlab.net 34. P. Cignoni et al., Meshlab:an open-source mesh processing tool. Eurographics Italian Chapter Conference, 129–136 (2008).
    Locate open access versionFindings
  • 35. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 13, 600–612 (2004).
    Google ScholarLocate open access versionFindings
  • 36. D. Grischkowsky, S. Keiding, M. Van Exter, C. Fattinger, Far-infrared time-domain spectroscopy with terahertz beams of dielectrics and semiconductors. JOSA B. 7, 2006–2015 (1990).
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments