Ultralow Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision.
Ragib Ahsan, Hyun Uk Chae, Seyedeh Atiyeh Abbasi Jalal, Zezhi Wu, Jun Tao, Subrata Das, Hefei Liu, Jiang-Bin Wu, Stephen B Cronin, Han Wang, Constantine Sideris, Rehan Kapadia
Author Information
Ragib Ahsan: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Hyun Uk Chae: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Seyedeh Atiyeh Abbasi Jalal: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States.
Zezhi Wu: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States.
Jun Tao: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States.
Subrata Das: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States.
Hefei Liu: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Jiang-Bin Wu: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Stephen B Cronin: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Han Wang: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
Constantine Sideris: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States.
Rehan Kapadia: Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States. ORCID
In-sensor and near-sensor computing architectures enable multiply accumulate operations to be carried out directly at the point of sensing. In-sensor architectures offer dramatic power and speed improvements over traditional von Neumann architectures by eliminating multiple analog-to-digital conversions, data storage, and data movement operations. Current in-sensor processing approaches rely on tunable sensors or additional weighting elements to perform linear functions such as multiply accumulate operations as the sensor acquires data. This work implements in-sensor computing with an oscillatory retinal neuron device that converts incident optical signals into voltage oscillations. A computing scheme is introduced based on the frequency shift of coupled oscillators that enables parallel, frequency multiplexed, nonlinear operations on the inputs. An experimentally implemented 3 × 3 focal plane array of coupled neurons shows that functions approximating edge detection, thresholding, and segmentation occur . An example of inference on handwritten digits from the MNIST database is also experimentally demonstrated with a 3 × 3 array of coupled neurons feeding into a single hidden layer neural network, approximating a liquid-state machine. Finally, the equivalent energy consumption to carry out image processing operations, including peripherals such as the Fourier transform circuits, is projected to be <20 fJ/OP, possibly reaching as low as 15 aJ/OP.
