Smart in-memory light sensors perform image recognition

Smart digital image sensors that can perform visual perception capabilities, such as scene recognition, are resulting from recent research at KAUST.

Exploiting the Nobel-Prize-winning technology of the charge-coupled device (CCD) image sensors found in early digital cameras, Dayanand Kumar, Nazek El-Atab and co-workers have adapted and enhanced the CCD’s core structure to create light-sensitive memory devices that can be programmed by light^[1]. In particular, the research team embedded the two-dimensional material MoS₂ into a semiconductor capacitor (MOSCAP) structure that underpins the charge-storing pixels of a CCD sensor.

The resulting Al/Al₂O₃/MoS₂/Al₂O₃/Si MOSCAP structures function as a charge-trapping “in-memory” sensor that is sensitive to visible light and can be programmed optically and erased electrically.

“The in-memory light sensors are smart multifunctional memory devices that can perform the roles of multiple — traditionally discrete — devices at once, including optical sensing, storage and computation,” explained El-Atab.

“Our long-term goal is to be able to demonstrate in-memory sensors that can detect different stimuli and compute,” El-Atab explains. “This overcomes the memory wall and allows for faster and more real-time data analysis using reduced power consumption, which is a requirement in many futuristic and state-of-the-art applications such as Internet of Things, autonomous cars and artificial intelligence, among others.”

Experiments with light with a wavelength anywhere in the blue to the red spectral region indicate that a photo-generated charge can be trapped or stored with an extremely long-lived retention time. The resulting “memory window” voltage of >2V can be stored for up to 10 years prior to being electrically erased by applying a +/-6V signal. Moreover, it can be operated for many millions of cycles.

The ultimate aim of the research is to create a single optoelectronic device that can perform optical sensing and storage with computing capabilities.

By combining their MoS₂ MOSCAP structure with a neural network, the team showed that it was possible to perform simple binary image recognition, successfully distinguishing between images of either a dog or an automobile, with an accuracy of 91 percent. Each image was 32×32 pixels in size, and only the blue information from the images was extracted since that corresponds to the device’s peak sensitivity.

The team would now like to further develop the level of optical control. “Current memory devices can be programmed optically but require erasing electrically,” commented Kumar. “In the future, we would like to explore in-memory optical sensors that can be fully optically operated.”

In related work, also on the topic of using optoelectronics to perform tasks for artificial visual perception, the team has explored the use of black phosphorus to create an optoelectronic memristive synapse that mimics the brain’s neurons for neuromorphic computing applications^[2].

Their multilayer device consists of a thin layer of black phosphorus and hafnium oxide that is sandwiched between a lower layer of platinum and an upper layer of copper. It operates as an optoelectronic memristor — a resistor that can have its electrical resistance programmed by visible light.

Experiments indicate that it offers highly stable synaptic features, such as long-term potentiation (a long-lasting increase in signal output), long-term depression (a long-lasting decrease in signal output) and short-term plasticity (change in response over time), which are all important neuronal behaviors.

The team constructed a 6×6 synaptic array from the devices, and in the future they hope that larger arrays could help realize a biomimetic retina. Importantly, the devices can be fabricated cost effectively by solution processing and are flexible with stable operation with a bend radius of 1 centimeter, offering possibilities for wearable applications.