This work proposes a Processing-In-Sensor Accelerator, namely PISA, as a flexible, energy-efficient, and high-performance solution for real-time and smart image processing in AI devices. PISA intrinsically implements a coarse-grained convolution operation in Binarized-Weight Neural Networks (BWNNs) leveraging a novel compute-pixel with non-volatile weight storage at the sensor side. This remarkably reduces the power consumption of data conversion and transmission to an off-chip processor. The design is completed with a bit-wise near-sensor in-memory computing unit to process the remaining network layers. Once the object is detected, PISA switches to typical sensing mode to capture the image for a fine-grained convolution using only a near-sensor processing unit. Our circuit-to-application co-simulation results on a BWNN acceleration demonstrate minor accuracy degradation on various image datasets in coarse-grained evaluation compared to baseline BWNN models, while PISA achieves a frame rate of 1000 and efficiency of $\sim$ 1.74 TOp/s/W. Lastly, PISA substantially reduces data conversion and transmission energy by $\sim$ 84% compared to a baseline.

中文翻译：

---

PISA：用于成像系统的非易失性传感器内处理加速器

这项工作提出了一种传感器内处理加速器，即 PISA，作为一种灵活、节能、高性能的解决方案，用于人工智能设备中的实时智能图像处理。PISA 本质上在二值化权重神经网络 (BWNN) 中实现了粗粒度卷积运算，利用传感器端具有非易失性权重存储的新型计算像素。这显着降低了数据转换和传输到片外处理器的功耗。该设计通过按位近传感器内存计算单元来完成，以处理其余的网络层。一旦检测到物体，PISA 就会切换到典型的传感模式，仅使用近传感器处理单元来捕获图像以进行细粒度卷积。我们关于 BWNN 加速的电路到应用联合仿真结果表明，与基线 BWNN 模型相比，粗粒度评估中各种图像数据集的精度略有下降，而 PISA 实现了 1000 的帧速率和 1000 的效率$\sim$1.74 顶部/秒/瓦。最后，PISA 通过以下方式大幅降低了数据转换和传输能耗：$\sim$与基线相比为 84%。