当前位置: X-MOL 学术IEEE Signal Process. Lett. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Event-Based Shutter Unrolling and Motion Deblurring in Dynamic Scenes
IEEE Signal Processing Letters ( IF 3.9 ) Pub Date : 2024-03-26 , DOI: 10.1109/lsp.2024.3381894
Yangguang Wang 1 , Chenxu Jiang 1 , Xu Jia 2 , Yufei Guo 3 , Lei Yu 1
Affiliation  

The Rolling Shutter (RS) effect and motion blur are common challenges in images captured by CMOS cameras during dynamic scenes. Inspired by biological vision principles, event cameras capture intensity changes asynchronously with low latency, providing valuable insights into image degradation during exposure. This study addresses the dual challenges of rolling shutter correction and deblurring using event data, merging them into a unified one-stage network. This streamlined approach reduces cumulative errors and inference time compared to traditional two-stage methods. To achieve this, we introduce an Event Representation for Rolling Shutter Deblurring, which explicitly models the conversion relationship between the input RS blurry frame and the latent image using events. To enhance the fusion of image and event information, we present a Time-guided Cross-Modal Attention module. Furthermore, we improve performance by incorporating a Multi-Scale Context-Aware Transformer Block, effectively addressing varying degrees of distortion and blurriness using a multi-scale attention mechanism. Extensive experiments validate that our method outperforms existing state-of-the-art approaches.

中文翻译:

动态场景中基于事件的快门展开和运动去模糊

滚动快门 (RS) 效果和运动模糊是 CMOS 相机在动态场景中捕获图像时面临的常见挑战。受生物视觉原理的启发,事件相机以低延迟异步捕获强度变化,为曝光期间的图像退化提供有价值的见解。这项研究解决了使用事件数据进行卷帘快门校正和去模糊的双重挑战,将它们合并到一个统一的单级网络中。与传统的两阶段方法相比,这种简化的方法减少了累积错误和推理时间。为了实现这一目标,我们引入了滚动快门去模糊的事件表示,它使用事件显式地模拟输入 RS 模糊帧和潜在图像之间的转换关系。为了增强图像和事件信息的融合,我们提出了时间引导的跨模态注意力模块。此外,我们通过合并多尺度上下文感知变压器块来提高性能,使用多尺度注意力机制有效地解决不同程度的失真和模糊问题。大量的实验验证了我们的方法优于现有的最先进的方法。
更新日期:2024-03-26
down
wechat
bug