It is an important aspect to fully leverage complementary sensors of images and point clouds for objects classification and six-dimensional (6D) pose estimation tasks. Prior works extract objects category from a single sensor such as RGB camera or LiDAR, limiting their robustness in the event that a key sensor is severely blocked or fails. In this work, we present a robust objects classification and 6D object pose estimation strategy by dual fusion of image and point cloud data. Instead of solely relying on 3D proposals or mature 2D object detectors, our model deeply integrates 2D and 3D information of heterogeneous data sources by a robustness dual fusion network and an attention-based nonlinear fusion function Attn-fun(.), achieving efficiency as well as high accuracy classification for even missed some data sources. Then, our method is also able to precisely estimate the transformation matrix between two input objects by minimizing the feature difference to achieve 6D object pose estimation, even under strong noise or with outliers. We deploy our proposed method not only to ModelNet40 datasets but also to a real fusion vision rotating platform for tracking objects in outer space based on the estimated pose.

充分利用图像和点云的互补传感器进行物体分类和六维（6D）姿态估计任务是一个重要方面。先前的工作从 RGB 相机或 LiDAR 等单个传感器中提取对象类别，从而限制了它们在关键传感器严重阻塞或故障时的鲁棒性。在这项工作中，我们通过图像和点云数据的双重融合提出了一种鲁棒的对象分类和 6D 对象姿态估计策略。我们的模型不是仅仅依赖 3D 提案或成熟的 2D 目标检测器，而是通过鲁棒性双融合网络和基于注意力的非线性融合函数 Attn-fun(.) 深度集成异构数据源的 2D 和 3D 信息，同时实现了效率即使错过了一些数据源，也能进行高精度分类。然后，我们的方法还能够通过最小化特征差异来精确估计两个输入对象之间的变换矩阵，以实现 6D 对象姿态估计，即使在强噪声或异常值的情况下也是如此。我们不仅将我们提出的方法部署到 ModelNet40 数据集，而且还部署到一个真实的融合视觉旋转平台，用于根据估计的姿态跟踪外层空间中的物体。