Yolov4-based hybrid feature enhancement network with robust object detection under adverse weather conditions

Abstract

Investigations into the behaviour of pedestrians and autonomous driving both frequently employ object detection. It has always been a popular area for research in computer vision and artificial intelligence. Because of the advancement of deep learning, object detectors are improving in accuracy and speed. However, the majority of them struggle to reconcile speed and precision. Because of this, the object detection model employed in this work, which is based on a better You Only Look Once Version 4 (YOLOv4) algorithm, considers both detection accuracy and efficiency. Several different sensor types, including cameras and mmWave (millimetre Wave) radar, are employed to observe the environment. This study employs a YOLOv4-based hybrid feature enhancement network for robust object detection under adverse weather conditions (AWC), utilizing the CRUW dataset. A camera–radar-fused approach that detects things that are cross-supervised by a deep radar object detection network, such as the YOLOv4 algorithm, reduces processing complexity. To correct category imbalance and further boost the robustness of the suggested model, the class-balanced sampling strategy (CBSS) is employed. Python software was employed to analyse the suggested method. In terms of mAP and accuracy, the suggested method is contrasted with the current OD ensemble framework and faster R-CNN methodologies. The suggested method’s accuracy and mAP are higher than those of the other existing methods; as a result, the proposed method’s mAP produces 40% and its accuracy produces 98.7%, respectively. Additionally, the suggested method performs better when classifying the object and detecting it, but it also strengthens the system’s stability and reduces category imbalance.

Appendix

Variables	Description
$\left( {\rho_{c} , \theta_{c} } \right)$	Projected location in range-azimuth coordinates
$\left( {x^{c} - x_{{{\text{or}}}} } \right)$	Place of the object in camera BEV coordinates
$\left( {x_{{{\text{or}}}} ,z_{{{\text{or}}}} } \right)$	Place of the radar origin in camera BEV coordinates
${\text{cls}}$	Object class
$d_{i}$	Depth of the object
$c_{i}$	Depth of confidence
$\delta \left( {{\text{cls}}} \right)$	Typical azimuth error for camera localization
$N\left( \cdot \right)$	Object’s probability map
$\epsilon \left( \cdot \right)$	Azimuth resolution of the radar
$\delta_{j}^{r}$	Range resolution
$P_{{\left( {{\text{cls}}} \right)}}^{{{\text{CRF}}}}$	Fused probability map
$P_{{{\text{local}}}}$	Point-wise features
$P$	Point cloud
$V$	Voxel features
$D$	Dimension
$R^{3}$	Voxel grids number
$B$	Fixed number
$h_{\theta }$	MLP’s shared learnable parameters
$W$	Width
$H$	Height
$C$	Channel count
$F$	Feature map
$\overline{F}^{w,h}$	Local surrounding region
$\overline{W}$	Width of local region
$\overline{H}$	Height of local region
$\left( {w, h} \right)$	Weights associated with each pixel
$P_{{{\text{local}}}}^{w,h}$	Attentive contextual feature
$\overline{f}_{i}^{w,h}$	Feature at the $i^{th}$ location
$\hat{\phi }_{j}$	Balanced softmax output
$\hat{\phi }_{y}$	Conditional probability of the correct class
$x^{ + }$	Positive sample
$x^{ - }$	Negative sample
$j$	Class
$D_{{{\text{train}}}}$	Training set
$1/n_{j}$	Inverse loss weight
$1/n_{j}^{2}$	Over-balance problem