The core of sentence classification is to extract sentence semantic features. The existing hybrid methods have huge parameters and complex models. Due to the limited dataset, these methods are prone to feature redundancy and overfitting. To address this issue, this paper proposes an orthogonal independent Bi-GRU sentence classification model with multi-scale feature extraction, called Multi-scale Orthogonal Independent Bi-GRU (MODE-Bi-GRU). First, the hidden state of the Bi-GRU model is split into multiple small hidden states, and the corresponding recursive matrix is constrained orthogonally. Then, multiple sliding windows of different sizes are defined according to the forward and reverse angles of the sentence, and the sliding window is obtained. Finally, different sentence fragments are superimposed and input to the model, and the output results of multiple small Bi-GRU models are spliced and processed by soft pooling. The improved focal loss function is adopted to speed up the convergence of the model. Compared to the existing models, our proposed model achieves better results on four benchmark datasets, and it has better generalization ability with fewer parameters.

句子分类的核心是提取句子语义特征。现有的混合方法参数庞大、模型复杂。由于数据集有限，这些方法容易出现特征冗余和过拟合。为了解决这个问题，本文提出了一种具有多尺度特征提取的正交独立Bi-GRU句子分类模型，称为多尺度正交独立Bi-GRU（MODE-Bi-GRU）。首先，将Bi-GRU模型的隐藏状态分割成多个小的隐藏状态，并正交约束相应的递归矩阵。然后根据句子的正反角度定义多个不同大小的滑动窗口，得到滑动窗口。最后将不同的句子片段叠加输入到模型中，将多个小型Bi-GRU模型的输出结果进行软池化拼接处理。采用改进的焦点损失函数来加速模型的收敛。与现有模型相比，我们提出的模型在四个基准数据集上取得了更好的结果，并且以更少的参数具有更好的泛化能力。