Abstract

An appropriate pretreatment method is crucial to establish a reliable model for near-infrared spectroscopy. It is usually selected by comparing model performance with different pretreatment methods, such as the root mean square error of internal cross-validation or the root mean square error of prediction. When the statistical indexes of the models are similar and indistinguishable, or the results of model calibration and cross-validation are too different, indicating an overfitting situation, it may be due to the selection of an inappropriate pretreatment. In this paper, an approach is proposed to select appropriate pretreatment methods by chemical components valid and invalid absorption wavebands. Due to there being no chemical groups absorption in wavebands of 1250–1350 nm and 1780–1850 nm, they are considered chemical invalid absorption bands. The root mean square distance in the invalid wavebands characterizes the difference in samples’ physical state, such as particle size and instrument error. Except for the invalid absorption band, other wavebands in the near-infrared region are considered chemical valid absorption wavebands. The root mean square distance in the valid absorption wavebands mainly characterize the chemical composition differences between different samples. So, we define effective information rate of spectra. The larger effective information rate of spectra, the more favorable it is to establish a quantitative model. In this study, spectra of 60 wheat powder samples were used to compare seven pretreatment methods. Moreover, the average effective information rates of spectra were calculated for the seven pretreatment methods respectively. The results show that the combination of standard normal variate transformation and first derivative with Savitzky–Golay smoothing is the best. Finally, a comparison of model prediction with different pretreatment methods was made. The results are the same as the former. Therefore, using the effective information rate of spectra for selecting an appropriate near-infrared spectroscopy pretreatment method is practical and can be used not only for near-infrared modeling but also for designing near-infrared instruments as an index for optimization.

摘要

适当的预处理方法对于建立可靠的近红外光谱模型至关重要。通常通过比较模型性能与不同的预处理方法来选择，例如内部交叉验证的均方根误差或预测的均方根误差。当模型的统计指标相似且无法区分时，或者模型校准和交叉验证的结果相差太大，表明存在过拟合的情况，可能是由于选择了不合适的预处理。本文提出了一种通过化学成分有效和无效吸收波段选择合适的预处理方法的方法。由于在 1250-1350 nm 和 1780-1850 nm 波段没有化学基团吸收，因此它们被认为是化学无效吸收带。无效波段中的均方根距离表征样品物理状态的差异，例如粒度和仪器误差。除无效吸收波段外，近红外区的其他波段均视为化学有效吸收波段。有效吸收波段的均方根距离主要表征不同样品之间的化学成分差异。因此，我们定义了光谱的有效信息率。光谱的有效信息率越大，越有利于建立定量模型。在本研究中，使用 60 个小麦粉样品的光谱来比较七种预处理方法。此外，分别计算了七种预处理方法的光谱平均有效信息率。结果表明，标准正态变量变换和一阶导数与 Savitzky-Golay 平滑的组合是最好的。最后，将模型预测与不同预处理方法进行了比较。结果与前者相同。因此，利用光谱的有效信息率来选择合适的近红外光谱预处理方法具有实用性，不仅可以用于近红外建模，还可以作为优化近红外仪器的指标。