Evolutionary Algorithms (EAs) play a crucial role in the architectural configuration and training of Artificial Deep Neural Networks (DNNs), a process known as neuroevolution. However, neuroevolution is hindered by its inherent computational expense, requiring multiple generations, a large population, and numerous epochs. The most computationally intensive aspect lies in evaluating the fitness function of a single candidate solution. To address this challenge, we employ Surrogate-assisted EAs (SAEAs). While a few SAEAs approaches have been proposed in neuroevolution, none have been applied to truly large DNNs due to issues like intractable information usage. In this work, drawing inspiration from Genetic Programming semantics, we use phenotypic distance vectors, outputted from DNNs, alongside Kriging Partial Least Squares (KPLS), an approach that is effective in handling these large vectors, making them suitable for search. Our proposed approach, named Neuro-Linear Genetic Programming surrogate model (NeuroLGP-SM), efficiently and accurately estimates DNN fitness without the need for complete evaluations. NeuroLGP-SM demonstrates competitive or superior results compared to 12 other methods, including NeuroLGP without SM, convolutional neural networks, support vector machines, and autoencoders. Additionally, it is worth noting that NeuroLGP-SM is 25% more energy-efficient than its NeuroLGP counterpart. This efficiency advantage adds to the overall appeal of our proposed NeuroLGP-SM in optimising the configuration of large DNNs.

中文翻译：

---

NeuroLGP-SM：深度神经网络的可扩展代理辅助神经进化

进化算法 (EA) 在人工深度神经网络 (DNN) 的架构配置和训练中发挥着至关重要的作用，这一过程称为神经进化。然而，神经进化受到其固有的计算成本的阻碍，需要多代、大量人口和无数时期。计算量最大的方面在于评估单个候选解的适应度函数。为了应对这一挑战，我们采用代理辅助 EA (SAEA)。虽然神经进化中已经提出了一些 SAEA 方法，但由于难以处理的信息使用等问题，没有一种方法能够应用于真正的大型 DNN。在这项工作中，从遗传编程语义中汲取灵感，我们使用从 DNN 输出的表型距离向量以及克里金偏最小二乘法 (KPLS)，这是一种有效处理这些大向量的方法，使它们适合搜索。我们提出的方法称为神经线性遗传规划代理模型 (NeuroLGP-SM)，可以高效、准确地估计 DNN 适合度，而无需进行完整评估。与其他 12 种方法（包括不带 SM 的 NeuroLGP、卷积神经网络、支持向量机和自动编码器）相比，NeuroLGP-SM 表现出具有竞争力或优越的结果。此外，值得注意的是，NeuroLGP-SM 的能效比 NeuroLGP 同类产品高 25%。这种效率优势增加了我们提出的 NeuroLGP-SM 在优化大型 DNN 配置方面的整体吸引力。