This article presents a model-free Q-Learning algorithm for addressing the optimal control problem in cyber–physical systems (CPS) exposed to denial-of-service (DoS) attacks and false data injection (FDI) attacks. The problem is formulated as a non-cooperative game within the framework of the Stackelberg game, in which the control strategy acts as the leader, while the FDI attacks strategy serves as the follower. Guided by the principle of optimality, we derive the optimal control policy, which depends on the solution of an associated game algebraic Riccati equation (GARE). Moreover, we formulate adequate conditions ensuring the presence of a solution to the GARE. To locate this solution, we employ a Q-Learning algorithm, eliminating the necessity for knowledge of system dynamics and state. Ultimately, we provide simulation results that demonstrate the effectiveness of our proposed approach.

中文翻译：

---

混合攻击下网络物理系统的数据驱动安全控制：Stackelberg 博弈方法

本文提出了一种无模型 Q-Learning 算法，用于解决遭受拒绝服务 (DoS) 攻击和虚假数据注入 (FDI) 攻击的网络物理系统 (CPS) 中的最优控制问题。该问题被表述为 Stackelberg 博弈框架内的非合作博弈，其中控制策略充当领导者，而 FDI 攻击策略充当跟随者。在最优性原理的指导下，我们推导出最优控制策略，该策略取决于相关博弈代数 Riccati 方程（GARE）的解。此外，我们制定了充分的条件，确保 GARE 的解决方案存在。为了找到这个解决方案，我们采用了 Q-Learning 算法，消除了对系统动力学和状态知识的需要。最终，我们提供模拟结果来证明我们提出的方法的有效性。