Artificially intelligent systems are nowadays presented as systems that should, among other things, be explainable and ethical. In parallel, both in the popular culture and within the scientific literature, there is a tendency to anthropomorphize Artificial Intelligence (AI) and reify intelligent systems as persons. From the perspective of machine ethics and ethical AI, this has resulted in the belief that truly autonomous ethical agents (i.e., machines and algorithms) can be defined, and that machines could, by themselves, behave ethically and perform actions that are justified (explainable) from a normative (ethical) standpoint. Under this assumption, and given that utilities and risks are generally seen as quantifiable, many scholars have seen consequentialism (or utilitarianism) and rational choice theory as likely candidates to be implemented in automated ethical decision procedures, for instance to assess and manage risks as well as maximize expected utility. While some see this implementation as unproblematic, there are important limitations to such attempts that need to be made explicit so that we can properly understand what artificial autonomous ethical agents are, and what they are not. From the perspective of explainable AI, there are value-laden technical choices made during the implementation of automated ethical decision procedures that cannot be explained as decisions made by the system. Building on a recent example from the machine ethics literature, we use computer simulations to study whether autonomous ethical agents can be considered as explainable AI systems. Using these simulations, we argue that technical issues with ethical ramifications leave room for reasonable disagreement even when algorithms are based on ethical and rational foundations such as consequentialism and rational choice theory. By doing so, our aim is to illustrate the limitations of automated behavior and ethical AI and, incidentally, to raise awareness on the limits of so-called autonomous ethical agents.

如今，人工智能系统被认为应该是可解释且符合道德的系统。与此同时，在流行文化和科学文献中，都存在将人工智能拟人化并将智能系统具体化为人的趋势。从机器伦理和人工智能伦理的角度来看，这导致人们相信真正自主的伦理主体（即机器和算法）是可以被定义的，并且机器本身可以道德地行事并执行合理的行为（可解释的）。 ）从规范（道德）的角度来看。在这种假设下，鉴于效用和风险通常被认为是可量化的，许多学者将结果主义（或功利主义）和理性选择理论视为在自动道德决策程序中实施的可能候选者，例如评估和管理风险作为最大化预期效用。虽然有些人认为这种实施没有问题，但这种尝试存在重要的局限性，需要明确这些局限性，以便我们能够正确理解什么是人工自主道德主体，什么不是。从可解释人工智能的角度来看，在自动化道德决策程序的实施过程中做出了一些有价值的技术选择，但这些选择无法解释为系统做出的决策。基于机器伦理文献中最近的一个例子，我们使用计算机模拟来研究自主伦理代理是否可以被视为可解释的人工智能系统。通过这些模拟，我们认为，即使算法基于结果主义和理性选择理论等伦理和理性基础，具有伦理影响的技术问题也为合理的分歧留下了空间。通过这样做，我们的目的是说明自动化行为和道德人工智能的局限性，顺便提一下，提高人们对所谓的自主道德代理人的局限性的认识。