When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This “Algorithm Journey Map” enumerates all social and technical activities throughout the AI solution’s procurement, development, integration, and full lifecycle management. In addition to mapping the “who?” and “what?” of the adoption of the AI tool, we also show several ‘lessons learned’ throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles—in algorithmic systems.

在医疗保健环境中集成人工智能工具时，技术和主要用户之间复杂的交互并不总是完全理解或可见。这种缺陷和模糊的理解阻碍了医疗保健组织采用人工智能/机器学习的尝试，也为研究人员寻找简化采用和开发使用基于人工智能的解决方案的最佳实践的机会带来了新的挑战。我们的研究通过记录杜克大学医疗系统设计、构建和维护名为 SepsisWatch 的人工智能解决方案的过程来填补这一空白。我们对工程师和科学家团队进行了 20 次采访，他们领导了多年来构建该工具、将其集成到实践中并维护解决方案的工作。这张“算法旅程图”列举了整个人工智能解决方案的采购、开发、集成和全生命周期管理的所有社会和技术活动。除了映射“谁？”和“什么？”在采用人工智能工具的过程中，我们还展示了整个算法旅程地图中的一些“经验教训”，包括建模假设、利益相关者包容性和组织结构。在此过程中，我们确定了有关如何识别和克服医疗保健环境中采用人工智能/机器学习的障碍的普遍见解。我们期望这项工作将进一步发展在算法系统中实施和维护道德原则的最佳实践。