We propose an approach to manipulate existing interactive visualizations to answer users' natural language queries. We analyze the natural language tasks and propose a design space of a hierarchical task structure, which allows for a systematic decomposition of complex queries. We introduce a four-level visualization manipulation space to facilitate in-situ manipulations for visualizations, enabling a fine-grained control over the visualization elements. Our methods comprise two essential components: the natural language-to-task translator and the visualization manipulation parser. The natural language-to-task translator employs advanced NLP techniques to extract structured, hierarchical tasks from natural language queries, even those with varying degrees of ambiguity. The visualization manipulation parser leverages the hierarchical task structure to streamline these tasks into a sequence of atomic visualization manipulations. To illustrate the effectiveness of our approach, we provide real-world examples and experimental results. The evaluation highlights the precision of our natural language parsing capabilities and underscores the smooth transformation of visualization manipulations.

中文翻译：

---

为现有可视化注入新的生命：自然语言驱动的操作框架

我们提出了一种操纵现有交互式可视化来回答用户自然语言查询的方法。我们分析自然语言任务并提出分层任务结构的设计空间，该设计空间允许对复杂查询进行系统分解。我们引入了四级可视化操作空间，以促进可视化的原位操作，从而实现对可视化元素的细粒度控制。我们的方法包含两个基本组件：自然语言到任务翻译器和可视化操作解析器。自然语言到任务的翻译器采用先进的 NLP 技术从自然语言查询中提取结构化、分层的任务，甚至是那些具有不同程度歧义的任务。可视化操作解析器利用分层任务结构将这些任务简化为一系列原子可视化操作。为了说明我们方法的有效性，我们提供了现实世界的例子和实验结果。该评估凸显了我们自然语言解析能力的精确性，并强调了可视化操作的平滑转换。