Tensors are a popular programming interface for developing artificial intelligence (AI) algorithms. Layout refers to the order of placing tensor data in the memory and will affect performance by affecting data locality; therefore the deep neural network library has a convention on the layout. Since AI applications can use arbitrary layouts, and existing AI systems do not provide programming abstractions to shield the layout conventions of libraries, operator developers need to write a lot of layout-related code, which reduces the efficiency of integrating new libraries or developing new operators. Furthermore, the developer assigns the layout conversion operation to the internal operator to deal with the uncertainty of the input layout, thus losing the opportunity for layout optimization. Based on the idea of polymorphism, we propose a layout-agnostic virtual tensor programming interface, namely the VTensor framework, which enables developers to write new operators without caring about the underlying physical layout of tensors. In addition, the VTensor framework performs global layout inference at runtime to transparently resolve the required layout of virtual tensors, and runtime layout-oriented optimizations to globally minimize the number of layout transformation operations. Experimental results demonstrate that with VTensor, developers can avoid writing layout-dependent code. Compared with TensorFlow, for the 16 operations used in 12 popular networks, VTensor can reduce the lines of code (LOC) of writing a new operation by 47.82% on average, and improve the overall performance by 18.65% on average.

张量是一种用于开发人工智能 (AI) 算法的流行编程接口。布局是指张量数据在内存中的放置顺序，会通过影响数据局部性来影响性能；因此深度神经网络库在布局上有一个约定。由于AI应用可以使用任意布局，而现有的AI系统没有提供编程抽象来屏蔽库的布局约定，因此算子开发人员需要编写大量布局相关的代码，这降低了集成新库或开发新算子的效率。此外，开发者将布局转换操作分配给内部算子来处理输入布局的不确定性，从而失去了布局优化的机会。基于多态性的思想，我们提出了一种与布局无关的虚拟张量编程接口，即VTensor框架，它使开发人员能够编写新的运算符，而无需关心张量的底层物理布局。此外，VTensor框架在运行时执行全局布局推理，以透明地解析所需的虚拟张量布局，并进行运行时面向布局的优化，以全局最小化布局转换操作的数量。实验结果表明，使用 VTensor，开发人员可以避免编写依赖于布局的代码。与TensorFlow相比，对于12种流行网络中使用的16种运算，VTensor可以平均减少编写新运算的代码行数（LOC）47.82%，整体性能平均提高18.65%。