We present the new distributed-memory run-time system (RTS) of the C++-based open-source structured parallel programming library FastFlow. The new RTS enables the execution of FastFlow shared-memory applications written using its Building Blocks (BBs) on distributed systems with minimal changes to the original program. The changes required are all high-level and deal with introducing distributed groups (dgroup), i.e., logical partitions of the BBs composing the application streaming graph. A dgroup, which in turn is implemented using FastFlow’s BBs, can be deployed and executed on a remote machine and communicate with other dgroups according to the original shared-memory FastFlow streaming programming model. We present how to define the distributed groups and how we faced the problem of data serialization and communication performance tuning through transparent messages’ batching and their scheduling. Finally, we present a study of the overhead introduced by dgroups considering some benchmarks on a sixteen-node cluster.

我们介绍了基于 C++ 的开源结构化并行编程库FastFlow的新分布式内存运行时系统 (RTS) 。新的 RTS 允许在分布式系统上执行使用其构建块( BB ) 编写的FastFlow共享内存应用程序，而对原始程序的更改最少。所需的更改都是高级的，涉及引入分布式组( dgroup )，即组成应用程序流图的 BB 的逻辑分区。一个dgroup，它又使用FastFlow的BBs实现，可以在远程机器上部署和执行，并根据原始共享内存FastFlow流式编程模型与其他dgroups通信。我们介绍了如何定义分布式组，以及我们如何通过透明消息的批处理和调度来解决数据序列化和通信性能调整的问题。最后，我们研究了dgroups引入的开销，考虑了 16 节点集群上的一些基准。