Abstract
IPC (inter-process communication) is a critical mechanism for modern OSes, including not only microkernels such as seL4, QNX, and Fuchsia where system functionalities are deployed in user-level processes, but also monolithic kernels like Android where apps frequently communicate with plenty of user-level services. However, existing IPC mechanisms still suffer from long latency. Previous software optimizations of IPC usually cannot bypass the kernel that is responsible for domain switching and message copying/remapping across different address spaces; hardware solutions such as tagged memory or capability replace page tables for isolation, but usually require non-trivial modification to existing software stack to adapt to the new hardware primitives. In this article, we propose a hardware-assisted OS primitive, XPC (Cross Process Call), for efficient and secure synchronous IPC. XPC enables direct switch between IPC caller and callee without trapping into the kernel and supports secure message passing across multiple processes without copying. We have implemented a prototype of XPC based on the ARM AArch64 with Gem5 simulator and RISC-V architecture with FPGA boards. The evaluation shows that XPC can reduce IPC call latency from 664 to 21 cycles, 14×–123× improvement on Android Binder (ARM), and improve the performance of real-world applications on microkernels by 1.6× on Sqlite3.
- [1] Arm Inc. 2018. Arm System Modeling Research Enablement Kit. Retrieved from https://developer.arm.com/research/research-enablement/system-modeling.Google Scholar
- [2] Google Inc. 2018. Fuchsia. Retrieved from https://fuchsia.googlesource.com/zircon.Google Scholar
- [3] Intel Inc. 2018. An Introduction to the Intel QuickPath Interconnect. Retrieved from https://www.intel.de/content/dam/doc/white-paper/quick-path-interconnect-introduction-paper.pdf.Google Scholar
- [4] lowRISC Project. 2018. lowRISC. Retrieved from https://www.lowrisc.org/.Google Scholar
- [5] lwIP Project. 2018. lwIP. Retrieved from https://savannah.nongnu.org/projects/lwip/.
Referenced May 2018. Google Scholar - [6] seL4 Project. 2018. seL4 Benchmark. Retrieved from https://sel4.systems/About/Performance.Google Scholar
- [7] SQLite Project. 2018. SQLite. Retrieved from https://www.sqlite.org/index.html.
Referenced May 2018. Google Scholar - [8] Xilinx Inc. 2018. Vivado Design Suite. Retrieved from https://www.xilinx.com/products/design-tools/vivado.html.
Referenced August 2018. Google Scholar - [9] John Stultz. 2019. Anonymous shared memory (ashmem) subsystem [LWN.net]. Retrieved from https://lwn.net/Articles/452035/.Google Scholar
- [10] Dianne Hackborn. 2019. LKML: Dianne Hackborn: Re: [PATCH 1/6] staging: android: binder: Remove some funny usage. Retrieved from https://lkml.org/lkml/2009/6/25/3.Google Scholar
- [11] Common Weakness Enumeration. 2021. CWE-367: Time-of-check Time-of-use (TOCTOU) Race Condition (4.5). Retrieved from https://cwe.mitre.org/data/definitions/367.html.Google Scholar
- [12] Fiasco.OC Project. 2021. The Fiasco microkernel - Overview. Retrieved from https://os.inf.tu-dresden.de/fiasco/.
Referenced Oct. 2021. Google Scholar - [13] Barrelfish Project. 2021. Message Notifications, Barrelfish Technical Note 9. http://www.barrelfish.org/publications/TN-009-Notifications.pdf.Google Scholar
- [14] SiFive Inc. 2021. SiFive. Retrieved from https://www.sifive.com/.Google Scholar
- [15] Sohil Mehta. 2021. User Interrupts: A faster way to signal. Retrieved from https://linuxplumbersconf.org/event/11/contributions/985/attachments/756/1417/User_Interrupts_LPC_2021.pdf.Google Scholar
- [16] seL4 Project. 2022. seL4 Dynamic Libraries: IPC. Retrieved from https://docs.sel4.systems/Tutorials/dynamic-2.html.Google Scholar
- [17] . 2020. Don’t shoot down TLB shootdowns! In
EuroSys’20 ). Association for Computing Machinery.DOI: Google ScholarDigital Library - [18] . 2016. The Rocket Chip Generator. Technical Report. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2016-17.Google Scholar
- [19] . 2016. M3: A hardware/operating-system co-design to tame heterogeneous manycores. In ASPLOS. ACM, New York, NY.Google Scholar
- [20] . 2009. The multikernel: A new OS architecture for scalable multicore systems. In ACM SIGOPS.Google Scholar
- [21] . 1990. Lightweight remote procedure call. ACM Trans. Comput. Syst. 8, 1 (1990), 37–55.Google ScholarDigital Library
- [22] . 1991. User-level interprocess communication for shared memory multiprocessors. ACM Trans. Comput. Syst. 9, 2 (1991), 175–198.Google ScholarDigital Library
- [23] . 2011. The Gem5 simulator. SIGARCH Comput. Archit. News 39, 2 (Aug. 2011), 1–7.Google ScholarDigital Library
- [24] . 1994. Hardware support for fast capability-based addressing. In ACM SIGPLAN Notices. ACM.Google Scholar
- [25] . 1994. Sharing and protection in a single-address-space operating system. ACM Trans. Comput. Syst. 12, 4 (1994).Google ScholarDigital Library
- [26] . 2015. Using crash Hoare logic for certifying the FSCQ file system. In SOSP.Google Scholar
- [27] . 1992. An architectural overview of the Alpha real-time distributed kernel. In USENIX Workshop on Microkernels and other Kernel Architectures.Google Scholar
- [28] . 2008. CuriOS: Improving reliability through operating system structure. In OSDI.Google Scholar
- [29] . 2013. From L3 to seL4 what have we learnt in 20 years of L4 microkernels? In SOSP.Google Scholar
- [30] 1995. Exokernel: An operating system architecture for application-level resource management. In SOSP’95. ACM, New York, NY.Google Scholar
- [31] . 1994. Evolving Mach 3.0 to a migrating thread model. In USENIX Winter.Google Scholar
- [32] . 1999. Tornado: Maximizing locality and concurrency in a shared memory multiprocessor operating system. In OSDI, Vol. 99. 87–100.Google Scholar
- [33] . 1997. The performance of \( \mu \)-kernel-based systems. In ACM SIGOPS Operating Systems Review, Vol. 31. ACM.Google Scholar
- [34] . 2009. seL4: Formal verification of an OS kernel. In ACM SIGOPS.Google Scholar
- [35] . 1992. Architecture Support for Single Address Space Operating Systems. Vol. 27. ACM.Google ScholarDigital Library
- [36] . 2011. HAQu: Hardware-accelerated queueing for fine-grained threading on a chip multiprocessor. In HPCA.Google Scholar
- [37] . 1984. Capability-based Computer Systems. Digital Press.Google ScholarDigital Library
- [38] . 2015. Reducing world switches in virtualized environment with flexible cross-world calls. In ISCA.Google Scholar
- [39] . 1993. Improving IPC by kernel design. ACM SIGOPS Oper. Syst. Rev. 27, 5 (Dec. 1993), 175–188.Google ScholarDigital Library
- [40] . 1993. A persistent system in real use-experiences of the first 13 years. In 3rd International Workshop on Object Orientation in Operating Systems. IEEE.Google Scholar
- [41] . 1995. On Micro-kernel Construction. Vol. 29. ACM.Google ScholarDigital Library
- [42] . 1997. Achieved IPC performance (still the foundation for extensibility). In 6th Workshop on Hot Topics in Operating Systems. IEEE.Google Scholar
- [43] . 2018. Scheduling-context capabilities: A principled, light-weight operating-system mechanism for managing time. In 13th EuroSys Conference. ACM.Google ScholarDigital Library
- [44] . 2018. DAMN: Overhead-free IOMMU protection for networking. In 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. ACM.Google ScholarDigital Library
- [45] . 1996. LMbench: Portable tools for performance analysis. In USENIX. 279–294.Google Scholar
- [46] . 2019. SkyBridge: Fast and secure inter-process communication for microkernels. In EuroSys. ACM.Google Scholar
- [47] . 2018. Solros: A data-centric operating system architecture for heterogeneous computing. In EuroSys. ACM.Google Scholar
- [48] . 2019. LXDs: Towards isolation of kernel subsystems. In USENIX. USENIX Association, 269–284. Retrieved from https://www.usenix.org/conference/atc19/presentation/narayanan.Google Scholar
- [49] . 2020. Lightweight kernel isolation with virtualization and VM functions. In
VEE’20 . Association for Computing Machinery, New York, NY, 157–171.DOI: Google ScholarDigital Library - [50] . 2019. libmpk: Software abstraction for Intel memory protection keys (Intel MPK). In USENIX. USENIX Association, 241–254. Retrieved from https://www.usenix.org/conference/atc19/presentation/park-soyeon.Google Scholar
- [51] . 1974. Protection and the control of information sharing in multics. Commun. ACM 17, 7 (1974), 388–402.Google ScholarDigital Library
- [52] . 1999. EROS: A fast capability system. Vol. 33. ACM.Google Scholar
- [53] . 2010. NOVA: A microhypervisor-based secure virtualization architecture. In EUROSYS.Google Scholar
- [54] . 2007. The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops). In ExpCS. ACM.Google Scholar
- [55] . 2014. CODOMs: Protecting software with code-centric memory domains. In ACM SIGARCH Computer Architecture News. IEEE Press.Google Scholar
- [56] . 2017. Direct inter-process communication (dIPC): Repurposing the CODOMs architecture to accelerate IPC. In EuroSys. ACM.Google Scholar
- [57] . 2014. The RISC-V Instruction Set Manual. Volume 1: User-Level ISA, Version 2.0.
Technical Report . California University Berkeley Department of Electrical Engineering and Computer Sciences.Google ScholarCross Ref - [58] . 2015. Cheri: A hybrid capability-system architecture for scalable software compartmentalization. In SP. IEEE.Google Scholar
- [59] . 2016. Fast protection-domain crossing in the CHERI capability-system architecture. IEEE Micro 36, 5 (2016), 38–49.Google ScholarDigital Library
- [60] . 2002. Mondrian memory protection. In ASPLOS. ACM, New York, NY.Google Scholar
- [61] . 2005. Mondrix: Memory isolation for Linux using Mondriaan memory protection. In SOSP. ACM.Google Scholar
Index Terms
- Boosting Inter-process Communication with Architectural Support
Recommendations
A Design to Adapt Microkernel Inter-process Communication Mechanism
ICIIP '18: Proceedings of the 3rd International Conference on Intelligent Information ProcessingIn order to improve the efficiency of inter-process communication among microkernel operating systems, this paper proposes an inter-process communication mechanism for microkernels. The mechanism runs IPC (Inter-Process Communication) as a set of ...
XPC: architectural support for secure and efficient cross process call
ISCA '19: Proceedings of the 46th International Symposium on Computer ArchitectureMicrokernel has many intriguing features like security, fault-tolerance, modularity and customizability, which recently stimulate a resurgent interest in both academia and industry (including seL4, QNX and Google's Fuchsia OS). However, IPC (inter-...
Micro-CLK: returning to the asynchronicity with communication-less microkernel
APSys '21: Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on SystemsInter-process communication (IPC) has always been the "Achilles heel" of microkernels, determining their overall performance. The entire history of microkernel development is tightly coupled to the debates about IPC, its efficiency, and the bottleneck ...
Comments