skip to main content
research-article

Boosting Inter-process Communication with Architectural Support

Authors Info & Claims
Published:05 July 2022Publication History
Skip Abstract Section

Abstract

IPC (inter-process communication) is a critical mechanism for modern OSes, including not only microkernels such as seL4, QNX, and Fuchsia where system functionalities are deployed in user-level processes, but also monolithic kernels like Android where apps frequently communicate with plenty of user-level services. However, existing IPC mechanisms still suffer from long latency. Previous software optimizations of IPC usually cannot bypass the kernel that is responsible for domain switching and message copying/remapping across different address spaces; hardware solutions such as tagged memory or capability replace page tables for isolation, but usually require non-trivial modification to existing software stack to adapt to the new hardware primitives. In this article, we propose a hardware-assisted OS primitive, XPC (Cross Process Call), for efficient and secure synchronous IPC. XPC enables direct switch between IPC caller and callee without trapping into the kernel and supports secure message passing across multiple processes without copying. We have implemented a prototype of XPC based on the ARM AArch64 with Gem5 simulator and RISC-V architecture with FPGA boards. The evaluation shows that XPC can reduce IPC call latency from 664 to 21 cycles, 14×–123× improvement on Android Binder (ARM), and improve the performance of real-world applications on microkernels by 1.6× on Sqlite3.

REFERENCES

  1. [1] Arm Inc. 2018. Arm System Modeling Research Enablement Kit. Retrieved from https://developer.arm.com/research/research-enablement/system-modeling.Google ScholarGoogle Scholar
  2. [2] Google Inc. 2018. Fuchsia. Retrieved from https://fuchsia.googlesource.com/zircon.Google ScholarGoogle Scholar
  3. [3] Intel Inc. 2018. An Introduction to the Intel QuickPath Interconnect. Retrieved from https://www.intel.de/content/dam/doc/white-paper/quick-path-interconnect-introduction-paper.pdf.Google ScholarGoogle Scholar
  4. [4] lowRISC Project. 2018. lowRISC. Retrieved from https://www.lowrisc.org/.Google ScholarGoogle Scholar
  5. [5] lwIP Project. 2018. lwIP. Retrieved from https://savannah.nongnu.org/projects/lwip/. Referenced May 2018.Google ScholarGoogle Scholar
  6. [6] seL4 Project. 2018. seL4 Benchmark. Retrieved from https://sel4.systems/About/Performance.Google ScholarGoogle Scholar
  7. [7] SQLite Project. 2018. SQLite. Retrieved from https://www.sqlite.org/index.html. Referenced May 2018.Google ScholarGoogle Scholar
  8. [8] Xilinx Inc. 2018. Vivado Design Suite. Retrieved from https://www.xilinx.com/products/design-tools/vivado.html. Referenced August 2018.Google ScholarGoogle Scholar
  9. [9] John Stultz. 2019. Anonymous shared memory (ashmem) subsystem [LWN.net]. Retrieved from https://lwn.net/Articles/452035/.Google ScholarGoogle Scholar
  10. [10] Dianne Hackborn. 2019. LKML: Dianne Hackborn: Re: [PATCH 1/6] staging: android: binder: Remove some funny usage. Retrieved from https://lkml.org/lkml/2009/6/25/3.Google ScholarGoogle Scholar
  11. [11] Common Weakness Enumeration. 2021. CWE-367: Time-of-check Time-of-use (TOCTOU) Race Condition (4.5). Retrieved from https://cwe.mitre.org/data/definitions/367.html.Google ScholarGoogle Scholar
  12. [12] Fiasco.OC Project. 2021. The Fiasco microkernel - Overview. Retrieved from https://os.inf.tu-dresden.de/fiasco/. Referenced Oct. 2021.Google ScholarGoogle Scholar
  13. [13] Barrelfish Project. 2021. Message Notifications, Barrelfish Technical Note 9. http://www.barrelfish.org/publications/TN-009-Notifications.pdf.Google ScholarGoogle Scholar
  14. [14] SiFive Inc. 2021. SiFive. Retrieved from https://www.sifive.com/.Google ScholarGoogle Scholar
  15. [15] Sohil Mehta. 2021. User Interrupts: A faster way to signal. Retrieved from https://linuxplumbersconf.org/event/11/contributions/985/attachments/756/1417/User_Interrupts_LPC_2021.pdf.Google ScholarGoogle Scholar
  16. [16] seL4 Project. 2022. seL4 Dynamic Libraries: IPC. Retrieved from https://docs.sel4.systems/Tutorials/dynamic-2.html.Google ScholarGoogle Scholar
  17. [17] Amit Nadav, Tai Amy, and Wei Michael. 2020. Don’t shoot down TLB shootdowns! In EuroSys’20). Association for Computing Machinery. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Asanovic Krste, Avizienis Rimas, Bachrach Jonathan, Beamer Scott, Biancolin David, Celio Christopher, Cook Henry, Dabbelt Daniel, Hauser John, Izraelevitz Adam, et al. 2016. The Rocket Chip Generator. Technical Report. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2016-17.Google ScholarGoogle Scholar
  19. [19] Asmussen Nils, Völp Marcus, Nöthen Benedikt, Härtig Hermann, and Fettweis Gerhard. 2016. M3: A hardware/operating-system co-design to tame heterogeneous manycores. In ASPLOS. ACM, New York, NY.Google ScholarGoogle Scholar
  20. [20] Baumann Andrew, Barham Paul, Dagand Pierre-Evariste, Harris Tim, Isaacs Rebecca, Peter Simon, Roscoe Timothy, Schüpbach Adrian, and Singhania Akhilesh. 2009. The multikernel: A new OS architecture for scalable multicore systems. In ACM SIGOPS.Google ScholarGoogle Scholar
  21. [21] Bershad Brian N., Anderson Thomas E., Lazowska Edward D., and Levy Henry M.. 1990. Lightweight remote procedure call. ACM Trans. Comput. Syst. 8, 1 (1990), 37–55.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Bershad Brian N., Anderson Thomas E., Lazowska Edward D., and Levy Henry M.. 1991. User-level interprocess communication for shared memory multiprocessors. ACM Trans. Comput. Syst. 9, 2 (1991), 175198.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Binkert Nathan, Beckmann Bradford, Black Gabriel, Reinhardt Steven K., Saidi Ali, Basu Arkaprava, Hestness Joel, Hower Derek R., Krishna Tushar, Sardashti Somayeh, Sen Rathijit, Sewell Korey, Shoaib Muhammad, Vaish Nilay, Hill Mark D., and Wood David A.. 2011. The Gem5 simulator. SIGARCH Comput. Archit. News 39, 2 (Aug. 2011), 1–7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Carter Nicholas P., Keckler Stephen W., and Dally William J.. 1994. Hardware support for fast capability-based addressing. In ACM SIGPLAN Notices. ACM.Google ScholarGoogle Scholar
  25. [25] Chase Jeffrey S., Levy Henry M., Feeley Michael J., and Lazowska Edward D.. 1994. Sharing and protection in a single-address-space operating system. ACM Trans. Comput. Syst. 12, 4 (1994).Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Chen Haogang, Ziegler Daniel, Chajed Tej, Chlipala Adam, Kaashoek M. Frans, and Zeldovich Nickolai. 2015. Using crash Hoare logic for certifying the FSCQ file system. In SOSP.Google ScholarGoogle Scholar
  27. [27] Clark Raymond K., Jensen E. Douglas, and Reynolds Franklin D.. 1992. An architectural overview of the Alpha real-time distributed kernel. In USENIX Workshop on Microkernels and other Kernel Architectures.Google ScholarGoogle Scholar
  28. [28] David Francis M., Chan Ellick, Carlyle Jeffrey C., and Campbell Roy H.. 2008. CuriOS: Improving reliability through operating system structure. In OSDI.Google ScholarGoogle Scholar
  29. [29] Elphinstone Kevin and Heiser Gernot. 2013. From L3 to seL4 what have we learnt in 20 years of L4 microkernels? In SOSP.Google ScholarGoogle Scholar
  30. [30] Engler D. R., Kaashoek M. F., and Jr. J. O’Toole,1995. Exokernel: An operating system architecture for application-level resource management. In SOSP’95. ACM, New York, NY.Google ScholarGoogle Scholar
  31. [31] Ford Bryan and Lepreau Jay. 1994. Evolving Mach 3.0 to a migrating thread model. In USENIX Winter.Google ScholarGoogle Scholar
  32. [32] Gamsa Benjamin, Krieger Orran, Appavoo Jonathan, and Stumm Michael. 1999. Tornado: Maximizing locality and concurrency in a shared memory multiprocessor operating system. In OSDI, Vol. 99. 87100.Google ScholarGoogle Scholar
  33. [33] Härtig Hermann, Hohmuth Michael, Liedtke Jochen, Wolter Jean, and Schönberg Sebastian. 1997. The performance of \( \mu \)-kernel-based systems. In ACM SIGOPS Operating Systems Review, Vol. 31. ACM.Google ScholarGoogle Scholar
  34. [34] Klein Gerwin, Elphinstone Kevin, Heiser Gernot, Andronick June, Cock David, Derrin Philip, Elkaduwe Dhammika, Engelhardt Kai, Kolanski Rafal, Norrish Michael, et al. 2009. seL4: Formal verification of an OS kernel. In ACM SIGOPS.Google ScholarGoogle Scholar
  35. [35] Koldinger Eric J., Chase Jeffrey S., and Eggers Susan J.. 1992. Architecture Support for Single Address Space Operating Systems. Vol. 27. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Lee Sanghoon, Tiwari Devesh, Solihin Yan, and Tuck James. 2011. HAQu: Hardware-accelerated queueing for fine-grained threading on a chip multiprocessor. In HPCA.Google ScholarGoogle Scholar
  37. [37] Levy Henry M.. 1984. Capability-based Computer Systems. Digital Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Li Wenhao, Xia Yubin, Chen Haibo, Zang Binyu, and Guan Haibing. 2015. Reducing world switches in virtualized environment with flexible cross-world calls. In ISCA.Google ScholarGoogle Scholar
  39. [39] Liedtke Jochen. 1993. Improving IPC by kernel design. ACM SIGOPS Oper. Syst. Rev. 27, 5 (Dec. 1993), 175–188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Liedtke Jochen. 1993. A persistent system in real use-experiences of the first 13 years. In 3rd International Workshop on Object Orientation in Operating Systems. IEEE.Google ScholarGoogle Scholar
  41. [41] Liedtke Jochen. 1995. On Micro-kernel Construction. Vol. 29. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Liedtke Jochen, Elphinstone Kevin, Schonberg Sebastian, Hartig Hermann, Heiser Gernot, Islam Nayeem, and Jaeger Trent. 1997. Achieved IPC performance (still the foundation for extensibility). In 6th Workshop on Hot Topics in Operating Systems. IEEE.Google ScholarGoogle Scholar
  43. [43] Lyons Anna, McLeod Kent, Almatary Hesham, and Heiser Gernot. 2018. Scheduling-context capabilities: A principled, light-weight operating-system mechanism for managing time. In 13th EuroSys Conference. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Markuze Alex, Smolyar Igor, Morrison Adam, and Tsafrir Dan. 2018. DAMN: Overhead-free IOMMU protection for networking. In 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] McVoy Larry W., Staelin Carl, et al. 1996. LMbench: Portable tools for performance analysis. In USENIX. 279294.Google ScholarGoogle Scholar
  46. [46] Mi Zeyu, Li Dingji, Yang Zihan, Wang Xinran, and Chen Haibo. 2019. SkyBridge: Fast and secure inter-process communication for microkernels. In EuroSys. ACM.Google ScholarGoogle Scholar
  47. [47] Min Changwoo, Kang Woonhak, Kumar Mohan, Kashyap Sanidhya, Maass Steffen, Jo Heeseung, and Kim Taesoo. 2018. Solros: A data-centric operating system architecture for heterogeneous computing. In EuroSys. ACM.Google ScholarGoogle Scholar
  48. [48] Narayanan Vikram, Balasubramanian Abhiram, Jacobsen Charlie, Spall Sarah, Bauer Scott, Quigley Michael, Hussain Aftab, Younis Abdullah, Shen Junjie, Bhattacharyya Moinak, and Burtsev Anton. 2019. LXDs: Towards isolation of kernel subsystems. In USENIX. USENIX Association, 269284. Retrieved from https://www.usenix.org/conference/atc19/presentation/narayanan.Google ScholarGoogle Scholar
  49. [49] Narayanan Vikram, Huang Yongzhe, Tan Gang, Jaeger Trent, and Burtsev Anton. 2020. Lightweight kernel isolation with virtualization and VM functions. In VEE’20. Association for Computing Machinery, New York, NY, 157171. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Park Soyeon, Lee Sangho, Xu Wen, Moon HyunGon, and Kim Taesoo. 2019. libmpk: Software abstraction for Intel memory protection keys (Intel MPK). In USENIX. USENIX Association, 241254. Retrieved from https://www.usenix.org/conference/atc19/presentation/park-soyeon.Google ScholarGoogle Scholar
  51. [51] Saltzer Jerome H.. 1974. Protection and the control of information sharing in multics. Commun. ACM 17, 7 (1974), 388402.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Shapiro Jonathan S., Smith Jonathan M., and Farber David J.. 1999. EROS: A fast capability system. Vol. 33. ACM.Google ScholarGoogle Scholar
  53. [53] Steinberg Udo and Kauer Bernhard. 2010. NOVA: A microhypervisor-based secure virtualization architecture. In EUROSYS.Google ScholarGoogle Scholar
  54. [54] Tsafrir Dan. 2007. The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops). In ExpCS. ACM.Google ScholarGoogle Scholar
  55. [55] Vilanova Lluïs, Ben-Yehuda Muli, Navarro Nacho, Etsion Yoav, and Valero Mateo. 2014. CODOMs: Protecting software with code-centric memory domains. In ACM SIGARCH Computer Architecture News. IEEE Press.Google ScholarGoogle Scholar
  56. [56] Vilanova Lluís, Jordà Marc, Navarro Nacho, Etsion Yoav, and Valero Mateo. 2017. Direct inter-process communication (dIPC): Repurposing the CODOMs architecture to accelerate IPC. In EuroSys. ACM.Google ScholarGoogle Scholar
  57. [57] Waterman Andrew, Lee Yunsup, Patterson David A., and Asanovi Krste. 2014. The RISC-V Instruction Set Manual. Volume 1: User-Level ISA, Version 2.0. Technical Report. California University Berkeley Department of Electrical Engineering and Computer Sciences.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Watson Robert N. M., Laurie Ben, et al. 2015. Cheri: A hybrid capability-system architecture for scalable software compartmentalization. In SP. IEEE.Google ScholarGoogle Scholar
  59. [59] Watson Robert N. M., Norton Robert M., Woodruff Jonathan, Moore Simon W., Neumann Peter G., Anderson Jonathan, Chisnall David, Davis Brooks, Laurie Ben, Roe Michael, et al. 2016. Fast protection-domain crossing in the CHERI capability-system architecture. IEEE Micro 36, 5 (2016), 38–49.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Witchel Emmett, Cates Josh, and Asanović Krste. 2002. Mondrian memory protection. In ASPLOS. ACM, New York, NY.Google ScholarGoogle Scholar
  61. [61] Witchel Emmett, Rhee Junghwan, and Asanović Krste. 2005. Mondrix: Memory isolation for Linux using Mondriaan memory protection. In SOSP. ACM.Google ScholarGoogle Scholar

Index Terms

  1. Boosting Inter-process Communication with Architectural Support

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Computer Systems
        ACM Transactions on Computer Systems  Volume 39, Issue 1-4
        November 2021
        216 pages
        ISSN:0734-2071
        EISSN:1557-7333
        DOI:10.1145/3543986
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 5 July 2022
        • Online AM: 9 May 2022
        • Revised: 1 April 2022
        • Accepted: 1 April 2022
        • Received: 1 February 2021
        Published in tocs Volume 39, Issue 1-4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format