Matthieu Gouel
Justin P. Rohrer
Skip Abstract Section
Abstract
We describe a new system for distributed tracing at the IP level of the routes that packets take through the IPv4 internet. Our Zeph algorithm coordinates route tracing efforts across agents at multiple vantage points, assigning to each agent a number of /24 destination prefixes in proportion to its probing budget and chosen according to a reinforcement learning heuristic that aims to maximize the number of multipath links discovered. Zeph runs on top of Iris, our fault tolerant system for orchestrating internet measurements across distributed agents of heterogeneous probing capacities. Iris is built around third party free open source software and modern containerization technology, thereby presenting a new model for assembling a resilient and maintainable internet measurement architecture. We show that carefully choosing the destinations to probe from which vantage point matters to optimize topology discovery and that a system can learn which assignment will maximize the overall discovery based on previous measurements. After 10 cycles of probing, Zeph is capable of discovering 2.4M nodes and 10M links in a cycle of 6 hours, when deployed on 5 Iris agents. This is at least 2 times more nodes and 5 times more links than other production systems for the same number of prefixes probed.
- [n. d.]. PlanetLab Europe. https://planet-lab.eu/. ([n. d.]). Accessed February 2, 2022.Google Scholar
- 2004. Oregon Route Views. http://routeviews.org/. (2004). June 8, 2004; accessed February 2, 2022.Google Scholar
- 2008. The CAIDA UCSD IPv4 Routed /24 Topology Dataset. https://www.caida.org/catalog/datasets/ipv4_routed_24_topology_dataset/. (2008). February 1, 2008; version of July 8, 2020.Google Scholar
- 2014. The Impact of the Archipelago Measurement Platform. https://www.caida.org/projects/ark/impact/. (2014). July 3, 2014; version of November 15, 2019.Google Scholar
- Bernhard Ager, Nikolaos Chatzis, Anja Feldmann, Nadi Sarrar, Steve Uhlig, and Walter Willinger. 2012. Anatomy of a Large European IXP. In Proc. ACM SIGCOMM Conf. (SIGCOMM '12). Google ScholarDigital Library
- Brice Augustin, Xavier Cuvellier, Benjamin Orgogozo, Fabien Viger, Timur Friedman, Matthieu Latapy, Clémence Magnien, and Renata Teixeira. 2006. Avoiding Traceroute Anomalies with Paris Traceroute. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '06). Google ScholarDigital Library
- Paul Barford, Azer Bestavros, John Byers, and Mark Crovella. 2001. On the Marginal Utility of Network Topology Measurements. In Proc. ACM SIGCOMM Internet Measurement Workshop (IMW '01). Google ScholarDigital Library
- Robert Beverly. 2016. Yarrp'ing the Internet: Randomized High-Speed Active Topology Discovery. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '16). Google ScholarDigital Library
- Robert Beverly, Arthur Berger, and Geoffrey G. Xie. 2010. Primitives for Active Internet Topology Mapping: Toward High-Frequency Characterization. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '10). Google ScholarDigital Library
- Kimberly Claffy, Young Hyun, Ken Keys, Marina Fomenkov, and Dmitri Krioukov. 2009. Internet Mapping: From Art to Science. In Proc. 2009 Cyber-security Applications Technology Conf. for Homeland Security (CATCH). Google ScholarDigital Library
- Graham Cormode, Howard Karloff, and Anthony Wirth. 2010. Set Cover Algorithms for Very Large Datasets. In Proc. ACM Intl. Conf. on Information and Knowledge Management (CIKM '10). Google ScholarDigital Library
- Berat Can Şenel, Maxime Mouchet, Justin Cappos, Olivier Fourmaux, Timur Friedman, and Rick McGeer. 2021. EdgeNet: A Multi-Tenant and Multi-Provider Edge Cloud. In In Proc. ACM Intl. Workshop on Edge Systems, Analytics and Networking (EdgeSys '21). Google ScholarDigital Library
- Italo Cunha, Pietro Marchetta, Matt Calder, Yi-Ching Chiu, Bruno V. A. Machado, Antonio Pescapè, Vasileios Giotsas, Harsha V. Madhyastha, and Ethan Katz-Bassett. 2016. Sibyl: A Practical Internet Route Oracle. In Proc. USENIX Symposium on Networked Systems Design and Implementation (NSDI '16). Google ScholarDigital Library
- Ítalo Cunha, Renata Teixeira, Darryl Veitch, and Christophe Diot. 2014. DTRACK: A System to Predict and Track Internet Path Changes. IEEE/ACM Trans. on Networking 22, 4 (2014), 1025–1038. Google ScholarDigital Library
- Benoit Donnet, Matthew Luckie, Pascal Mérindol, and Jean-Jacques Pansiot. 2012. Revealing MPLS Tunnels Obscured from Traceroute. ACM SIGCOMM Computer Communications Rev. 42, 2 (Mar. 2012), 87–93. 0146-4833 Google ScholarDigital Library
- Benoit Donnet, Philippe Raoult, Timur Friedman, and Mark Crovella. 2005. Efficient Algorithms for Large-Scale Topology Discovery. In Proc. ACM SIGMETRICS Conf. (SIGMETRICS '05). Google ScholarDigital Library
- Constantine Dovrolis, Krishna Gummadi, Aleksandar Kuzmanovic, and Sascha D. Meinrath. 2010. Measurement Lab: Overview and an Invitation to the Research Community. ACM SIGCOMM Computer Communications Rev. 40, 3 (Jun. 2010), 53–56. 0146-4833 Google ScholarDigital Library
- Vasileios Giotsas, Thomas Koch, Elverton Fazzion, Ítalo Cunha, Matt Calder, Harsha V. Madhyastha, and Ethan Katz-Bassett. 2020. Reduce, Reuse, Recycle: Repurposing Existing Measurements to Identify Stale Traceroutes. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '20). Google ScholarDigital Library
- Vasileios Giotsas, Matthew Luckie, Bradley Huffaker, and kc claffy. 2014. Inferring Complex AS Relationships. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '14). Google ScholarDigital Library
- R. Govindan and H. Tangmunarunkit. 2000. Heuristics for Internet map discovery. In Proc. IEEE INFOCOM '00. Google ScholarCross Ref
- Yuchen Huang, Michael Rabinovich, and Rami Al-Dalky. 2020. FlashRoute: Efficient Traceroute on a Massive Scale. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '20). Google ScholarDigital Library
- Van Jacobson. 1988. 4BSD routing diagnostic tool available for ftp. Email [email protected] to the IETF and end2end-interest e-mail lists. (1988).Google Scholar
- Yuchen Jin, Colin Scott, Amogh Dhamdhere, Vasileios Giotsas, Arvind Krishnamurthy, and Scott Shenker. 2019. Stable and Practical AS Relationship Inference with ProbLink. In Proc. USENIX Symposium on Networked Systems Design and Implementation (NSDI '19). Google ScholarDigital Library
- Ethan Katz-Bassett, John P. John, Arvind Krishnamurthy, David Wetherall, Thomas Anderson, and Yatin Chawathe. 2006. Towards IP Geolocation Using Delay and Topology Measurements. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '06). Google ScholarDigital Library
- Volodymyr Kuleshov and Doina Precup. 2014. Algorithms for multi-armed bandit problems. (2014). [arxiv]1402.6028Google Scholar
- Matthew Luckie and Robert Beverly. 2017. The Impact of Router Outages on the AS-Level Internet. In Proc. ACM SIGCOMM Conf. (SIGCOMM '17). Google ScholarDigital Library
- Matthew Luckie, Robert Beverly, William Brinkmeyer, and kc claffy. 2013. Speedtrap: Internet-Scale IPv6 Alias Resolution. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '13). Google ScholarDigital Library
- Harsha V. Madhyastha, Tomas Isdal, Michael Piatek, Colin Dixon, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. 2006. IPlane: An Information Plane for Distributed Services. In Proc. USENIX Symposium on Operating Systems Design and Implementation (OSDI '06). Google ScholarDigital Library
- Pascal Mérindol, Benoit Donnet, Jean-Jacques Pansiot, Matthew Luckie, and Young Hyun. 2011. MERLIN: MEasure the router level of the INternet. In Proc. Conference on Next Generation Internet Networks (EURO-NGI '11). Google ScholarCross Ref
- Yuval Shavitt and Eran Shir. 2005. DIMES: Let the Internet Measure Itself. ACM SIGCOMM Computer Communications Rev. 35, 5 (Oct. 2005), 71–74. 0146-4833 Google ScholarDigital Library
- Neil Spring, Ratul Mahajan, and David Wetherall. 2002. Measuring ISP Topologies with Rocketfuel. In Proc. ACM Sigcomm Conf. (SIGCOMM '02). Google ScholarDigital Library
- RIPE NCC Staff. 2015. RIPE Atlas: A Global Internet Measurement Network. The Internet Protocol Journal 18, 3 (Sept. 2015), 2–26. http://ipj.dreamhosters.com/wp-content/uploads/2015/10/ipj18.3.pdfGoogle Scholar
- James P.G. Sterbenz, Egemen K. Çetinkaya, Mahmood A. Hameed, Abdul Jabbar, Shi Qian, and Justin P. Rohrer. 2011. Evaluation of network resilience, survivability, and disruption tolerance: Analysis, topology generation, simulation, and experimentation. Telecommunication Systems 52, 2 (Dec. 2011), 705–736. 1018-4864, 1572-9451 Google ScholarDigital Library
- Yves Vanaubel, Jean-Romain Luttringer, Pascal Mérindol, Jean-Jacques Pansiot, and Benoit Donnet. 2019. TNT, Watch me Explode: A Light in the Dark for Revealing MPLS Tunnels. In Proc. Network Traffic Measurement and Analysis Conference (TMA '19). Google ScholarCross Ref
- Darryl Veitch, Brice Augustin, Renata Teixeira, and Timur Friedman. 2009. Failure control in multipath route tracing. In Proc. IEEE INFOCOM '09. Google ScholarCross Ref
- Kevin Vermeulen, Justin P. Rohrer, Robert Beverly, Olivier Fourmaux, and Timur Friedman. 2020. Diamond-Miner: Comprehensive Discovery of the Internet's Topology Diamonds. In Proc. USENIX Symposium on Networked Systems Design and Implementation (NSDI '20). https://www.usenix.org/conference/nsdi20/presentation/vermeulen Google ScholarDigital Library
- Kevin Vermeulen, Stephen D. Strowes, Olivier Fourmaux, and Timur Friedman. 2018. Multilevel MDA-Lite Paris Traceroute. In Proc. ACM SIGCOMM Internet Measurement Conf. (IMC '18). Google ScholarDigital Library
- Walter Willinger, David Alderson, and John C Doyle. 2009. Mathematics and the internet: A source of enormous confusion and great potential. Notices of the American Mathematical Society 56, 5 (2009), 586–599. https://www.ams.org/notices/200905/rtx090500586p.pdfGoogle Scholar
- Zheng Zhang, Ying Zhang, Y. Charlie Hu, Z. Morley Mao, and Randy Bush. 2008. Ispy: Detecting Ip Prefix Hijacking on My Own. ACM SIGCOMM Computer Communications Rev. 38, 4 (Aug. 2008), 327–338. 0146-4833 Google ScholarDigital Library
Index Terms
- Zeph & Iris map the internet: A resilient reinforcement learning approach to distributed IP route tracing
Recommendations
Study of internet autonomous system interconnectivity from BGP routing tables
Special issue: In memroy of Olga CasalsRecently, several authors have proposed models based on power-laws to characterize Internet topologies. Most of these works use the BGP (Border Gateway Protocol) tables published by Oregon Route Views. The adjacency matrix containing AS (autonomous ...
Quantifying path exploration in the internet
Previous measurement studies have shown the existence of path exploration and slow convergence in the global Internet routing system, and a number of protocol enhancements have been proposed to remedy the problem. However, existing measurements were ...
Towards capturing representative AS-level Internet topologies
Recent studies on AS-level Internet connectivity have attracted considerable attention. These studies exclusively relied on BGP data from the Oregon route-views [University of Oregon Route Views Project, http:// www.routeviews.org] to derive some ...
Comments