Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

An M-Lab Overview

A Presentation given at the FCC Broadband Performance Measurement Workshop 2010-07-07
by

Dan Blah

on 18 November 2010

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of An M-Lab Overview

M-Lab Goals An Open Platform Measurement Lab (M-Lab) is an open, distributed server platform for researchers to deploy Internet measurement tools. The goal of M-Lab is to advance network research and empower the public with useful information about their broadband connections. By enhancing Internet transparency, M-Lab helps sustain a healthy, innovative Internet. M-Lab's Mission Partners People Are Using M-Lab http://measurementlab.net/news/2010/jul/07/m-lab-overview-prezi Municipalities States & Territories National Governments Research & Academic Institutions The Curious General Public Why Are They Using It? More than 100,000 people a day! In other words, this is an excellent testing tool and infrastructure.

Analyzing the publically available data from this test [M-Lab's NDT] has been very helpful in advancing our own understanding of the performance bottlenecks on today’s broadband networks. Understanding Broadband Speed Measurement
Clark, Bauer, and Lehr (June 2010)
Massachusetts Institute of Technology
http://mitas.csail.mit.edu/papers/Bauer_Clark_Lehr_Broadband_Speed_Measurements.pdf Open Platform Open Source Tools Publicly Available Open Data The M-Lab platform makes available to tool developers a large number of purpose-built and well-connected measurement servers in strategic locations around the globe. MLab 1 Nodes
mlab1.sea01.measurement-lab.org
mlab1.nuq01.measurement-lab.org
mlab1.lax01.measurement-lab.org
mlab1.dfw01.measurement-lab.org
mlab1.ord01.measurement-lab.org
mlab1.lga01.measurement-lab.org
mlab1.lga02.measurement-lab.org
mlab1.atl01.measurement-lab.org
mlab1.mia01.measurement-lab.org
mlab1.lhr01.measurement-lab.org
mlab1.ams01.measurement-lab.org
mlab1.ams02.measurement-lab.org
mlab1.par01.measurement-lab.org
mlab1.ath01.measurement-lab.org
mlab1.ham01.measurement-lab.org
mlab1.syd01.measurement-lab.org
MLab 2 Nodes
mlab2.sea01.measurement-lab.org
mlab2.nuq01.measurement-lab.org
mlab2.lax01.measurement-lab.org
mlab2.dfw01.measurement-lab.org
mlab2.ord01.measurement-lab.org
mlab2.lga01.measurement-lab.org
mlab2.lga02.measurement-lab.org
mlab2.atl01.measurement-lab.org
mlab2.mia01.measurement-lab.org
mlab2.lhr01.measurement-lab.org
mlab2.ams01.measurement-lab.org
mlab2.ams02.measurement-lab.org
mlab2.par01.measurement-lab.org
mlab2.ath01.measurement-lab.org
mlab2.ham01.measurement-lab.org
mlab2.syd01.measurement-lab.org
MLab 3 Nodes
mlab3.sea01.measurement-lab.org
mlab3.nuq01.measurement-lab.org
mlab3.lax01.measurement-lab.org
mlab3.dfw01.measurement-lab.org
mlab3.ord01.measurement-lab.org
mlab3.lga01.measurement-lab.org
mlab3.lga02.measurement-lab.org
mlab3.atl01.measurement-lab.org
mlab3.mia01.measurement-lab.org
mlab3.lhr01.measurement-lab.org
mlab3.ams01.measurement-lab.org
mlab3.ams02.measurement-lab.org
mlab3.par01.measurement-lab.org
mlab3.ath01.measurement-lab.org
mlab3.ham01.measurement-lab.org
mlab3.syd01.measurement-lab.org Measurement Lab: Overview and an Invitation to the Research Community
C. Dovrolis, K. Gummadi, A. Kuzmanovic, S. D. Meinrath (June 2010)
http://ccr.sigcomm.org/online/files/p53-3v40n3m-dovrolisPS.pdf U.S. 8 National Traffic Exchange Points, 9 sites, 27 Nodes Seattle Mountain View Los Angeles Chicago Dallas Miami Atlanta New York E.U. 4 Locations, 6 sites, 18 Nodes London Paris Amsterdam Hamburg Athens S.E. Asia, S. Pacific 1 Location, 1 Site, 3 Nodes Sydney Current Node List What's at a Site? Each Site is composed of 3 identical servers (Nodes) for redundancy and load balancing. Nodes are all "server-class" and well provisioned for conducting high-bandwidth measurements at large scale. Hardware
At least three enterprise-class servers
- 8x CPU Cores
- 8GB RAM (or 8GB for 64-bit systems)
- dual 100+GB Disks
- 1 Gbps Ethernet Network Adapter
- Built-in remote diagnostic, configuration, & power reset interface (such as Dell DRAC, HP iLO or OpenIPMI, Intel AMT, etc) IP Addresses
A /26 of IPv4 address space is required to support a site with three servers. IPv6 connectivity is a plus. Network connectivity
The minimum requirement is 1 Gbps to an upstream provider with no filtering or firewall. Collect information and data on how the Internet is operating today
Stimulate research activities to collect and analyze data to improve the understanding of the Internet
Assist consumers to better understand their broadband connection with publicly accesible tools and data
Support public debates about policy and regulation by supplying unbiased measurement data Inside an M-Lab Node Each node's base is a custom Open Source Linux Operating System maintained by PlanetLab. This system makes use of the Web100 instruments in the Linux TCP/IP stack to expose over a 150 network related variables for hosted measurement tests.
c2sspd, s2cspd, CurRwinRcvd, MaxRwinRcvd, loss, Ping, MaxRTT, avgrtt, jitter, CurRTO, WaitSec, SACKsRcvd, host, osVer, osName, osArch, javaVer, mismatch, Bad_cable, congestion, cwndtime, AccessTech, pctRcvrLimited, optimalRcvrBuffer, clientIP

A current list of variables:
http://web100.org/download/kernel/2.5.28/tcp-kis.txt
PlanetLab is the primary node maintainer
- Core node operations
- Assists in creating new measurement environments
- M-Lab nodes are not available to all PlanetLab Researchers Slices are created on each node to allocate specific resources for each tool.
Implemented as Linux-Vservers, which implements both namespace and performance isolation among slices on a single machine
Network virtualization is implemented using VNET
Tests show insignificant virtualization overhead (about 1-2%)

PlanetLab Architecture Overview:
https://www.planet-lab.org/doc/guides/user We Need You! M-Lab depends on the contribution of servers and connectivity from academic and industrial partners
Develop a unique tool that can run on the M-Lab platform
Data storage partners
Visualization/Infographic developers

A collaborative effort: we welcome suport from researchers, institutions, and companies that share M-Lab’s goals

http://www.measurementlab.org/getinvolved Virtualization Open Source Measurement Tools NDT Advanced Network Diagnostic Tool
Report measured upload & download speeds
Determine what problems are limiting speed i.e. client computer, home network, provider network, or testing server
Identify if the network connection is behind a NAT
Detect Firewall and possible end-to-end network settings
All tools are created by individual researchers, not M-Lab itself
All researchers are required to publish client and server software source code to allow for 3rd party review
All researchers' tools must be operated and licensed in such a way as to allow 3rd parties to develop client-side software for measurements
User Results Researcher Results NPAD Advanced Diagnostic tester
Analyze host configuration and network path
Identify if routers/switches would impact WAN performance
Web-based output simplifies problem reporting

http://www.psc.edu/networking/projects/pathdiag/ Glasnost Determine if an ISP is blocking email, HTTP or SSH transfer, Flash video, and P2P apps including BitTorrent, eMule and Gnutella.

http://broadband.mpi-sws.org/transparency/bttest-mlab.php Pathload2 Measures available bandwidth
Reports “Raw” bps capacity independent of transport or application protocol

http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/Papers/NetDov0248.pdf Sidestream Collect TCP details (web100 variables) from regular web traffic on a client computer
Collects information as a byproduct of general client network usage
No special test stream required
A way to evaluate, compare, and validate other active test results

http://www.measurementlab.net/news/2010/jan/19/side-stream-experiment
Windrider Attempts to detect whether your mobile broadband provider is performing application or service specific differentiation, i.e. prioritizing or slowing traffic to certain websites, applications, or content.

http://www.cs.northwestern.edu/~ict992/mobile.htm ShaperProbe Detects whether your ISP performs "traffic shaping" in either the upload or download directions. In the even of shapping, ShaperProbe reports the shaping rate and the "maximum burst size" before shaping begins.

http://www.cc.gatech.edu/~partha/diffprobe/shaperprobe.html Test Client Interfaces For Users In Web Browser Mobile Devices Open Data All data collected through M-Lab is intended to be made publicly available and placed in the public domain either immediately or after an optional 1 year embargo Collecting over 60TB of Data Raw measurement data (web100 and TCPdump logs) is created when a user runs a tool hosted on M-Lab
Raw data is stored locally on each M-Lab node
Seperate data collecting servers regularly collect all raw data
M-Lab tools consolidate raw data and re-format for 3rd party services
Formated test records are pushed to Amazon's EC2 and Google BigQuery

http://measurementlab.org/news/2009/dec/10/calling-all-researchers-m-lab-data-now-available-amazon-ec2

http://www.measurementlab.org/news/2010/jun/18/more-good-thing-analyze-and-access-m-lab-data-using-google-bigquery-and-google-stor Next? Closest Node M-Lab tools have access to DONAR.
A distributed system that provides name resolution and server selection for replication services.
Dynamic mapping policies direct users based on geographic location and platform load distribution.

http://donardns.org Measurement Lab (M-Lab) What Makes M-Lab Unique?
http://e2epi.internet2.edu/ndt An overprovisioned, extensible, scalable, robust open platform for open measurement tools The M-Lab Platform Using only 1% of total resources Operational Tools Coming Soon! Where is M-Lab M-Lab Hosted Tools Enabling deep critcal data analysis with transparent and open source tools and methodologies
Full transcript