Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/aliireza/iommu-bench

Overcoming the IOTLB Wall for Multi-100-Gbps Linux-based Networking

https://github.com/aliireza/iommu-bench

Science Score: 59.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 3 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: zenodo.org
  • ✓
    Committers with academic emails
    1 of 1 committers (100.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.7%) to scientific vocabulary
Last synced: 7 months ago · JSON representation

Repository

Overcoming the IOTLB Wall for Multi-100-Gbps Linux-based Networking

Basic Info
Statistics
  • Stars: 22
  • Watchers: 3
  • Forks: 1
  • Open Issues: 0
  • Releases: 1
Created over 3 years ago · Last pushed almost 3 years ago
Metadata Files
Readme License

README.md

iommu-bench

DOI

This repository contains information to benchmark IOMMU by (re)doing some of the experiments performed in the paper.

Experiments

The experiments are located at experiments/. The folder has a Makefile and README.md that can be used to run the experiments.

Note: Before running the experiments, you need to prepare your testbed according to the our guidelines.

Custom Kernels

Our paper uses different Linux kernels to perform experiments, see https://github.com/aliireza/linux. The following list summarizes the relevant patches:

  • Aritifial TCP drops: This branch is based on a kernel patch developed by Luigi Rizzo (see here). It artifically drops TCP packets to induce TCP re-transmissions and put more pressure on IOMMU/IOTLB. You need to set three parameters as follows:

    • lossylocalport, lossyremoteport: If non zero, they indicate the sockets matching one of these ports that will ignore drops.
    • drop_freq: If non zero, the packets will be artificially dropped on the receive side one every drop_freq.

    ```bash

    Example

    echo 2345 > /sys/module/tcpinput/parameters/lossylocalport echo 10 > /sys/module/tcpinput/parameters/drop_freq # drop one in 10 ```

  • Page Pool API with hugepage bulk allocation: This branch contains changes to perform bulk allocations backed by hugepages within the Page Pool API in order to rely on hugepage IOTLB mappings.

    • There is another branch that provides an ethtool option to enable/disable the hugepage support, but it is not tested. bash sudo ethtool -K <interface_name> rx-hp-alloc off

Citing our paper

If you use PacketMill or X-Change in any context, please cite our paper:

bibtex @article{farshin-iommu, title = {Overcoming the IOTLB wall for multi-100-Gbps Linux-based networking}, author = {Farshin, Alireza and Rizzo, Luigi and Elmeleegy, Khaled and Kostić, Dejan}, year = 2023, month = may, keywords = {200 Gbps, Hugepages, iPerf, IOMMU, IOTLB, Linux kernel, Packet processing}, abstract = { This article explores opportunities to mitigate the performance impact of IOMMU on high-speed network traffic, as used in the Linux kernel. We first characterize IOTLB behavior and its effects on recent Intel Xeon Scalable & AMD EPYC processors at 200 Gbps, by analyzing the impact of different factors contributing to IOTLB misses and causing throughput drop (up to 20% compared to the no-IOMMU case in our experiments). Secondly, we discuss and analyze possible mitigations, including proposals and evaluation of a practical hugepage-aware memory allocator for the network device drivers to employ hugepage IOTLB entries in the Linux kernel. Our evaluation shows that using hugepage-backed buffers can completely recover the throughput drop introduced by IOMMU. Moreover, we formulate a set of guidelines that enable network developers to tune their systems to avoid the “IOTLB wall”, \textit{i.e}., the point where excessive IOTLB misses cause throughput drop. Our takeaways signify the importance of having a call to arms to rethink Linux-based I/O management at higher data rates. }, volume = 9, pages = {e1385}, journal = {PeerJ Computer Science}, issn = {2376-5992}, url = {https://doi.org/10.7717/peerj-cs.1385}, doi = {10.7717/peerj-cs.1385} }

Getting Help

If you have any questions regarding our code or the paper, you can contact Alireza Farshin (farshin at kth.se).

Owner

  • Name: Alireza Farshin
  • Login: aliireza
  • Kind: user
  • Location: Stockholm, Sweden
  • Company: KTH

Networked Systems Researcher | Doctoral Student

GitHub Events

Total
  • Watch event: 9
  • Fork event: 1
Last Year
  • Watch event: 9
  • Fork event: 1

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 14
  • Total Committers: 1
  • Avg Commits per committer: 14.0
  • Development Distribution Score (DDS): 0.0
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Alireza Farshin f****n@k****e 14
Committer Domains (Top 20 + Academic)
kth.se: 1

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels