networked-linux-memsync
Efficient Synchronization of Linux Memory Regions over a Network: A Comparative Study and Implementation (Bachelor's Thesis)
Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.8%) to scientific vocabulary
Keywords
Repository
Efficient Synchronization of Linux Memory Regions over a Network: A Comparative Study and Implementation (Bachelor's Thesis)
Basic Info
- Host: GitHub
- Owner: pojntfx
- License: apache-2.0
- Language: Jupyter Notebook
- Default Branch: main
- Homepage: https://pojntfx.github.io/networked-linux-memsync/
- Size: 762 MB
Statistics
- Stars: 17
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 49
Topics
Metadata Files
README.md
Efficient Synchronization of Linux Memory Regions over a Network: A Comparative Study and Implementation
Bachelor's thesis by Felicitas Pojtinger.
University: Hochschule der Medien Stuttgart
Course of Study: Media Informatics
Date: 2023-08-03
Academic Degree: Bachelor of Science
Primary Supervisor: Prof. Dr. Martin Goik
Secondary Supervisor: M.Sc. Philip Betzler
Abstract
Current solutions for access, synchronization and migration of resources over a network are characterized by application-specific protocols and interfaces, which result in fragmentation and barriers to adoption. This thesis aims to address these issues by presenting a universal approach that enables direct operation on a memory region, circumventing the need for custom-built solutions. Various methods to achieve this are evaluated on parameters such as implementation overhead, initialization time, latency and throughput, and an outline of each method's architectural constraints and optimizations is provided. The proposed solution is suitable for both LAN and WAN environments, thanks to a novel approach based on block devices in user space with background push and pull mechanisms. It offers a unified API that enables mounting and migration of nearly any state over a network with minimal changes to existing applications. Illustrations of real-world use cases, configurations and backends are provided, together with a production-ready reference implementation of the full mount and migration APIs via the open-source r3map (remote mmap) library.
Overview
This repository contains the LaTeX and Markdown markup, as well as citations, benchmarks and visualization code for the thesis. The resulting built document is published as a PDF by CI/CD:
The accompanying reference implementation of the presented approach for working with remote memory regions and migrating them between hosts, r3map, can also be found on GitHub:
Looking for an even higher-performance, production-ready library? Check out Loophole Labs Silo. Silo is an alternative implementation of a subset of r3map's features that focuses on performance and offers support for continuous, push-based migrations.
Additionally, you can find the thesis in different formats such as HTML and EPUB on GitHub pages, download them from GitHub releases or check out the source on GitHub. If you want to cite this work, see CITATION.cff.
Contributing
To contribute, please use the GitHub flow and follow our Code of Conduct.
To build and open a note locally, run the following:
```shell $ git clone https://github.com/pojntfx/networked-linux-memsync.git $ cd networked-linux-memsync $ ./configure $ make depend $ make dev-pdf/your-note # Use Bash completion to list available targets
In another terminal
$ make open-pdf/your-note # Use Bash completion to list available targets ```
The note should now be opened. Whenever you change a source file, it will automatically be re-compiled.
License
Efficient Synchronization of Linux Memory Regions over a Network: A Comparative Study and Implementation (c) 2024 Felicitas Pojtinger and contributors
SPDX-License-Identifier: Apache-2.0
Owner
- Name: Felicitas Pojtinger
- Login: pojntfx
- Kind: user
- Location: Vancouver, BC
- Company: @loopholelabs
- Website: https://felicitas.pojtinger.com/
- Twitter: pojntfx
- Repositories: 220
- Profile: https://github.com/pojntfx
Developer building digital infrastructure ✨ | #linux #kubernetes #virtualization #wasm #gnome
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this thesis in your work, please cite it as below."
authors:
- family-names: "Pojtinger"
given-names: "Felicitas"
title: "Efficient Synchronization of Linux Memory Regions over a Network: A Comparative Study and Implementation"
version: 1.0.0
date-released: "2023-08-03"
url: "https://github.com/pojntfx/networked-linux-memsync"
GitHub Events
Total
- Watch event: 2
Last Year
- Watch event: 2
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
- Total downloads: unknown
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 1
proxy.golang.org: github.com/pojntfx/networked-linux-memsync
- Homepage: https://github.com/pojntfx/networked-linux-memsync
- Documentation: https://pkg.go.dev/github.com/pojntfx/networked-linux-memsync#section-documentation
- License: Apache-2.0
-
Latest release: v0.0.0-20230831170701-3544159371e8
published almost 3 years ago
Rankings
Dependencies
- pojntfx/deliverance latest composite
- github.com/cespare/xxhash/v2 v2.2.0
- github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
- github.com/edsrzf/mmap-go v1.1.0
- github.com/go-ini/ini v1.67.0
- github.com/gocql/gocql v1.4.0
- github.com/golang/protobuf v1.5.3
- github.com/golang/snappy v0.0.4
- github.com/google/uuid v1.3.0
- github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
- github.com/loopholelabs/common v0.4.9
- github.com/loopholelabs/frisbee-go v0.7.1
- github.com/loopholelabs/polyglot-go v0.5.1
- github.com/loopholelabs/userfaultfd-go v0.1.1
- github.com/mattn/go-colorable v0.1.13
- github.com/mattn/go-isatty v0.0.19
- github.com/minio/minio-go v6.0.14+incompatible
- github.com/mitchellh/go-homedir v1.1.0
- github.com/pilebones/go-udev v0.9.0
- github.com/pkg/errors v0.9.1
- github.com/pojntfx/dudirekta v0.5.0
- github.com/pojntfx/go-nbd v0.1.10
- github.com/pojntfx/r3map v0.0.0-20230702125402-aa6d3def2dc8
- github.com/redis/go-redis/v9 v9.0.5
- github.com/rs/zerolog v1.29.1
- github.com/teivah/broadcast v0.1.0
- go.uber.org/atomic v1.11.0
- golang.org/x/crypto v0.10.0
- golang.org/x/net v0.11.0
- golang.org/x/sync v0.3.0
- golang.org/x/sys v0.9.0
- golang.org/x/text v0.10.0
- google.golang.org/genproto/googleapis/rpc v0.0.0-20230629202037-9506855d4529
- google.golang.org/grpc v1.56.1
- google.golang.org/protobuf v1.31.0
- gopkg.in/inf.v0 v0.9.1
- github.com/bitly/go-hostpool v0.0.0-20171023180738-a3a6125de932
- github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869
- github.com/bsm/ginkgo/v2 v2.7.0
- github.com/bsm/gomega v1.26.0
- github.com/cespare/xxhash/v2 v2.2.0
- github.com/coreos/go-systemd/v22 v22.5.0
- github.com/davecgh/go-spew v1.1.0
- github.com/davecgh/go-spew v1.1.1
- github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
- github.com/edsrzf/mmap-go v1.1.0
- github.com/go-ini/ini v1.67.0
- github.com/gocql/gocql v1.4.0
- github.com/godbus/dbus/v5 v5.0.4
- github.com/golang/protobuf v1.5.0
- github.com/golang/protobuf v1.5.3
- github.com/golang/snappy v0.0.3
- github.com/golang/snappy v0.0.4
- github.com/google/go-cmp v0.5.5
- github.com/google/go-cmp v0.5.9
- github.com/google/uuid v1.3.0
- github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
- github.com/kr/pretty v0.1.0
- github.com/kr/pretty v0.2.1
- github.com/kr/pty v1.1.1
- github.com/kr/text v0.1.0
- github.com/loopholelabs/common v0.4.9
- github.com/loopholelabs/frisbee-go v0.7.1
- github.com/loopholelabs/polyglot-go v0.5.1
- github.com/loopholelabs/testing v0.2.3
- github.com/loopholelabs/userfaultfd-go v0.1.1
- github.com/mattn/go-colorable v0.1.12
- github.com/mattn/go-colorable v0.1.13
- github.com/mattn/go-isatty v0.0.14
- github.com/mattn/go-isatty v0.0.16
- github.com/mattn/go-isatty v0.0.19
- github.com/minio/minio-go v6.0.14+incompatible
- github.com/mitchellh/go-homedir v1.1.0
- github.com/pilebones/go-udev v0.9.0
- github.com/pkg/errors v0.9.1
- github.com/pmezard/go-difflib v1.0.0
- github.com/pojntfx/dudirekta v0.5.0
- github.com/pojntfx/go-nbd v0.1.10
- github.com/pojntfx/r3map v0.0.0-20230702125402-aa6d3def2dc8
- github.com/redis/go-redis/v9 v9.0.5
- github.com/rs/xid v1.4.0
- github.com/rs/zerolog v1.29.1
- github.com/stretchr/objx v0.1.0
- github.com/stretchr/testify v1.3.0
- github.com/stretchr/testify v1.8.4
- github.com/teivah/broadcast v0.1.0
- go.uber.org/atomic v1.11.0
- go.uber.org/goleak v1.2.1
- golang.org/x/crypto v0.10.0
- golang.org/x/net v0.11.0
- golang.org/x/sync v0.3.0
- golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
- golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6
- golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab
- golang.org/x/sys v0.6.0
- golang.org/x/sys v0.9.0
- golang.org/x/text v0.10.0
- golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
- google.golang.org/genproto/googleapis/rpc v0.0.0-20230629202037-9506855d4529
- google.golang.org/grpc v1.56.1
- google.golang.org/protobuf v1.26.0-rc.1
- google.golang.org/protobuf v1.26.0
- google.golang.org/protobuf v1.31.0
- gopkg.in/inf.v0 v0.9.1
- gopkg.in/yaml.v3 v3.0.1