DFaaS: Decentralized Function-as-a-Service for Federated Edge

This repository holds DFaaS, a novel decentralized FaaS-based architecture designed to automatically and autonomously balance the traffic load across edge nodes belonging to federated Edge Computing ecosystems.

DFaaS implementation relies on an overlay peer-to-peer network and a distributed control algorithm that takes decisions on load redistribution. Although preliminary, our results confirm the feasibility of the approach, showing that the system can transparently redistribute the load across edge nodes when they become overloaded.

Our prototype is based on OpenFaaS and implements the control logic within Go P2P agents.

This research is conducted by the DatAI (formerly Insid&s) and REDS laboratories of the University of Milan-Bicocca.

If you wish to reuse this source code, please consider citing our article describing the first prototype:

@inproceedings{Ciavotta_DFaaS_2021, author = {Ciavotta, Michele and Motterlini, Davide and Savi, Marco and Tundo, Alessandro}, doi = {10.1109/CloudNet53349.2021.9657141}, pages = {1--4}, series = {2021 IEEE 10th International Conference on Cloud Networking (CloudNet)}, title = {{DFaaS: Decentralized Function-as-a-Service for Federated Edge Computing}}, url = {https://ieeexplore.ieee.org/document/9657141}, year = {2021} }

Scenario

The above figure depicts the considered network scenario. A set of geographically-distributed FaaS-enabled edge nodes (or simply edge nodes) is deployed at the edge of the access network.

Each of these nodes deploys a DFaaS platform for the execution of serverless functions, and is connected to a wireless or wired access point (e.g. a base station, a broadband network gateway, a WiFi access point, etc.).

The edge node can receive functions' execution requests, in the form of HTTP requests, generated by the users served by the access point.

Architecture

Prototype

This prototype relies on HAProxy to implement the proxy component, and on faasd, a lightweight version of OpenFaaS, that does not require Kubernetes, to implement the FaaS platform.

Also, we exploit Sysbox, an open-source and free container runtime (a specialized "runc") that enhances containers in two key ways:

• improves container isolation
• enables containers to run same workloads as VMs

Thanks to Sysbox we are able to run our prototype as a standalone Docker container that executes our agent, the HAProxy and faasd all together. This way, we can run several emulated edge nodes by simply executing multiple Docker containers.

How to run DFaaS

This section outlines the steps to install and deploy the DFaaS prototype on a single node. You have two options:

1. Automated deployment with Ansible: use the provided Ansible playbook to automatically build and deploy a node. This can be done either on the machine running Ansible or on a remote machine.

2. Manual deployment: build and deploy the node manually by executing each required command step-by-step.

Regardless of which of the options you choose, the deployment of the DFaaS prototype has been tested with Ubuntu 24.04.2 LTS.

We suggest to spin up a virtual machine from scratch with Ubuntu, with a user with a password and sudo enabled.

Automated deployment with Ansible

Install Ansible on the control node following the official documentation (use the Ansible PPA). Then use the provided playbook (setup_playbook.yaml) to deploy a DFaaS node on a specific managed node specified in an inventory file. Note that you can specify multiple managed nodes.

An example of inventory.yaml file is

yaml all: hosts: <node-name>: ansible_host: <ip_address> ansible_user: <user> ansible_password: <password> ansible_become: true

We assume that the managed node has a user with root privileges and can connect via SSH with a password. This is for testing purposes only!

To test the inventory, you can try the example playbook on the official Ansible documentation.

Important: on the control node you need to have the DFaaS Git repository and to build the DFaaS Agent. This can be done with the following:

console $ git clone https://github.com/unimib-datAI/dfaas.git $ sudo apt install golang-go $ go build -C dfaas/dfaasagent

Then you can run the playbook with ansible-playbook (make sure to be on the DFaaS directory!):

console $ ansible-playbook -i inventory.yaml setup_playbook.yaml

This deploys a basic, fully functional DFaaS node using the Node Margin Strategy. You can modify the dfaasagent.env file to configure the agent and replay the playbook to deploy it.

You can make automatic calls to the node with the operator. More information about it in the dedicated directory.

If you have four different VMs it's recommended to deploy the entire system exploiting the playbook and configuration files in test_environment. This is still a work in progress.

Manual deployment

This deployment setup is ideal for deploying a single node on the host machine without using Ansible or Docker. This means that you need to build, deploy, and run each component of the DFaaS prototype, which is useful for testing purposes. In this case, the only requirement is Ubuntu.

The # symbol in the following points denotes a shell session running as the root user.

1. Update the host packages:

   ```console

   apt update && apt upgrade

   ```

2. Install required packages:

   ```console

   apt install golang-go haproxy python3-pip python3-venv

   ```

3. Clone (or copy) this repository in the /opt/dfaas-src directory:

   ```console

   mkdir --parents /opt/

   git clone https://github.com/unimib-datAI/dfaas.git /opt/dfaas-src

   ```

4. Download, install, configure and run OpenFaaS:

   ```console

   mkdir /opt/faasd

   git clone --depth 1 --branch 0.19.6 https://github.com/openfaas/faasd.git /opt/faasd/

   cd /opt/faasd

   ./hack/install.sh

   echo 'admin' > /var/lib/faasd/secrets/basic-auth-password

   cp /opt/dfaas-src/docker/files/faasd/prometheus.yml /var/lib/faasd/prometheus.yml

   systemctl restart faasd.service faasd-provider.service

   ```

5. Download, install and run cAdvisor (used to monitor the containers):

   ```console

   mkdir /opt/cadvisor

   wget https://github.com/google/cadvisor/releases/download/v0.49.2/cadvisor-v0.49.2-linux-amd64 -O /opt/cadvisor/cadvisor

   chmod u+x /opt/cadvisor/cadvisor

   cp /opt/dfaas-src/docker/files/faasd/cadvisor.service /etc/systemd/system/cadvisor.service

   systemctl daemon-reload

   systemctl enable cadvisor

   systemctl start cadvisor

   ```

6. Download, install and configure Prometheus node exporter (used to monitor the node).

   ```console

   mkdir /opt/node_exporter

   wget https://github.com/prometheus/nodeexporter/releases/download/v1.9.0/nodeexporter-1.9.0.linux-amd64.tar.gz -O /opt/nodeexporter/nodeexported.tar.gz

   cd /opt/node_exporter

   tar xvfz node_exported.tar.gz

   mv nodeexporter-1.9.0.linux-amd64/nodeexporter node_exporter

   cp /opt/dfaas-src/docker/files/faasd/node-exporter.service /etc/systemd/system/node-exporter.service

   systemctl daemon-reload

   systemctl enable node-exporter

   ```

7. Install and run the DFaaS Forecaster:

   ```console

   mkdir /opt/forecaster

   cd /opt/forecaster

   sudo python3 -m venv pyenv

   /opt/forecaster/pyenv/bin/python3 -m pip install "fastapi[all]" scikit-learn lightgbm joblib pandas numpy

   cp --recursive /opt/dfaas-src/docker/forecaster/* /opt/forecaster/

   cp /opt/dfaas-src/docker/files/forecaster/forecaster.service /etc/systemd/system/

   systemctl daemon-reload

   systemctl enable forecaster.service

   systemctl start forecaster.service

   ```

8. Build and configure the DFaaS agent with an example configuration, using the Node Margin Strategy.

   ```console

   cd /opt/dfaas-src/dfaasagent

   go build

   mkdir /opt/dfaasagent

   cp /opt/dfaas-src/docker/files/dfaasagent/dfaasagent.service /etc/systemd/system/dfaasagent.service

   cp /opt/dfaas-src/dfaasagent/agent/loadbalancer/haproxycfgrecalc.tmpl /opt/dfaasagent/

   cp /opt/dfaas-src/dfaasagent/agent/loadbalancer/haproxycfgnms.tmpl /opt/dfaasagent/

   cp /opt/dfaas-src/docker/files/dfaasagent/group_list.json /opt/dfaasagent/

   cp /opt/dfaas-src/dfaasagent/dfaasagent /opt/dfaasagent/

   cp /opt/dfaas-src/dfaasagent.env /opt/dfaasagent/

   systemctl daemon-reload

   systemctl enable dfaasagent.service

   ```

9. Deploy some example functions (ocr, shasum and figlet) to the local OpenFaaS instance:

   console $ chmod u+x docker/files/deploy_functions.sh $ /opt/dfaas-src/docker/files/deploy_functions.sh $ curl http://localhost:8080/function/figlet -d 'Hello DFaaS world!'

10. Run the DFaaS agent and do an example call to the proxy (not OpenFaaS!):

    ```console

    systemctl start dfaasagent.service

    $ curl http://localhost:80/function/figlet -d 'Hello DFaaS world!' ```

Deploy functions

This script deploy the same set of functions on each of the nodes by using docker/files/deploy_functions.sh. The deploy_functions.sh script waits for the OpenFaaS gateway to be up (max 20 retries, 10s delay), then deploys 3 functions (ocr, shasum, figlet) from the OpenFaas store.

The script has 3 arguments: - 1st arg: number of nodes (e.g., 3) - 2nd arg: node name prefix (e.g., dfaas-node-) - 3rd arg: node name suffix (e.g., -1)

The resulting node name (container) will be dfaas-node-1-1, that is, the default name you get when using the provided docker-compose.yml file. shell ./utils/deploy-functions-to-nodes.sh 3 "dfaas-node-" "-1"

Alternatively you can exploit the deployment functionalities of the operator.

Invoke a function

Each node exposes port 808x:80 (e.g., node-1 exposed port is 8081:80), where port 80 is the HAProxy port. This example assumes you run DFaaS nodes via Docker Compose with the provided docker-compose.yml file.

You can invoke a function (i.e., via the first node) by simply contact the proxy on http://localhost:8081/function/{function_name}. shell curl http://localhost:8081/function/figlet -d 'Hello DFaaS world!'

Execute workload to a node using vegeta

We provide an example that use vegeta HTTP load testing tool to run workload on a node and demonstrate the load distribution over the federation.

You can install vegeta by executing the following commands: shell wget https://github.com/tsenart/vegeta/releases/download/v12.8.4/vegeta_12.8.4_linux_amd64.tar.gz tar -xf vegeta_12.8.4_linux_amd64.tar.gz && rm vegeta_12.8.4_linux_amd64.tar.gz sudo mv vegeta /usr/local/bin/

This example uses the vegeta json format and requires jq.

In a nutshell: - it runs a vegeta attack (duration: 5 minutes, rate: 50 req/s) to the figlet function on the first node - it saves the results and produces report ever 200ms

```shell

Create the vegeta results directory

mkdir -p vegeta-results export VEGFOLDER="vegeta-results/$(date +%Y-%m-%d-%H%M%S)" mkdir -p $VEGFOLDER

jq -ncM '{method: "GET", url: "http://localhost:8081/function/figlet", body: "Hello DFaaS world!" | @base64, header: {"Content-Type": ["text/plain"]}}' | \ vegeta attack -duration=5m -rate=50 -format=json | \ tee $VEGFOLDER/results.bin | \ vegeta report -every=200ms ```

You can also start multiple parallel Vegeta attacks exploiting operator functionalities.

Create plots from vegeta results

You can produce some plots from vegeta results by exploiting the vegeta plot command or our plot-results.py script, which is automatically executed after tests execution with the operator. To use our script, you need to install the required Python packages listed in plot-requirements.txt.

```shell

Encode results as JSON

cat $VEGFOLDER/results.bin | vegeta encode > $VEGFOLDER/results.json

Create plot with vegeta

cat $VEGFOLDER/results.bin | vegeta plot > $VEGFOLDER/plot.html

1st arg: path results.json

2nd arg: path output folder

3rd arg: rate req/s used for the attack (if merged is True specify rate=0)

4th arg: boolean merged (is the input file merged from multiple attacks?)

./operator/docker/files/plot-results.py $VEGFOLDER/results.json $VEGFOLDER/plots 50 False ```

Forwarding traffic as a malicious node

You can impersonate a malicious node that is not part of the federation by adding the header Dfaas-Node-Id with a value that is not a valid peer id of the network (e.g., Dfaas-Node-Id: malicious-id). All of its requests will be rejected.

Troubleshooting

```shell

Substitute the CONTAINER_NAME value with the desired container name

export CONTAINERNAME="dfaas-node-1-1" docker exec -it ${CONTAINERNAME} bash journalctl --follow --unit dfaasagent # ...or whatever you prefer to inspect (e.g., haproxy, faasd, faasd-provider) ```

Emulator

For a complex setup running several emulated edge nodes with different topologies see emulator directory. We provide instructions and examples to execute DFaaS nodes via Containernet emulator.

Simulator

We also provide a simulator to test and compare different load balancing techniques. The simulation code is available into the simulation directory. Data gathered by the DFaaS system used for simulation are available here.

For more information read associated README file.

License

Copyright © 2021-2025 The DFaaS Authors.

The source code in this repository is licensed under the GNU Affero General Public License (AGPL), version 2.0 or later. See the LICENSE file for more information.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

The complete list of The DFaaS Authors can be fond in the AUTHORS file or in the contributors page on the DFaaS GitHub repository.