Data

Tools

Indexes

Applications

Experiments

Open Source Science

tcc

Optimizing IoT Cloud Applications for Scalability: Leveraging RPC with Distributed Queues for Seamless Operations

https://github.com/lombardidaniel/tcc

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (7.2%) to scientific vocabulary
Last synced: 8 months ago · JSON representation ·

Repository

Optimizing IoT Cloud Applications for Scalability: Leveraging RPC with Distributed Queues for Seamless Operations

Basic Info
  • Host: GitHub
  • Owner: LombardiDaniel
  • Language: Go
  • Default Branch: main
  • Size: 4.51 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 1 year ago · Last pushed 10 months ago
Metadata Files
Readme Citation

README.md

Optimizing IoT Cloud Applications for Scalability: Leveraging RPC with Distributed Queues for Seamless Operations

TCC

architecture

Router enables worker to have purelly sync code while still allowing for horizontal scalability in all sides.

  • Routing Queue acts as both RPC framework and service and load balancer.
  • Shared Memory acts as service discovery, storage for return addrs of workers (via ephemeral in-memroy queues in RabbitMQ).

rpc

This creates a decoupled system that does not need a long-lived connection between the router and the worker, allowing for better scalability.

Different levels of Async and Sync communication are common pitfals for scalability, where a single service may be needed to avoid resource locks and such. With this architecture, the TaskWorker (where business logic reside) is purelly synchronous. This allows for more reliable development and testing.

In a regular architecture, the Router would not be able to scale horizontally, because a reply from MQTT (round-robin) may fall on a different node, one that did not make the request, making it unable to ack the request (made by the worker).

Check out the docker-compose.yml! Note that all of our services have multiple replicas and the requests are still able to be routed correctly.

Happy path:

sh docker compose up --build

Access http://localhost:15672/, login with user: guest as password: guest. In task_queue queue, publish the example task:

json { "action": "test", "transaction_id": "1234567890", "product_id": "1234567890", "ts": "2025-06-25T18:30:32.56958179Z" }

Example reply:

```sh docker run -ti --network tcc_default eclipse-mosquitto:1.6.15 ash

mosquittopub -h mqtt -t /gw/GWMAC/response -m {\"deviceMac\":\"000000000001\",\"ack\":true} && mosquittopub -h mqtt -t /gw/GWMAC/response -m {\"deviceMac\":\"000000000002\",\"ack\":true} ```

To just run a simple experiment, configure the docker-compose.yml file with appropriate parameters and run it. Remember to uncomment mongodb if you wish to run it locally.

sh docker compose up --build -d && docker compose logs -t -f ; docker compose down

Owner

  • Name: Daniel Lombardi
  • Login: LombardiDaniel
  • Kind: user
  • Location: São Paulo, Brazil

Eng. Comp - UFSCar

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - family-names: Lombardi
    given-names: Daniel
    orcid: https://orcid.org/0000-0002-1892-3415
title: "Optimizing IoT Cloud Applications for Scalability: Leveraging RPC with Distributed Queues for Seamless Operations"
version: 1.0.0
# identifiers:
#   - type: doi
#     value: 10.5281/zenodo.1234
date-released: 2025-07-11

GitHub Events

Total
  • Push event: 41
  • Create event: 2
Last Year
  • Push event: 41
  • Create event: 2

Dependencies

docker-compose.yml docker
  • eclipse-mosquitto 1.6.15
  • rabbitmq 3-management
  • redis latest
router/Dockerfile docker
  • alpine 3.21.3 build
  • golang 1.23.8 build
task_worker/Dockerfile docker
  • alpine 3.21.3 build
  • golang 1.23.8 build
router/go.mod go
  • github.com/cespare/xxhash/v2 v2.2.0
  • github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
  • github.com/eclipse/paho.mqtt.golang v1.5.0
  • github.com/gorilla/websocket v1.5.3
  • github.com/rabbitmq/amqp091-go v1.10.0
  • github.com/redis/go-redis/v9 v9.7.3
  • golang.org/x/net v0.27.0
  • golang.org/x/sync v0.7.0
router/go.sum go
  • github.com/cespare/xxhash/v2 v2.2.0
  • github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
  • github.com/eclipse/paho.mqtt.golang v1.5.0
  • github.com/gorilla/websocket v1.5.3
  • github.com/rabbitmq/amqp091-go v1.10.0
  • github.com/redis/go-redis/v9 v9.7.3
  • go.uber.org/goleak v1.3.0
  • golang.org/x/net v0.27.0
  • golang.org/x/sync v0.7.0
task_worker/go.mod go
  • github.com/rabbitmq/amqp091-go v1.10.0
task_worker/go.sum go
  • github.com/rabbitmq/amqp091-go v1.10.0
  • go.uber.org/goleak v1.3.0