accountability-solution

This repository includes a Docker-based accountability solution based on Sysdig, Librdkafka producer, Kafka and MongoDB. This approach aims to identify the causes that have triggered a set of specific events, thanks to the use of the syscalls run by the monitored system. Features such as being completely decoupled from the monitored system, real-time analysis and optimized querying make this solution an optimal choice when it comes to understanding the root causes of a system's behaviour. Different assessment scenarios have been developed to define the best strategy to reduce the impact of the audit process and logging tasks.

Software artifacts

Sysdig (version 0.28.0)

Librdkafka (version 1.7.0)

Zookeeper (version 7.0.1)

Kafka (version 7.0.1)

Kafka-connect (version 7.0.1)

MongoDB (version 5.0.5)

MongoDB Atlas (version 5.0.6 Enterprise)

Docker-compose (version 1.26.0)

Installation

Dependencies can be installed with setup.sh. The kernel headers must be installed in the host operating system, before running sysdig.

Configuration and usage

Host IP must be set in the Docker environment variable BROKERKAFKAADVERTISEDHOSTNAME, defined in .env.

To enable TLS support, self-signed certificates, keystores and truststores can be generated by running the script create-secrets.sh.

To study autonomous systems different from ROS framework, audited processes and syscalls can be specified in settings.lua

Scenario I. ROS logging engine

ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be uncommented in talker.py and in listener.py. ROS execution can be started from ROS folder by running. docker-compose up

Scenario II. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and MongoDB (local)

Replace producer.cpp with producer-nossl.cpp to avoid TLS configuration.

MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to mongodb://root:admin@mongo:27017 The scenario can be deployed by running docker-compose -f docker-compose-notls.yml up -d ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running. docker-compose up

Scenario III. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and Atlas MongoDB

Replace producer.cpp with producer-nossl.cpp to avoid TLS configuration.

MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to mongodb+srv://root:admin@cluster0.ecipx.mongodb.net The scenario can be deployed by running docker-compose -f docker-compose-notls-atlas.yml up -d ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running. docker-compose up

Scenario IV. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and MongoDB (local) with TLSv1.3

MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to mongodb://root:admin@mongo:27017/admin?ssl=true The scenario can be deployed by running docker-compose -f docker-compose-tls.yml up -d ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running. docker-compose up

Scenario V. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and Atlas MongoDB with TLSv1.3

MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to mongodb+srv://root:admin@cluster0.ecipx.mongodb.net/admin?ssl=true The scenario can be deployed by running docker-compose -f docker-compose-tls-atlas.yml up -d ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running. docker-compose up

Acknowledgments

DMARCE (EDMAR+CASCAR) Project: EDMAR PID2021-126592OB-C21 -- CASCAR PID2021-126592OB-C22 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe