:star: Nicla Vision Drivers :star:

Plug-and-play Arduino drivers for Arduino Nicla Vision sensors.

Description

This package contains the Arduino (and deprecated Micropython) drivers enabling the Arduino Nicla Vision board to be ready-to-use in the ROS world! :boom:

The developed scripts allows for streaming of sensors data from the Arduino Nicla Vision board to a ROS-running machine via Serial, TCP or UDP sockets. This package provide the optimised drivers that will be running on the Nicla Vision board, allowing to: - connect the board to Serial/WiFi, - sense all the sensors, - serialize the sensors data, - send the sensors data through the socket.

Here a list of the available sensors onboard the Arduino Nicla Vision board: - 2MP color camera - Time-of-Flight (distance) sensor
- Microphone
- Imu

Table of Contents

1. Arduino
2. Micropython
3. Package List
4. Video
5. License
6. Cite

Arduino

suggested
The Arduino version is the one suggested, working, and the one that will be maintained.

Installation (Arduino)

Step-by-step instructions on how to get the drivers running onboard the Arduino Nicla Vision:

1. Download Arduino IDE. For any additional info you can check the official Arduino Nicla Vision doc
2. Install the necessary core from Arduino IDE (Tools > Board > Boards Manager...): Mbed OS Nicla Boards
3. Download the necessary libraries from Arduino IDE (Tools -> Manage Libraries). Depending on the Nicla sensor you have to install:
   • IMU Arduino_LSM6DSOX by Arduino
   • Camera JPEGENC by Larry Bank
   • TOF VL53L1X by Pololu
   • Microphone nothing
   • CRC32 by Christopher Baker - crc-32 for image (only Serial)
4. Download the arduino/main folder of this repo for UDP/TCP connection, or the arduino/mainSerial for Serial (UART over USB) connection.
5. Modify accordingly the config.h/configSerial.h file for network and sensors configuration.
6. Open the main.ino file in Arduino IDE and upload it on the Nicla.

Usage (Arduino)

Follow the below two steps for enjoying your Arduino Nicla Vision board! 🚀

1. Just power the board and wait for leds check. Note that ENABLE_ARDUINO_IDE_SERIAL_MONITOR in the config file must be set to false when the board is not tethered to a Arduino IDE running pc.

Note: Look at the LED of your board! The first seconds after having turned it on, the LED should be Green or Blue. - When the board is correctly connected and it is streaming, the LED will turn off. - If you are having connection issues, the LED turns Blue. - If you are having other runtime errors, the LED turns Red.

:sunny: Now you are ready to go! Check out the Nicla Vision ROS2 repository or the Nicla Vision ROS repository for unlocking the Nicla Vision board in the ROS ecosystem! :sunny:

Micropython

unmaintained
We had problems with the micropython library for the microphone, but the rest of the sensors should be good. Anyway, we are not maintaining the micropython version anymore, having switched to the Arduino version definetevely.

Installation (micropython)

Step-by-step instructions on how to get the drivers running onboard the Arduino Nicla Vision:

1. Connect the board to the pc.
2. If it is your first time with the board, please follow the Getting Started tutorial here, for getting the latest available firmware of the board through OpenMV IDE.
3. Download the scripts/main.py and scripts/config.json files of this repository
4. Copy the main.py and config.json files ìnside the memory of the Nicla Vision board.
5. Reboot the board (unplug and plug again the board to the pc)

Usage (micropython)

Follow the below two steps for enjoying your Arduino Nicla Vision board! 🚀

1. Turn on a Hotspot connection on your pc.

Note: if you are on Ubuntu, click on Settings->WiFi->top three dots->"Turn on WiFi Hotspot...". If "Turn on WiFi Hotspot..." is grayed out, click first on the Network tab and then follow again the procedure starting from the WiFi tab. If the Hotspot is already configured, run this command in a terminal: $ nmcli connection up Hotspot

1. $ ifconfig on your pc, and copy the "inet" address under the "enp" voice (following point)
2. Set the parameters in the config.json file:
   • ssid: the name of the activated Hotspot
   • password: the password of the activated Hotspot
   • ip: IP address copied at point 2.
   • connection_type: "tcp" or "udp" (the user can choose if the board should transmit the sensors data by UDP or TCP socket connection)
   • verbose: true or false (for visualizing some debug prints in OpenMV IDE)
3. Run the main.py:
   • just turn on the board (e.g. connecting it to the pc)
   • using OpenMV IDE (Getting Started tutorial here)

Note: Look at the LED of your board! The first seconds (about 15 sec) after having turned it on, the LED should be Blue. - When the board is correctly connected and it is streaming, the LED will turn off. - If you are having connection issues, the LED will be Blue again. - If during execution you see a Green LED, it is for unforseen errors. - If during execution you see a Red LED, it is for memory errors (usually picture quality too high).

:sunny: Now you are ready to go! Check out the Nicla Vision ROS2 repository or the Nicla Vision ROS repository for unlocking the Nicla Vision board in the ROS ecosystem! :sunny:

Video Demonstration

https://github.com/ADVRHumanoids/niclavisionros2/assets/63496571/247952a0-86cb-4514-a034-d8736c8b70ba

Package List

Here some useful links:

• Nicla Vision Drivers repository
• Nicla Vision ROS repository
• Nicla Vision ROS2 repository

License

Distributed under the Apache-2.0 License. See LICENSE for more information.

Citation

:raised_hands: If you use this work or take inspiration from it, please cite (to be published): bibtex @inproceedings {DelBianco2024, author = {Del Bianco, Edoardo and Torielli, Davide and Rollo, Federico and Gasperini, Damiano and Laurenzi, Arturo and Baccelliere, Lorenzo and Muratore, Luca and Roveri, Marco and Tsagarakis, Nikos G.}, booktitle={2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)}, title = {A High-Force Gripper with Embedded Multimodal Sensing for Powerful and Perception Driven Grasping}, year={2024}, volume={}, number={}, pages={149-156}, doi={10.1109/Humanoids58906.2024.10769951} }