CoastalLens: A MATLAB UAV Video Stabilization & Rectification Framework

Software to stabilize and rectify coastal imagery UAV data.
Developed from the CIRN Qualitative Coastal Imagining Toolbox.

Uncrewed aerial vehicles (UAVs) are an important tool for coastal monitoring with their relatively low-cost and rapid deployment capabilities. To generate scientific-grade image products, the UAV images/videos must be stabilized and rectified into world coordinates. Due to the limited stable region of coastal images suitable for control points, the processing of UAV-obtained videos can be time-consuming and resource-intensive. The CIRN Qualitative Coastal Imagining Toolbox provided a first-of-its-kind open-sourced code for rectifying these coastal UAV videos. Limitations of the toolbox, however, prompted the development of CoastalLens with an efficient data input procedure, providing capabilities to obtain drone position (extrinsics) from LiDAR surveys, and using a feature detection and matching algorithm to stabilize the video prior to rectification. This framework reduces the amount of human oversight, now only required during the data input processes. Removing the dependency on threshold stability control points can also result in less time in the field. We hope this framework will allow for more efficient processing of the ever-increasing coastal UAV datasets.

Installation

Requires MATLAB (min v2022b - for estworldpose function, see Input Requirements for an alternative if using an older MATLAB version). Ubuntu users: See Issue 11
Required MATLAB toolboxes: - Image Processing Toolbox - Computer Vision Toolbox - LiDAR Toolbox (to use pointcloud).

Requires exiftool (or metadata csv file : See Input Requirements)
Requires ffmpeg.
See Installation Help for installation guides based on your OS.

Running the Toolbox

Download or clone the repository. Run 'UAV_rectification.m'
Will run core scripts in CODES/scripts/
Requires dependencies in CODES/basicFunctions and CODES/helperFunctions

Running ver in the Command Window will show your MATLAB version and installed toolboxes.

You can find more details, including the information you need to run this on your own data in our wiki!

Demo / Getting Started

'UAVrectificationDEMO.m' runs a demo version of the code and can also be used to compare the new algorithm versus the CIRN Stability Control Points method. This method requires stability control points to be visible within the field of view.

Data to test the code is provided in the DATA folder and the video can be downloaded here (716MB). Save this video in DATA/20211215Torrey/Flight04/.

Here is the input information required for the DEMO version. We recommend making a similar table to keep track of all the necessary information for your own drone flights.

Flight information:

Origin information:

Products information:

User prompts/direction is printed in the Command Window.

Testing

This toolbox is currently in testing phase on the following systems: - MacBook Pro M1 2020 (OS 12.6), Matlab 2022b - MacBook Pro M2 2023 (OS 13.2.1), Matlab 2023a - Linux (Ubuntu 22.04.3 LTS), Matlab 2022b - DJI Drones

Email Updates

The code allows you to recieve email updates as it processes the data. If you do not want to recieve these, please select 'No' to 'Recieve update emails?'.
If you do, we have set up a Gmail account 'coastallens1903' to use that will be sending the emails, although we recommend you setting up your own account and generating a static App password (16-character) for it moving forward to avoid any security risks.

General Folder Structure:

Please set up your CODES and DATA folder in the following structure. The DATA folder may be located in a different folder than your general path, but must be organized as indicated, with all flights in the relevant day/location folder.

bash . CODES scripts basicFunctions helperFunctions DATA YYYYMMDD_Location1 Flight_01 Flight_02 Flight_03 YYYYMMDD_Location2 Flight_01 Flight_02 Flight_03

Core Scripts

Data Output

Known Bugs

• Estimating the world camera pose based on from 3-D to 2-D point correspondences can sometime require different combinations of points. More points are not always better. We are looking into ways to improve the robustness of this step.

Contributing

Contributions to the toolbox are very welcome! Here are some ways to do that:
- A number of features that we want to include in the future are listed as 'issues'.
- We also want to make sure that we can accommodate other UAV platforms/locations. If you include any changes to be able to process your data in a forked branch, please open a 'pull request' to merge it so other's can also use the addition.
- One of our goals is to write a Python version of this toolbox, so please let us know if you are interested.
- Let us know of any other features you would want included.

If you run into any problems while running the code, or think other things should be included, please let us know by opening an 'issue' or emailing me at alange@ucsd.edu.

License

CoastalLens is provided under the MIT license.

Cite As

A.M.Z. Lange, H. Lange, B.L. Bruder and J.W. Fiedler "CoastalLens: A MATLAB UAV Video Stabilization & Rectification Framework" (2024)