https://github.com/ai4er-cdt/gtc-exposure

Guided Team Challenge 2021: Exposure Team Project

https://github.com/ai4er-cdt/gtc-exposure

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
  • .zenodo.json file
  • DOI references
  • Academic publication links
  • Committers with academic emails
    3 of 12 committers (25.0%) from academic institutions
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (11.8%) to scientific vocabulary

Keywords

change-detection-algorithms damage-assessment exposure image-segmentation informal-settlements satellite-imagery sentinel-2 supervised-deep-learning
Last synced: 5 months ago · JSON representation

Repository

Guided Team Challenge 2021: Exposure Team Project

Basic Info
  • Host: GitHub
  • Owner: ai4er-cdt
  • Language: Jupyter Notebook
  • Default Branch: main
  • Homepage:
  • Size: 1.72 GB
Statistics
  • Stars: 1
  • Watchers: 6
  • Forks: 2
  • Open Issues: 4
  • Releases: 0
Topics
change-detection-algorithms damage-assessment exposure image-segmentation informal-settlements satellite-imagery sentinel-2 supervised-deep-learning
Created about 5 years ago · Last pushed almost 5 years ago

https://github.com/ai4er-cdt/gtc-exposure/blob/main/

[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ai4er-cdt/gtc-exposure/8d75f32827b072a658c02d4ffb8400957fcd6e22)

Cam logo      ESA logo        RMS logo     WTW logo         DL logo

# Repository for the Exposure Team of the Guided Team Challenge

## 1. Overview

This repository contains all code written for this challenge.

This project focuses on assessing change in the exposure of Caribbean informal settlements over time. This is done firstly by segmenting satellite images to locate informal settlements, and then repeating this process at different times to determine change. Three different methods were used for image segmentation, a Random Forest model as well as two semi-supervised Deep Learning models. This can identify growth or recession of informal settlements. 

Change detection algorithms were then developed, aiming to classify the effect of natural hazards on informal settlements, and hence determine a measure of vulnerability of these settlements. For example, following a disaster, change detection algorithms aim to determine the extent of damage suffered (e.g. destroyed, majorly damaged, undamaged). This was first approached with a ratio method, comparing the intensities of certain bands of pairs of satellite images to determine change. This simple method was built upon with a supervised deep learning approach, which was found to have limited success, likely due to the relatively low resolution of Sentinel-2 satellite imagery. To show the plausibility of such an approach, given high resolution data, a similar algorithm was applied to the labelled xBD dataset to classify damage sustained by buildings following a natural disaster.

This repository is split according to the structure of the write-up, with separate directories for settlement segmentation, change detection, and exposure quantification. Each contain notebooks that can be run to illustrate the different sections of the report.

## 2. Project Structure
```
 LICENSE
 README.md                   <- Main README.
 settlement_segmentation     <- Settlement segmentation section.
   
    deepcluster             <- DeepCluster model as well as training and testing notebooks
   
    liunsupervised          <- Unsupervised feature learning - model building, training, testing          
|   |
|    randomforest            <- RF Classifier training + testing
|
 change_detection            <- Change detection section.
    archive                 <- Archive of old code for this section
   
    deep_change_detection   <- Code for the deep learning approach to change detection on Sentinel-2 data.
   
    ratio_method            <- Code for the image ratio methods including thresholding and U-Net classification
|   |
|    xbd_hi_res_attempt      <- Code for semantic change detection applied to high resolution data
|
 exposure_quantification

```

---

Owner

  • Name: AI for Environmental Risk
  • Login: ai4er-cdt
  • Kind: organization
  • Location: Cambridge, UK

UKRI Centre for Doctoral Training in the Application of AI to the study of Environmental Risks, University of Cambridge and British Antarctic Survey

GitHub Events

Total
Last Year

Committers

Last synced: about 2 years ago

All Time
  • Total Commits: 331
  • Total Committers: 12
  • Avg Commits per committer: 27.583
  • Development Distribution Score (DDS): 0.628
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Ira Shokar 4****r 123
shmh40 5****0 68
Matt m****8@c****k 50
Luke Cullen 5****t 27
luke-scot l****3@c****k 25
Ira Shokar i****r@g****m 14
JoycelynLongdon 5****n 7
Ira Shokar I****r 7
mja2106 3****6 4
Natalie Yao 7****y 3
Ira Shokar i****0@s****k 2
Anita Faul 5****l 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: about 2 years ago

All Time
  • Total issues: 12
  • Total pull requests: 32
  • Average time to close issues: 16 days
  • Average time to close pull requests: about 2 hours
  • Total issue authors: 1
  • Total pull request authors: 5
  • Average comments per issue: 0.25
  • Average comments per pull request: 0.0
  • Merged pull requests: 28
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • luke-scot (12)
Pull Request Authors
  • luke-scot (10)
  • Ira-Shokar (8)
  • shmh40 (8)
  • mataln (4)
  • JoycelynLongdon (2)
Top Labels
Issue Labels
priority (3)
Pull Request Labels

Dependencies

binder/environment.yml conda
  • pip
  • python 3.8.*
binder/descartesLabsSetup/descartes_labs/requirements.txt pypi
  • descarteslabs ==1.5.0
  • ipyleaflet ==0.13.1
  • matplotlib ==3.2.2
  • numpy ==1.18.5
change_detection/deep_change_detection/requirements.txt pypi
  • Babel ==2.9.0
  • Bottleneck ==1.3.2
  • CacheControl ==0.12.6
  • Cython ==0.29.22
  • Django ==3.1.7
  • Flask ==1.1.2
  • GDAL ==2.2.2
  • HeapDict ==1.0.1
  • Jinja2 ==2.11.3
  • Keras ==2.4.3
  • Keras-Preprocessing ==1.1.2
  • LunarCalendar ==0.0.9
  • Markdown ==3.3.4
  • MarkupSafe ==1.1.1
  • Pillow ==7.0.0
  • PyDrive ==1.3.1
  • PyMeeus ==0.5.9
  • PyOpenGL ==3.1.5
  • PySocks ==1.7.1
  • PyWavelets ==1.1.1
  • PyYAML ==3.13
  • Pygments ==2.6.1
  • QtPy ==1.9.0
  • SQLAlchemy ==1.3.23
  • Send2Trash ==1.5.0
  • Shapely ==1.7.1
  • SoundFile ==0.10.3.post1
  • Sphinx ==1.8.5
  • Theano ==1.0.5
  • Werkzeug ==1.0.1
  • absl-py ==0.10.0
  • alabaster ==0.7.12
  • albumentations ==0.1.12
  • altair ==4.1.0
  • appdirs ==1.4.4
  • argon2-cffi ==20.1.0
  • asgiref ==3.3.1
  • astor ==0.8.1
  • astropy ==4.2
  • astunparse ==1.6.3
  • async-generator ==1.10
  • atari-py ==0.2.6
  • atomicwrites ==1.4.0
  • attrs ==20.3.0
  • audioread ==2.1.9
  • autograd ==1.3
  • backcall ==0.2.0
  • beautifulsoup4 ==4.6.3
  • bleach ==3.3.0
  • blis ==0.4.1
  • bokeh ==2.1.1
  • branca ==0.4.2
  • bs4 ==0.0.1
  • cachetools ==4.2.1
  • catalogue ==1.0.0
  • certifi ==2020.12.5
  • cffi ==1.14.5
  • chainer ==7.4.0
  • chardet ==3.0.4
  • click ==7.1.2
  • cloudpickle ==1.3.0
  • cmake ==3.12.0
  • cmdstanpy ==0.9.5
  • colorlover ==0.3.0
  • community ==1.0.0b1
  • contextlib2 ==0.5.5
  • convertdate ==2.3.1
  • coverage ==3.7.1
  • coveralls ==0.5
  • crcmod ==1.7
  • cufflinks ==0.17.3
  • cupy-cuda101 ==7.4.0
  • cvxopt ==1.2.6
  • cvxpy ==1.0.31
  • cycler ==0.10.0
  • cymem ==2.0.5
  • daft ==0.0.4
  • dask ==2.12.0
  • datascience ==0.10.6
  • debugpy ==1.0.0
  • decorator ==4.4.2
  • defusedxml ==0.7.1
  • descartes ==1.1.0
  • dill ==0.3.3
  • distributed ==1.25.3
  • dlib ==19.18.0
  • dm-tree ==0.1.5
  • docopt ==0.6.2
  • docutils ==0.16
  • dopamine-rl ==1.0.5
  • earthengine-api ==0.1.255
  • easydict ==1.9
  • ecos ==2.0.7.post1
  • editdistance ==0.5.3
  • en-core-web-sm ==2.2.5
  • entrypoints ==0.3
  • ephem ==3.7.7.1
  • et-xmlfile ==1.0.1
  • fa2 ==0.3.5
  • fancyimpute ==0.4.3
  • fastai ==1.0.61
  • fastdtw ==0.3.4
  • fastprogress ==1.0.0
  • fastrlock ==0.5
  • fbprophet ==0.7.1
  • feather-format ==0.4.1
  • filelock ==3.0.12
  • firebase-admin ==4.4.0
  • fix-yahoo-finance ==0.0.22
  • flatbuffers ==1.12
  • folium ==0.8.3
  • future ==0.16.0
  • gast ==0.3.3
  • gdown ==3.6.4
  • gensim ==3.6.0
  • geographiclib ==1.50
  • geopy ==1.17.0
  • gin-config ==0.4.0
  • glob2 ==0.7
  • google ==2.0.3
  • google-api-core ==1.26.1
  • google-api-python-client ==1.12.8
  • google-auth ==1.27.1
  • google-auth-httplib2 ==0.0.4
  • google-auth-oauthlib ==0.4.3
  • google-cloud-bigquery ==1.21.0
  • google-cloud-bigquery-storage ==1.1.0
  • google-cloud-core ==1.0.3
  • google-cloud-datastore ==1.8.0
  • google-cloud-firestore ==1.7.0
  • google-cloud-language ==1.2.0
  • google-cloud-storage ==1.18.1
  • google-cloud-translate ==1.5.0
  • google-colab ==1.0.0
  • google-pasta ==0.2.0
  • google-resumable-media ==0.4.1
  • googleapis-common-protos ==1.53.0
  • googledrivedownloader ==0.4
  • graphviz ==0.10.1
  • grpcio ==1.32.0
  • gspread ==3.0.1
  • gspread-dataframe ==3.0.8
  • gym ==0.17.3
  • h5py ==2.10.0
  • hijri-converter ==2.1.1
  • holidays ==0.10.5.2
  • holoviews ==1.13.5
  • html5lib ==1.0.1
  • httpimport ==0.5.18
  • httplib2 ==0.17.4
  • httplib2shim ==0.0.3
  • humanize ==0.5.1
  • hyperopt ==0.1.2
  • ideep4py ==2.0.0.post3
  • idna ==2.10
  • image ==1.5.33
  • imageio ==2.4.1
  • imagesize ==1.2.0
  • imbalanced-learn ==0.4.3
  • imblearn ==0.0
  • imgaug ==0.2.9
  • importlib-metadata ==3.7.2
  • importlib-resources ==5.1.2
  • imutils ==0.5.4
  • inflect ==2.1.0
  • iniconfig ==1.1.1
  • intel-openmp ==2021.1.2
  • intervaltree ==2.1.0
  • ipykernel ==4.10.1
  • ipython ==5.5.0
  • ipython-genutils ==0.2.0
  • ipython-sql ==0.3.9
  • ipywidgets ==7.6.3
  • itsdangerous ==1.1.0
  • jax ==0.2.10
  • jaxlib ==0.1.62
  • jdcal ==1.4.1
  • jedi ==0.18.0
  • jieba ==0.42.1
  • joblib ==1.0.1
  • jpeg4py ==0.1.4
  • jsonschema ==2.6.0
  • jupyter ==1.0.0
  • jupyter-client ==5.3.5
  • jupyter-console ==5.2.0
  • jupyter-core ==4.7.1
  • jupyterlab-pygments ==0.1.2
  • jupyterlab-widgets ==1.0.0
  • kaggle ==1.5.10
  • kapre ==0.1.3.1
  • keras-vis ==0.4.1
  • kiwisolver ==1.3.1
  • knnimpute ==0.1.0
  • korean-lunar-calendar ==0.2.1
  • librosa ==0.8.0
  • lightgbm ==2.2.3
  • llvmlite ==0.34.0
  • lmdb ==0.99
  • lucid ==0.3.8
  • lxml ==4.2.6
  • matplotlib ==3.2.2
  • matplotlib-venn ==0.11.6
  • missingno ==0.4.2
  • mistune ==0.8.4
  • mizani ==0.6.0
  • mkl ==2019.0
  • mlxtend ==0.14.0
  • more-itertools ==8.7.0
  • moviepy ==0.2.3.5
  • mpmath ==1.2.1
  • msgpack ==1.0.2
  • multiprocess ==0.70.11.1
  • multitasking ==0.0.9
  • murmurhash ==1.0.5
  • music21 ==5.5.0
  • natsort ==5.5.0
  • nbclient ==0.5.3
  • nbconvert ==5.6.1
  • nbformat ==5.1.2
  • nest-asyncio ==1.5.1
  • networkx ==2.5
  • nibabel ==3.0.2
  • nltk ==3.2.5
  • notebook ==5.3.1
  • np-utils ==0.5.12.1
  • numba ==0.51.2
  • numexpr ==2.7.3
  • numpy ==1.19.5
  • nvidia-ml-py3 ==7.352.0
  • oauth2client ==4.1.3
  • oauthlib ==3.1.0
  • okgrade ==0.4.3
  • opencv-contrib-python ==4.1.2.30
  • opencv-python ==4.1.2.30
  • openpyxl ==2.5.9
  • opt-einsum ==3.3.0
  • osqp ==0.6.2.post0
  • packaging ==20.9
  • palettable ==3.3.0
  • pandas ==1.1.5
  • pandas-datareader ==0.9.0
  • pandas-gbq ==0.13.3
  • pandas-profiling ==1.4.1
  • pandocfilters ==1.4.3
  • panel ==0.9.7
  • param ==1.10.1
  • parso ==0.8.1
  • pathlib ==1.0.1
  • patsy ==0.5.1
  • pexpect ==4.8.0
  • pickleshare ==0.7.5
  • pip-tools ==4.5.1
  • plac ==1.1.3
  • plotly ==4.4.1
  • plotnine ==0.6.0
  • pluggy ==0.7.1
  • pooch ==1.3.0
  • portpicker ==1.3.1
  • prefetch-generator ==1.0.1
  • preshed ==3.0.5
  • prettytable ==2.1.0
  • progressbar2 ==3.38.0
  • prometheus-client ==0.9.0
  • promise ==2.3
  • prompt-toolkit ==1.0.18
  • protobuf ==3.12.4
  • psutil ==5.4.8
  • psycopg2 ==2.7.6.1
  • ptyprocess ==0.7.0
  • py ==1.10.0
  • pyarrow ==3.0.0
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pycocotools ==2.0.2
  • pycparser ==2.20
  • pyct ==0.4.8
  • pydata-google-auth ==1.1.0
  • pydot ==1.3.0
  • pydot-ng ==2.0.0
  • pydotplus ==2.0.2
  • pyemd ==0.5.1
  • pyerfa ==1.7.2
  • pyglet ==1.5.0
  • pygobject ==3.26.1
  • pymc3 ==3.7
  • pymongo ==3.11.3
  • pymystem3 ==0.2.0
  • pynndescent ==0.5.2
  • pyparsing ==2.4.7
  • pyrsistent ==0.17.3
  • pysndfile ==1.3.8
  • pystan ==2.19.1.1
  • pytest ==3.6.4
  • python-apt ==0.0.0
  • python-chess ==0.23.11
  • python-dateutil ==2.8.1
  • python-louvain ==0.15
  • python-slugify ==4.0.1
  • python-utils ==2.5.6
  • pytz ==2018.9
  • pyviz-comms ==2.0.1
  • pyzmq ==22.0.3
  • qdldl ==0.1.5.post0
  • qtconsole ==5.0.3
  • regex ==2019.12.20
  • requests ==2.23.0
  • requests-oauthlib ==1.3.0
  • resampy ==0.2.2
  • retrying ==1.3.3
  • rpy2 ==3.4.2
  • rsa ==4.7.2
  • scikit-image ==0.16.2
  • scikit-learn ==0.22.2.post1
  • scipy ==1.4.1
  • screen-resolution-extra ==0.0.0
  • scs ==2.1.2
  • seaborn ==0.11.1
  • setuptools-git ==1.2
  • simplegeneric ==0.8.1
  • six ==1.15.0
  • sklearn ==0.0
  • sklearn-pandas ==1.8.0
  • smart-open ==4.2.0
  • snowballstemmer ==2.1.0
  • sortedcontainers ==2.3.0
  • spacy ==2.2.4
  • sphinxcontrib-serializinghtml ==1.1.4
  • sphinxcontrib-websupport ==1.2.4
  • sqlparse ==0.4.1
  • srsly ==1.0.5
  • statsmodels ==0.10.2
  • sympy ==1.7.1
  • tables ==3.4.4
  • tabulate ==0.8.9
  • tblib ==1.7.0
  • tensorboard ==2.4.1
  • tensorboard-plugin-wit ==1.8.0
  • tensorflow ==2.4.1
  • tensorflow-datasets ==4.0.1
  • tensorflow-estimator ==2.4.0
  • tensorflow-gcs-config ==2.4.0
  • tensorflow-hub ==0.11.0
  • tensorflow-metadata ==0.28.0
  • tensorflow-probability ==0.12.1
  • termcolor ==1.1.0
  • terminado ==0.9.2
  • testpath ==0.4.4
  • text-unidecode ==1.3
  • textblob ==0.15.3
  • textgenrnn ==1.4.1
  • thinc ==7.4.0
  • tifffile ==2021.3.17
  • toml ==0.10.2
  • toolz ==0.11.1
  • torch ==1.8.0
  • torchsummary ==1.5.1
  • torchtext ==0.9.0
  • torchvision ==0.9.0
  • tornado ==5.1.1
  • tqdm ==4.41.1
  • traitlets ==5.0.5
  • tweepy ==3.10.0
  • typeguard ==2.7.1
  • typing-extensions ==3.7.4.3
  • tzlocal ==1.5.1
  • umap-learn ==0.5.1
  • uritemplate ==3.0.1
  • urllib3 ==1.24.3
  • vega-datasets ==0.9.0
  • wasabi ==0.8.2
  • wcwidth ==0.2.5
  • webencodings ==0.5.1
  • widgetsnbextension ==3.5.1
  • wordcloud ==1.5.0
  • wrapt ==1.12.1
  • xarray ==0.15.1
  • xgboost ==0.90
  • xkit ==0.0.0
  • xlrd ==1.1.0
  • xlwt ==1.3.0
  • yellowbrick ==0.9.1
  • zict ==2.0.0
  • zipp ==3.4.1
change_detection/xbd_hi_resolution_attempt/overlay_output_to_image/requirements.txt pypi
  • Click ==7.0
  • Pillow >=7.1.0
  • Shapely ==1.6.4.post2
  • affine ==2.3.0
  • attrs ==19.3.0
  • click-plugins ==1.1.1
  • cligj ==0.5.0
  • numpy ==1.18.1
  • pyparsing ==2.4.6
  • rasterio ==1.1.2
  • snuggs ==1.4.7
change_detection/xbd_hi_resolution_attempt/requirements.txt pypi
  • IPython *
  • Pillow *
  • chainer *
  • geopandas *
  • imantics *
  • imgaug *
  • keras *
  • libtiff *
  • matplotlib *
  • numpy *
  • opencv-python *
  • scikit-image *
  • scikit-learn *
  • scipy *
  • simplification *
  • tensorboard *
  • tensorboardX *
  • tqdm *