pytorch-widedeep
pytorch-widedeep: A flexible package for multimodal deep learning - Published in JOSS (2023)
awesome-mmps
Corpus of resources for multimodal machine learning with physiological signals (mmps).
content-moderation-deep-learning
Deep learning based content moderation from text, audio, video & image input modalities.
crop-forecasting
Predicting rice field yields through the integration of Microsoft Planetary satellite images, meteorological data, and field information in the 2023 EY Open Science Data Challenge - Crop Forecasting.
flair-2
Engage in a semantic segmentation challenge for land cover description using multimodal remote sensing earth observation data, delving into real-world scenarios with a dataset comprising 70,000+ aerial imagery patches and 50,000 Sentinel-2 satellite acquisitions.
hateful_memes-hate_detectron
Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge. https://arxiv.org/abs/2012.12975
https://github.com/aidotse/multimodal-skin-lesion-classification
Mutlimodality for skin lesions classification
https://github.com/cosmaadrian/multimodal-depression-from-video
Official source code for the paper: "Reading Between the Frames Multi-Modal Non-Verbal Depression Detection in Videos"
a-multi-modal-transformer-architecture-combining-sentiment-dynamics-temporal-market-data
Our approach uniquely fuses sentiment dynamics from social media and news sources with temporal market data and macroeconomic indicators to construct dynamic graph representations of interfirm relationships. Further, we employ state-of-the-art GNNs, such as temporal graph convolutions, that adapt to the changing market and significantly enhance it.
sutd-trafficqa
[CVPR2021] SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events
https://github.com/aehrc/cvt2distilgpt2
Improving Chest X-Ray Report Generation by Leveraging Warm-Starting
https://github.com/carlosholivan/audiogenerationdiffusion
State-of-the-art of Audio Generation with Diffusion Models