Data

Tools

Indexes

Applications

Experiments

Open Source Science

optical-flow-msthesis

Systematic Study of Optical Flow models

https://github.com/prajnan93/optical-flow-msthesis

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.4%) to scientific vocabulary

Keywords

optical-flow pytorch transformer-architecture
Last synced: 6 months ago · JSON representation ·

Repository

Systematic Study of Optical Flow models

Basic Info
  • Host: GitHub
  • Owner: prajnan93
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 30.8 MB
Statistics
  • Stars: 0
  • Watchers: 2
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Topics
optical-flow pytorch transformer-architecture
Created over 3 years ago · Last pushed about 2 years ago
Metadata Files
Readme License Citation

README.md

Exploring Training Recipes and Transformer Neural Networks for Optical Flow Estimation

[official thesis report] | [unpublished thesis draft]

This thesis makes the following contributions: - An empirical study of pre-training, dataset scheduling, and data augmentations on four generations of optical flow models to provide an Improved Training Recipe. - Understanding the efficacy of Transformer Neural Networks for the optical flow estimation task.

The majority of the code is supported by the EzFlow PyTorch Library which was developed as a prerequisite for the thesis study. This repository contains the training configuration files for all the experiments and the implementation of NAT-GM and SCCFlow end-to-end transformer architectures for optical flow estimation.

The improved training recipe can be found here: kubricimprovedaug

Four Generations of Optical Flow Models



Getting Started

  • Follow instructions to setup EzFlow and the conda environment from EzFlow Getting Started
  • Install the following additional packages: pip install git+https://github.com/huggingface/transformers pip3 install natten -f https://shi-labs.com/natten/wheels/cu113/torch1.10.1/index.html pip install timm
  • If natten package fails to install, follow the setup directions from: https://www.shi-labs.com/natten/ ____

The pretrained checkpoints for the improved results will be published in the EzFlow repository.

References

Citation

```bibtex

@article{ author={Goswami,Prajnan}, year={2022}, title={Exploring Training Recipes and Transformer Neural Networks for Optical Flow Estimation}, journal={ProQuest Dissertations and Theses}, url={https://www.proquest.com/docview/2789009042?pq-origsite=gscholar&fromopenview=true}, }

@software{ShahEzFlowAmodular2021, author = {Shah, Neelay and Goswami, Prajnan and Jiang, Huaizu}, license = {MIT}, month = {11}, title = {{EzFlow: A modular PyTorch library for optical flow estimation using neural networks}}, url = {https://github.com/neu-vig/ezflow}, year = {2021} } ```

Owner

  • Name: Prajnan Goswami
  • Login: prajnan93
  • Kind: user
  • Location: Boston
  • Company: Northeastern University

Grad student at Northeastern University, Boston, MA.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you find this paper useful, please cite it as below."
authors:
- family-names: "Goswami"
  given-names: "Prajnan"
- family-names: "Jiang"
  given-names: "Huaizu"
title: "Exploring Training Recipes and Transformer Neural Networks for Optical Flow Estimation"
date-released: 2022-12-15
url: "https://github.com/prajnan93/optical-flow-msthesis"
license: MIT
preferred-citation:
  type: article
  authors:
  - family-names: "Goswami"
    given-names: "Prajnan"
  - family-names: "Jiang"
    given-names: "Huaizu"
  url: "https://github.com/prajnan93/optical-flow-msthesis"
  month: 12
  # start: 1 # First page number
  # end: 29 # Last page number
  title: "Exploring Training Recipes and Transformer Neural Networks for Optical Flow Estimation"
  # issue: 1
  # volume: 1
  year: 2022

GitHub Events

Total
Last Year

Committers

Last synced: about 2 years ago

All Time
  • Total Commits: 180
  • Total Committers: 2
  • Avg Commits per committer: 90.0
  • Development Distribution Score (DDS): 0.117
Past Year
  • Commits: 43
  • Committers: 2
  • Avg Commits per committer: 21.5
  • Development Distribution Score (DDS): 0.465
Top Committers
Name Email Commits
Prajnan Goswami p****s@g****m 159
Prajnan Goswami 8****3 21

Issues and Pull Requests

Last synced: about 2 years ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

setup.py pypi