Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/google-deepmind/trecvit

https://github.com/google-deepmind/trecvit

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.2%) to scientific vocabulary
Last synced: 6 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: google-deepmind
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Size: 498 KB
Statistics
  • Stars: 9
  • Watchers: 9
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 1 year ago · Last pushed 10 months ago
Metadata Files
Readme Contributing License

README.md

TRecViT

Welcome to the Google DeepMind Github repository for TRecViT, a novel architecture for efficient video modelling.

TRecViT proposes a new factorisation that relies on gated linear recurrent units (LRUs) for mixing information over time, self-attention for mixing information over space, and MLPs for mixing over channels. This architecture achieves SOTA results on SSv2 when compared to causal models and outperforms or is competitive with non-causal vanilla Transformers on SSv2 and Kinetics400, while having significantly less parameters and smaller memory footprint and FLOPs count.

The model can be trained in supervised or self-supervised regimes (e.g. masked auto-encoding).

TRecViT Paper: https://arxiv.org/abs/2412.14294

architecture diagram

Installation

```bash

create environment (requires python>=3.10.0)

python3 -m venv trecvitenv source trecvitenv/bin/activate

get repo

git clone https://github.com/google-deepmind/trecvit.git cd trecvit

pip install . ```

Test the installation by importing our model:

bash from trecvit import trecvit_model model = trecvit_model.get_model()

This module depends on the recurrent_gemma library. It also uses forked files from big_vision.

Usage

Please check the quickstart colab to see how to instantiate the model and run inference.

Open In Colab

Pretrained Checkpoints

We release pretrained checkpoints for supervised classification using TRecViT-B on Kinetics400: https://storage.mtls.cloud.google.com/trecvit/modelcheckpoints/trecvitB_k400.npz

And masked-autoencoding: https://storage.googleapis.com/dm-perception-test/trecvit/trecvitBmae.npz

Additional checkpoints could be released in the future if requested.

Citing this work

To cite our work, please use:

@misc{pătrăucean2024trecvitrecurrentvideotransformer, title={TRecViT: A Recurrent Video Transformer}, author={Viorica Pătrăucean and Xu Owen He and Joseph Heyward and Chuhan Zhang and Mehdi S. M. Sajjadi and George-Cristian Muraru and Artem Zholus and Mahdi Karami and Ross Goroshin and Yutian Chen and Simon Osindero and João Carreira and Razvan Pascanu}, year={2024}, eprint={2412.14294}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2412.14294}, }

License and disclaimer

Copyright 2025 Google LLC

All software is licensed under the Apache License, Version 2.0 (Apache 2.0); you may not use this file except in compliance with the Apache 2.0 license. You may obtain a copy of the Apache 2.0 license at: https://www.apache.org/licenses/LICENSE-2.0

All other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY). You may obtain a copy of the CC-BY license at: https://creativecommons.org/licenses/by/4.0/legalcode

Unless required by applicable law or agreed to in writing, all software and materials distributed here under the Apache 2.0 or CC-BY licenses are distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the licenses for the specific language governing permissions and limitations under those licenses.

This is not an official Google product.

Owner

  • Name: Google DeepMind
  • Login: google-deepmind
  • Kind: organization

GitHub Events

Total
  • Issues event: 4
  • Watch event: 8
  • Issue comment event: 2
  • Member event: 3
  • Public event: 1
  • Push event: 9
  • Fork event: 1
  • Create event: 1
Last Year
  • Issues event: 4
  • Watch event: 8
  • Issue comment event: 2
  • Member event: 3
  • Public event: 1
  • Push event: 9
  • Fork event: 1
  • Create event: 1

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 10
  • Total Committers: 2
  • Avg Commits per committer: 5.0
  • Development Distribution Score (DDS): 0.2
Past Year
  • Commits: 10
  • Committers: 2
  • Avg Commits per committer: 5.0
  • Development Distribution Score (DDS): 0.2
Top Committers
Name Email Commits
Joe Heyward 4****a 8
viorik v****c@g****m 2

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 2
  • Total pull requests: 0
  • Average time to close issues: 15 days
  • Average time to close pull requests: N/A
  • Total issue authors: 2
  • Total pull request authors: 0
  • Average comments per issue: 1.5
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 0
  • Average time to close issues: 15 days
  • Average time to close pull requests: N/A
  • Issue authors: 2
  • Pull request authors: 0
  • Average comments per issue: 1.5
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • YJ-142150 (1)
  • orsveri (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

pyproject.toml pypi
  • einops *
  • flax *
  • jax *
  • numpy *
  • recurrentgemma @git+https://github.com/google-deepmind/recurrentgemma