Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/clulab/scala-transformers

Scala interfaces to huggingface transformers and tokenizers

https://github.com/clulab/scala-transformers

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.6%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

Scala interfaces to huggingface transformers and tokenizers

Basic Info
  • Host: GitHub
  • Owner: clulab
  • Language: Scala
  • Default Branch: main
  • Size: 94.6 MB
Statistics
  • Stars: 13
  • Watchers: 21
  • Forks: 0
  • Open Issues: 7
  • Releases: 0
Created almost 4 years ago · Last pushed over 1 year ago
Metadata Files
Readme Changelog

README.md

Build Status Maven Central

scala-transformers

Scala interfaces to newly trained Hugging Face/ONNX transformers and existing tokenizers

The libraries and models resulting from this project are incorporated into processors and generally don't need attention unless functionality is being modified, but here are some details about how it all works.

encoder

To incorporate the encoder subproject as a Scala library dependency, either to access an existing model or because you've trained a new one with the Python code there, you'll need to add something like this to your build.sbt file:

scala libraryDependencies += "org.clulab" %% "scala-transformers-encoder" % "0.4.0"

New models should generally be published to the CLU Lab's artifactory server so that they can be treated as library dependencies, although they can also be accessed as local files. Two models have been generated and published. They are incorporated into a Scala project with

```scala resolvers += "clulab" at "https://artifactory.clulab.org/artifactory/sbt-release"

// Pick one or more. libraryDependencies += "org.clulab" % "deberta-onnx-model" % "0.0.3" libraryDependencies += "org.clulab" % "roberta-onnx-model" % "0.0.2"

```

The models make reference to tokenizers which also need to be added according to instructions in the next section.

Please see the encoder README for information about how to generate models and how to download and package Hugging Face tokenizers for use in the tokenizer subproject.

tokenizer

To use the tokenizer subproject as a Scala library dependency, you'll need to add something like this to your build.sbt file:

scala libraryDependencies += "org.clulab" %% "scala-transformers-tokenizer" % "0.4.0"

See the tokenizer README for information about which tokenizers have already been packaged and how they are accessed.

Owner

  • Name: Computational Language Understanding Lab (CLU Lab) at University of Arizona
  • Login: clulab
  • Kind: organization
  • Location: Tucson, AZ

GitHub Events

Total
  • Watch event: 3
  • Delete event: 3
  • Issue comment event: 5
  • Push event: 10
  • Pull request review event: 2
  • Pull request review comment event: 3
  • Pull request event: 4
  • Create event: 5
Last Year
  • Watch event: 3
  • Delete event: 3
  • Issue comment event: 5
  • Push event: 10
  • Pull request review event: 2
  • Pull request review comment event: 3
  • Pull request event: 4
  • Create event: 5

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 88
  • Total Committers: 2
  • Avg Commits per committer: 44.0
  • Development Distribution Score (DDS): 0.341
Top Committers
Name Email Commits
Mihai Surdeanu s****u@g****m 58
Keith Alcock g****b@k****m 30
Committer Domains (Top 20 + Academic)
keithalcock.com: 1

Issues and Pull Requests

Last synced: about 3 years ago

All Time
  • Total issues: 22
  • Total pull requests: 72
  • Average time to close issues: 9 days
  • Average time to close pull requests: 8 days
  • Total issue authors: 2
  • Total pull request authors: 2
  • Average comments per issue: 5.05
  • Average comments per pull request: 2.32
  • Merged pull requests: 64
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 22
  • Pull requests: 72
  • Average time to close issues: 9 days
  • Average time to close pull requests: 8 days
  • Issue authors: 2
  • Pull request authors: 2
  • Average comments per issue: 5.05
  • Average comments per pull request: 2.32
  • Merged pull requests: 64
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • MihaiSurdeanu (4)
  • kwalcock (2)
Pull Request Authors
  • kwalcock (26)
  • MihaiSurdeanu (4)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 6
  • Total downloads: unknown
  • Total dependent packages: 3
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 48
repo1.maven.org: org.clulab:scala-transformers-common_2.13

Supports subprojects with common code

  • Versions: 8
  • Dependent Packages: 1
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Dependent packages count: 32.0%
Average: 35.0%
Stargazers count: 36.2%
Forks count: 39.8%
Last synced: 11 months ago
repo1.maven.org: org.clulab:scala-transformers-common_2.12

Supports subprojects with common code

  • Versions: 8
  • Dependent Packages: 1
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Dependent packages count: 32.0%
Average: 35.0%
Stargazers count: 36.2%
Forks count: 39.8%
Last synced: 11 months ago
repo1.maven.org: org.clulab:scala-transformers-common_2.11

Supports subprojects with common code

  • Versions: 8
  • Dependent Packages: 1
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Dependent packages count: 32.0%
Average: 35.0%
Stargazers count: 36.2%
Forks count: 39.8%
Last synced: 11 months ago
repo1.maven.org: org.clulab:scala-transformers-tokenizer_2.11

Provides a Scala interface to huggingface tokenizers written in Rust

  • Versions: 8
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Stargazers count: 36.2%
Average: 39.2%
Forks count: 39.8%
Dependent packages count: 48.9%
Last synced: 11 months ago
repo1.maven.org: org.clulab:scala-transformers-tokenizer_2.13

Provides a Scala interface to huggingface tokenizers written in Rust

  • Versions: 8
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Stargazers count: 36.2%
Average: 39.2%
Forks count: 39.8%
Dependent packages count: 48.9%
Last synced: 11 months ago
repo1.maven.org: org.clulab:scala-transformers-tokenizer_2.12

Provides a Scala interface to huggingface tokenizers written in Rust

  • Versions: 8
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 32.0%
Stargazers count: 36.2%
Average: 39.2%
Forks count: 39.8%
Dependent packages count: 48.9%
Last synced: 11 months ago

Dependencies

.github/workflows/ci.yml actions
  • actions/checkout v2 composite
  • actions/setup-java v1 composite