Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/intuit/metriks

Python package of commonly used metrics for evaluating information retrieval models.

https://github.com/intuit/metriks

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 5 committers (20.0%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (7.7%) to scientific vocabulary

Keywords

data-science information-retrieval metrics python36
Last synced: 6 months ago · JSON representation

Repository

Python package of commonly used metrics for evaluating information retrieval models.

Basic Info
  • Host: GitHub
  • Owner: intuit
  • License: mit
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 26.4 KB
Statistics
  • Stars: 25
  • Watchers: 8
  • Forks: 25
  • Open Issues: 8
  • Releases: 0
Topics
data-science information-retrieval metrics python36
Created over 6 years ago · Last pushed over 5 years ago
Metadata Files
Readme License Code of conduct Codeowners

README.rst 

.. image:: logo/metriks-logo.svg

|python| |build| |coverage|

.. |python| image:: https://img.shields.io/badge/python-3.6%20-blue.svg
    :target: https://www.python.org/downloads/release/python-360/
    :alt: Python Version

.. |build| image:: https://travis-ci.com/intuit/metriks.svg?branch=master
    :target: https://travis-ci.com/intuit/metriks

.. |coverage| image:: https://coveralls.io/repos/github/intuit/metriks/badge.svg?branch=master
    :target: https://coveralls.io/github/intuit/metriks?branch=master

-----

metriks is a Python package of commonly used metrics for evaluating information retrieval models.

Available Metrics
---------------------------
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| Python API                                                 | Description                                                                   |
+============================================================+===============================================================================+
| `metriks.recall_at_k(y_true, y_prob, k)`                   | Calculates recall at k for binary classification ranking problems.            |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.precision_at_k(y_true, y_prob, k)`                | Calculates precision at k for binary classification ranking problems.         |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.mean_reciprocal_rank(y_true, y_prob)`             | Gets a positional score on how well you did at rank 1, rank 2, etc.           |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.ndcg(y_true, y_prob, k)`                          | A score for measuring the quality of a set of ranked results.                 |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.label_mean_reciprocal_rank(y_true, y_prob)`       | Determines the average rank each label was placed across samples. Only labels |
|                                                            | that are relevant in the true data set are considered in the calculation.     |
+------------------------------------------------------------+-------------------------------------------------------------------------------+
| `metriks.confusion_matrix_at_k(y_true, y_prob, k)`         | Generates binary predictions from probabilities by evaluating the top k       |
|                                                            | items (in ranked order by y_prob) as true.                                    |
+------------------------------------------------------------+-------------------------------------------------------------------------------+

Installation
------------
Install using `pip `_
::

    pip install metriks

Alternatively, specific distributions can be downloaded from the
github `release `_
page. Once downloaded, install the ``.tar.gz`` file directly:
::

    pip install metriks-\*.tar.gz

Development
-----------
1. (*Optional*) If you have `virtualenv` and `virtualenvwrapper` create a new virtual environment:
::

    mkvirtualenv metriks

This isolates your specific project dependencies to avoid conflicts
with other projects.

2. Clone and install the repository:
::

    git clone git@github.com:intuit/metriks.git
    cd metriks
    pip install -e .
    

This will install a version to an isolated environment in editable
mode. As you update the code in the repository, the new code will
immediately be available to run within the environment (without the
need to `pip install` it again)

3. Run the tests using `tox`:
::

    pip install tox
    tox

Tox will run all of the tests in isolated environments

Owner

  • Name: Intuit
  • Login: intuit
  • Kind: organization
  • Location: Mountain View, CA

Powering prosperity around the world.

GitHub Events

Total
Last Year

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 17
  • Total Committers: 5
  • Avg Commits per committer: 3.4
  • Development Distribution Score (DDS): 0.471
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Leah Zhao L****o@i****m 9
klo1 k****2@g****m 4
ilalith i****h@a****m 2
Piyali Banerjee p****a@b****u 1
Andrew Lisowski l****4@g****m 1
Committer Domains (Top 20 + Academic)
buffalo.edu: 1 apixio.com: 1 intuit.com: 1

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 5
  • Total pull requests: 14
  • Average time to close issues: about 19 hours
  • Average time to close pull requests: about 14 hours
  • Total issue authors: 1
  • Total pull request authors: 8
  • Average comments per issue: 2.0
  • Average comments per pull request: 1.29
  • Merged pull requests: 7
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • wontonswaggie (5)
Pull Request Authors
  • karenclo (4)
  • wontonswaggie (3)
  • AnshumaJain (2)
  • suchig (1)
  • piyalibanerj (1)
  • iswaryalalith (1)
  • asaljalilvand (1)
  • hetaShah27 (1)
Top Labels
Issue Labels
gracehopperOSD (5)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 13 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 2
  • Total maintainers: 2
pypi.org: metriks

metriks is a Python package of commonly used metrics for evaluating information retrieval models.

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 13 Last month
Rankings
Forks count: 7.7%
Dependent packages count: 10.1%
Stargazers count: 12.3%
Average: 20.8%
Dependent repos count: 21.6%
Downloads: 52.5%
Maintainers (2)
jonathan.lunt WontonSwaggie
Last synced: 6 months ago

Dependencies

setup.py pypi
  • numpy *