@stdlib/ml-incr-binary-classification

Incrementally perform binary classification using stochastic gradient descent (SGD).

https://github.com/stdlib-js/ml-incr-binary-classification

Science Score: 57.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 6 DOI reference(s) in README
○
Academic publication links
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (15.3%) to scientific vocabulary

Keywords

algorithm binary class classification gradient-descent incremental javascript logistic machine-learning math mathematics ml node node-js nodejs online prediction statistics stats stdlib

Last synced: 4 months ago · JSON representation ·

Repository

Incrementally perform binary classification using stochastic gradient descent (SGD).

Basic Info

Host: GitHub
Owner: stdlib-js
License: apache-2.0
Language: JavaScript
Default Branch: main
Homepage: https://github.com/stdlib-js/stdlib
Size: 3.23 MB

Statistics

Stars: 6
Watchers: 3
Forks: 0
Open Issues: 0
Releases: 0

Topics

algorithm binary class classification gradient-descent incremental javascript logistic machine-learning math mathematics ml node node-js nodejs online prediction statistics stats stdlib

Created over 4 years ago · Last pushed 5 months ago

Metadata Files

Readme Changelog Contributing License Code of conduct Citation Security

We believe in a future in which the web is a preferred environment for numerical computation. To help realize this future, we've built stdlib. stdlib is a standard library, with an emphasis on numerical and scientific computation, written in JavaScript (and C) for execution in browsers and in Node.js.

The library is fully decomposable, being architected in such a way that you can swap out and mix and match APIs and functionality to cater to your exact preferences and use cases.

When you use stdlib, you can be absolutely certain that you are using the most thorough, rigorous, well-written, studied, documented, tested, measured, and high-quality code out there.

To join us in bringing numerical computing to the web, get started by checking us out on GitHub, and please consider financially supporting stdlib. We greatly appreciate your continued support!

incrBinaryClassification

Incrementally perform binary classification using stochastic gradient descent (SGD).

## Installation ```bash npm install @stdlib/ml-incr-binary-classification ``` Alternatively, - To load the package in a website via a `script` tag without installation and bundlers, use the [ES Module][es-module] available on the [`esm`][esm-url] branch (see [README][esm-readme]). - If you are using Deno, visit the [`deno`][deno-url] branch (see [README][deno-readme] for usage intructions). - For use in Observable, or in browser/node environments, use the [Universal Module Definition (UMD)][umd] build available on the [`umd`][umd-url] branch (see [README][umd-readme]). The [branches.md][branches-url] file summarizes the available branches and displays a diagram illustrating their relationships. To view installation and usage instructions specific to each branch build, be sure to explicitly navigate to the respective README files on each branch, as linked to above.

## Usage ```javascript var incrBinaryClassification = require( '@stdlib/ml-incr-binary-classification' ); ``` #### incrBinaryClassification( N\[, options] ) Returns an accumulator `function` which incrementally performs binary classification using [stochastic gradient descent][stochastic-gradient-descent]. ```javascript // Create an accumulator for performing binary classification on 3-dimensional data: var accumulator = incrBinaryClassification( 3 ); ``` The function accepts the following `options`: - **intercept**: `boolean` indicating whether to include an intercept. If `true`, an element equal to one is implicitly added to each provided feature vector (note, however, that the model does not perform regularization of the intercept term). If `false`, the model assumes that feature vectors are already centered. Default: `true`. - **lambda**: regularization parameter. The regularization parameter determines the amount of shrinkage inflicted on the model coefficients. Higher values reduce the variance of the model coefficient estimates at the expense of introducing bias. Default: `1.0e-4`. - **learningRate**: an array-like object containing the learning rate function and associated parameters. The learning rate function decides how fast or slow the model coefficients will be updated toward the optimal coefficients. Must be one of the following: - `['constant', ...]`: constant learning rate function. To set the learning rate, provide a second array element. By default, when the learn rate function is 'constant', the learning rate is set to `0.02`. - `['basic']`: basic learning rate function according to the formula `10/(10+t)` where `t` is the current iteration. - `['invscaling', ...]`: inverse scaling learning rate function according to the formula `eta0/pow(t, power_t)` where `eta0` is the initial learning rate and `power_t` is the exponent controlling how quickly the learning rate decreases. To set the initial learning rate, provide a second array element. By default, the initial learning rate is `0.02`. To set the exponent, provide a third array element. By default, the exponent is `0.5`. - `['pegasos']`: [Pegasos][@shalevshwartz:2011a] learning rate function according to the formula `1/(lambda*t)` where `t` is the current iteration and `lambda` is the regularization parameter. Default: `['basic']`. - **loss**: loss function. Must be one of the following: - `hinge`: hinge loss function. Corresponds to a soft-margin linear Support Vector Machine (SVM), which can handle non-linearly separable data. - `log`: logistic loss function. Corresponds to Logistic Regression. - `modifiedHuber`: Huber loss function [variant][@zhang:2004a] for classification. - `perceptron`: hinge loss function without a margin. Corresponds to the original perceptron by Rosenblatt (1957). - `squaredHinge`: squared hinge loss function SVM (L2-SVM). Default: `'log'`. By default, the model contains an intercept term. To omit the intercept, set the `intercept` option to `false`: ```javascript var array = require( '@stdlib/ndarray-array' ); // Create a model with the intercept term: var acc = incrBinaryClassification( 2, { 'intercept': true }); var coefs = acc( array( [ 1.4, 0.5 ] ), 1 ); // returns var dim = coefs.length; // returns 3 // Create a model without the intercept term: acc = incrBinaryClassification( 2, { 'intercept': false }); coefs = acc( array( [ 1.4, 0.5 ] ), -1 ); // returns dim = coefs.length; // returns 2 ``` #### accumulator( x, y ) If provided a feature vector `x` and response value `y` (either `+1` or `-1`), the accumulator function updates a binary classification model; otherwise, the accumulator function returns the current binary classification model coefficients. ```javascript var array = require( '@stdlib/ndarray-array' ); // Create an accumulator: var acc = incrBinaryClassification( 2 ); // Provide data to the accumulator... var x = array( [ 1.0, 0.0 ] ); var coefs = acc( x, -1 ); // returns x.set( 0, 0.0 ); x.set( 1, 1.0 ); coefs = acc( x, 1 ); // returns x.set( 0, 0.5 ); x.set( 1, 1.0 ); coefs = acc( x, 1 ); // returns coefs = acc(); // returns ``` #### accumulator.predict( X\[, type] ) Computes predicted response values for one or more observation vectors `X`. ```javascript var array = require( '@stdlib/ndarray-array' ); // Create a model with the intercept term: var acc = incrBinaryClassification( 2 ); // ... var label = acc.predict( array( [ 0.5, 2.0 ] ) ); // returns ``` Provided an [`ndarray`][@stdlib/ndarray/ctor] having shape `(..., N)`, where `N` is the number of features, the returned [`ndarray`][@stdlib/ndarray/ctor] has shape `(...)` (i.e., the number of dimensions is reduced by one) and data type `float64`. For example, if provided a one-dimensional [`ndarray`][@stdlib/ndarray/ctor], the method returns a zero-dimensional [`ndarray`][@stdlib/ndarray/ctor] whose only element is the predicted response value. By default, the method returns the predict label (`type='label'`). In order to return a prediction probability of a `+1` response value given either the logistic (`log`) or modified Huber (`modifiedHuber`) loss functions, set the second argument to `'probability'`. ```javascript var array = require( '@stdlib/ndarray-array' ); // Create a model with the intercept term: var acc = incrBinaryClassification( 2, { 'loss': 'log' }); // ... var phat = acc.predict( array( [ 0.5, 2.0 ] ), 'probability' ); // returns ``` In order to return the linear predictor (i.e., the signed distance to the hyperplane, which is computed as the dot product between the model coefficients and the provided feature vector `x`, plus the intercept), set the second argument to `'linear'`. ```javascript var array = require( '@stdlib/ndarray-array' ); // Create a model with the intercept term: var acc = incrBinaryClassification( 2, { 'loss': 'log' }); // ... var lp = acc.predict( array( [ 0.5, 2.0 ] ), 'linear' ); // returns ``` Given a feature vector `x = [x_0, x_1, ...]` and model coefficients `c = [c_0, c_1, ...]`, the linear predictor is equal to `(x_0*c_0) + (x_1*c_1) + ... + c_intercept`.

## Notes - The underlying binary classification model performs [L2 regularization][tikhonov-regularization] of model coefficients, shrinking them toward zero by penalizing their squared [euclidean norm][euclidean-norm]. - [Stochastic gradient descent][stochastic-gradient-descent] is sensitive to the scaling of the features. One is advised to either scale each feature to `[0,1]` or `[-1,1]` or to transform each feature into z-scores with zero mean and unit variance. One should keep in mind that the same scaling has to be applied to training data in order to obtain accurate predictions. - In general, the more data provided to an accumulator, the more reliable the model predictions.

## Examples ```javascript var normal = require( '@stdlib/random-base-normal' ); var binomial = require( '@stdlib/random-base-binomial' ); var array = require( '@stdlib/ndarray-array' ); var exp = require( '@stdlib/math-base-special-exp' ); var incrBinaryClassification = require( '@stdlib/ml-incr-binary-classification' ); // Create a new accumulator: var acc = incrBinaryClassification( 2, { 'intercept': true, 'lambda': 1.0e-3, 'loss': 'log' }); // Incrementally update the classification model... var phat; var x; var i; for ( i = 0; i < 10000; i++ ) { x = array( [ normal( 0.0, 1.0 ), normal( 0.0, 1.0 ) ] ); phat = 1.0 / ( 1.0+exp( -( ( 3.0*x.get(0) ) - ( 2.0*x.get(1) ) + 1.0 ) ) ); acc( x, ( binomial( 1, phat ) ) ? 1.0 : -1.0 ); } // Retrieve model coefficients: var coefs = acc(); console.log( 'Feature coefficients: %d, %d', coefs.get( 0 ), coefs.get( 1 ) ); console.log( 'Intercept: %d', coefs.get( 2 ) ); // Predict new observations... x = array( [ [ 0.9, 0.1 ], [ 0.1, 0.9 ], [ 0.9, 0.9 ] ] ); var out = acc.predict( x ); console.log( 'x = [%d, %d]; label = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) ); console.log( 'x = [%d, %d]; label = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) ); console.log( 'x = [%d, %d]; label = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) ); out = acc.predict( x, 'probability' ); console.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) ); console.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) ); console.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) ); out = acc.predict( x, 'linear' ); console.log( 'x = [%d, %d]; lp = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) ); console.log( 'x = [%d, %d]; lp = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) ); console.log( 'x = [%d, %d]; lp = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) ); ```

## References - Rosenblatt, Frank. 1957. "The Perceptron–a perceiving and recognizing automaton." 85-460-1. Buffalo, NY, USA: Cornell Aeronautical Laboratory. - Zhang, Tong. 2004. "Solving Large Scale Linear Prediction Problems Using Stochastic Gradient Descent Algorithms." In _Proceedings of the Twenty-First International Conference on Machine Learning_, 116. New York, NY, USA: Association for Computing Machinery. doi:[10.1145/1015330.1015332][@zhang:2004a]. - Shalev-Shwartz, Shai, Yoram Singer, Nathan Srebro, and Andrew Cotter. 2011. "Pegasos: primal estimated sub-gradient solver for SVM." _Mathematical Programming_ 127 (1): 3–30. doi:[10.1007/s10107-010-0420-4][@shalevshwartz:2011a].

* * * ## Notice This package is part of [stdlib][stdlib], a standard library for JavaScript and Node.js, with an emphasis on numerical and scientific computing. The library provides a collection of robust, high performance libraries for mathematics, statistics, streams, utilities, and more. For more information on the project, filing bug reports and feature requests, and guidance on how to develop [stdlib][stdlib], see the main project [repository][stdlib]. #### Community [![Chat][chat-image]][chat-url] --- ## License See [LICENSE][stdlib-license]. ## Copyright Copyright © 2016-2025. The Stdlib [Authors][stdlib-authors].

Owner

Name: stdlib
Login: stdlib-js
Kind: organization

Website: https://stdlib.io
Twitter: stdlibjs
Repositories: 3,692
Profile: https://github.com/stdlib-js

Standard library for JavaScript.

Citation (CITATION.cff)

cff-version: 1.2.0
title: stdlib
message: >-
  If you use this software, please cite it using the
  metadata from this file.

type: software

authors:
  - name: The Stdlib Authors
    url: https://github.com/stdlib-js/stdlib/graphs/contributors

repository-code: https://github.com/stdlib-js/stdlib
url: https://stdlib.io

abstract: |
  Standard library for JavaScript and Node.js.

keywords:
  - JavaScript
  - Node.js
  - TypeScript
  - standard library
  - scientific computing
  - numerical computing
  - statistical computing

license: Apache-2.0 AND BSL-1.0

date-released: 2016

GitHub Events

Total

Push event: 16

Last Year

Push event: 16

Committers

Last synced: almost 2 years ago

All Time

Total Commits: 47
Total Committers: 1
Avg Commits per committer: 47.0
Development Distribution Score (DDS): 0.0

Past Year

Commits: 9
Committers: 1
Avg Commits per committer: 9.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
stdlib-bot	n**y@s**o	47

Committer Domains (Top 20 + Academic)

stdlib.io: 1

Issues and Pull Requests

Last synced: 5 months ago

All Time

Total issues: 0
Total pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Total issue authors: 0
Total pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

Total packages: 1
Total downloads:
- npm 39 last-month

Total dependent packages: 3
Total dependent repositories: 1
Total versions: 10
Total maintainers: 4

npmjs.org: @stdlib/ml-incr-binary-classification

Incrementally perform binary classification using stochastic gradient descent (SGD).

Homepage: https://stdlib.io
License: Apache-2.0
Latest release: 0.2.2
published over 1 year ago

Versions: 10
Dependent Packages: 3
Dependent Repositories: 1
Downloads: 39 Last month

Rankings

Dependent packages count: 5.8%

Downloads: 7.6%

Average: 10.2%

Dependent repos count: 10.7%

Stargazers count: 11.2%

Forks count: 15.9%

Maintainers (4)

planeshifter rreusser stdlib-bot kgryte

Funding

type: opencollective
url: https://opencollective.com/stdlib

Last synced: 5 months ago

Dependencies

package.json npm

@stdlib/ndarray-array ^0.0.x development
@stdlib/random-base-binomial ^0.0.x development
@stdlib/random-base-normal ^0.0.x development
istanbul ^0.4.1 development
tap-spec 5.x.x development
tape git+https://github.com/kgryte/tape.git#fix/globby development
@stdlib/array-float64 ^0.0.x
@stdlib/assert-contains ^0.0.x
@stdlib/assert-has-own-property ^0.0.x
@stdlib/assert-is-array-like-object ^0.0.x
@stdlib/assert-is-boolean ^0.0.x
@stdlib/assert-is-ndarray-like ^0.0.x
@stdlib/assert-is-nonnegative-number ^0.0.x
@stdlib/assert-is-number ^0.0.x
@stdlib/assert-is-plain-object ^0.0.x
@stdlib/assert-is-positive-integer ^0.0.x
@stdlib/assert-is-positive-number ^0.0.x
@stdlib/assert-is-vector-like ^0.0.x
@stdlib/blas-base-dcopy ^0.0.x
@stdlib/blas-base-dscal ^0.0.x
@stdlib/blas-base-gaxpy ^0.0.x
@stdlib/blas-base-gdot ^0.0.x
@stdlib/math-base-special-exp ^0.0.x
@stdlib/math-base-special-expit ^0.0.x
@stdlib/math-base-special-max ^0.0.x
@stdlib/math-base-special-pow ^0.0.x
@stdlib/ndarray-base-numel ^0.0.x
@stdlib/ndarray-base-shape2strides ^0.0.x
@stdlib/ndarray-base-vind2bind ^0.0.x
@stdlib/ndarray-ctor ^0.0.x
@stdlib/string-format ^0.0.x
@stdlib/types ^0.0.x
@stdlib/utils-define-nonenumerable-read-only-accessor ^0.0.x
@stdlib/utils-define-nonenumerable-read-only-property ^0.0.x

.github/workflows/cancel.yml actions

styfle/cancel-workflow-action 0.11.0 composite

.github/workflows/close_pull_requests.yml actions

superbrothers/close-pull-request v3 composite

.github/workflows/examples.yml actions

actions/checkout v3 composite
actions/setup-node v3 composite

.github/workflows/npm_downloads.yml actions

actions/checkout v3 composite
actions/setup-node v3 composite
actions/upload-artifact v3 composite
distributhor/workflow-webhook v3 composite

.github/workflows/productionize.yml actions

act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite
stdlib-js/bundle-action main composite
stdlib-js/transform-errors-action main composite

.github/workflows/publish.yml actions

JS-DevTools/npm-publish v1 composite
act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite
styfle/cancel-workflow-action 0.11.0 composite

.github/workflows/test.yml actions

act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite

.github/workflows/test_bundles.yml actions

act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite
denoland/setup-deno v1 composite

.github/workflows/test_coverage.yml actions

act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite
codecov/codecov-action v3 composite
distributhor/workflow-webhook v3 composite

.github/workflows/test_install.yml actions

act10ns/slack v1 composite
actions/checkout v3 composite
actions/setup-node v3 composite

@stdlib/ml-incr-binary-classification

Science Score: 57.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

incrBinaryClassification

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

npmjs.org: @stdlib/ml-incr-binary-classification

Rankings

Maintainers (4)

Funding

Dependencies