https://github.com/vowpalwabbit/reinforcement_learning

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
○
.zenodo.json file
○
DOI references
○
Academic publication links
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.7%) to scientific vocabulary

Keywords

cpp machine-learning reinforcement-learning

Keywords from Contributors

active-learning contextual-bandits learning-to-search online-learning

Last synced: 6 months ago · JSON representation

Repository

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

Basic Info

Host: GitHub
Owner: VowpalWabbit
License: mit
Language: C++
Default Branch: master
Homepage:
Size: 15.7 MB

Statistics

Stars: 75
Watchers: 13
Forks: 41
Open Issues: 68
Releases: 0

Topics

cpp machine-learning reinforcement-learning

Created over 7 years ago · Last pushed over 1 year ago

Metadata Files

Readme License

RL Client Library

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

Compiling the library

Getting the source code

sh git clone --recursive https://github.com/VowpalWabbit/reinforcement_learning.git cd reinforcement_learning

Building with Vcpkg + CMake Presets

Requirements

CMake 3.20+
Ninja

Commands

sh cmake --preset=vcpkg-release cmake --build --preset=vcpkg-release

Alternative system-specific build instructions

Ubuntu

Dependencies

Install the dependencies for this project with the following commands. We recommend the use of vcpkg for installing cpprestsdk and flatbuffers. sh sudo apt-get install libboost-all-dev libssl-dev vcpkg install cpprestsdk flatbuffers

Configure + Build

When configuring a CMake project using vcpkg dependencies, we must provide the full path to the vcpkg.cmake toolchain file. ```sh

Configure

cmake -S . -B build -DCMAKETOOLCHAINFILE=/scripts/buildsystems/vcpkg.cmake

Build

cmake --build build -j nproc

Test

cmake --build build --target rltest -j nproc cmake --build build --target test ```

MacOS

Dependencies

MacOS dependencies can be managed through homebrew. sh brew install cpprestsdk flatbuffers openssl

Configure + Build

In order to build using homebrew dependencies, you must invoke cmake this way: sh cmake -S . -B build -DOPENSSL_ROOT_DIR=`brew --prefix openssl` -DOPENSSL_LIBRARIES=`brew --prefix openssl`/lib cmake --build build --target all -j 4

Windows

Dependencies

Dependencies on Windows should be managed using vcpkg. cmd vcpkg install --triplet x64-windows zlib boost-system boost-program-options boost-test boost-align boost-foreach boost-math boost-uuid cpprestsdk flatbuffers openssl

If needed, add the flatbuffers executables to your PATH: <vcpkg_root>\installed\x64-windows\tools\flatbuffers

Configure + Build in Visual Studio

Open the project directory in Visual Studio. Building the library can be easily done in the GUI. - Edit CMake command line settings with Project > CMake Settings for <project name> - Run CMake configuration with Project > Configure Cache. Use Project > Delete Cache and Reconfigure to force a full reconfiguration starting from a clean working directory. - Compile with Build > Build All - Run tests with Test > Run CTests for <project name>

This procedure has been verified to work well in Visual Studio 2022.

Configure with command line + Build in Visual Studio

Alternatively, CMake configuration can be done in the command line. cmd cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=<vcpkg_root>\scripts\buildsystems\vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows -A x64 -G "Visual Studio 16 2019" rem Generates a solution you can open and use in Visual Studio .\build\reinforcement_learning.sln

Set VcpkgIntegration environment variable to vcpkg.targets file on your machine. Example: VcpkgIntegration=c:\s\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets

Open reinforcement_learning.sln in Visual Studio. Build Release or Debug x64 configuration.

Troubleshooting

OpenSSL on MacOS

If you get an error similar to the following on MacOS when running cmake .., then you may be able to fix it by supplying the OpenSSL path to CMake. Make Error at /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message): Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_INCLUDE_DIR) Call Stack (most recent call first): /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindOpenSSL.cmake:413 (find_package_handle_standard_args) /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/CMakeFindDependencyMacro.cmake:48 (find_package) /usr/local/lib/cpprestsdk/cpprestsdk-config.cmake:11 (find_dependency) CMakeLists.txt:9 (find_package)

This can be fixed by invoking CMake similar to the following: bash cmake -DOPENSSL_ROOT_DIR=`brew --prefix openssl` -DOPENSSL_LIBRARIES=`brew --prefix openssl`/lib ..

System cpprestsdk on Linux

Installing cpprestsdk on Ubuntu18.04 using apt-get may result in cmake failing with: ``` CMake Error at CMakeLists.txt:9 (findpackage): By not providing "Findcpprestsdk.cmake" in CMAKEMODULE_PATH this project has asked CMake to find a package configuration file provided by "cpprestsdk", but CMake did not find one.

Could not find a package configuration file provided by "cpprestsdk" with any of the following names:

cpprestsdkConfig.cmake
cpprestsdk-config.cmake

Add the installation prefix of "cpprestsdk" to CMAKEPREFIXPATH or set "cpprestsdk_DIR" to a directory containing one of the above files. If "cpprestsdk" provides a separate development package or SDK, be sure it has been installed. ```

The workaround is to specify where to search cmake .. -DCMAKE_PREFIX_PATH=/usr/lib/x86_64-linux-gnu/cmake

Owner

Name: Vowpal Wabbit
Login: VowpalWabbit
Kind: organization

Website: https://vowpalwabbit.org
Repositories: 22
Profile: https://github.com/VowpalWabbit

GitHub Events

Total

Watch event: 2
Push event: 24
Pull request review event: 2
Pull request event: 6
Fork event: 1
Create event: 3

Last Year

Watch event: 2
Push event: 24
Pull request review event: 2
Pull request event: 6
Fork event: 1
Create event: 3

Committers

Last synced: 9 months ago

All Time

Total Commits: 596
Total Committers: 32
Avg Commits per committer: 18.625
Development Distribution Score (DDS): 0.819

Past Year

Commits: 4
Committers: 1
Avg Commits per committer: 4.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Jack Gerrits	j****s	108
ataymano@microsoft.com	A****v	78
olgavrou	o**u@g**m	78
Alexey Taymanov	4****o	69
Jacob Alber	j**r@f**m	40
Rodrigo Kumpera	k****a	34
Griffin Bassman	g**n@g**m	32
cheng-tan	c**3@g**m	21
Byron Xu	b**u@m**m	16
peterychang	4****g	15
Rajan	r**i@y**m	13
Chenxi Zhao	h**6@g**m	13
Eduardo Salinas	e**s@m**m	12
Oren Michaely	7****e	11
Sheetal Lahabar	l**h@m**m	9
yannstad	4****d	7
zwd-ms	7****s	7
Marco Rossi	m**i@m**m	4
Schuyler Goodman	s**n@g**m	4
v-jameslongo	v**o@m**m	4
Oren Michaely	3****y	3
Casey Irvine	4****T	3
Steve Kofsky	s**y@g**m	3
Tejaswi Paruchuri	8****i	2
jakub-szymanski	4****i	2
tyclintw	t**w@m**m	2
Dwaipayan Mukhopadhyay	d**m@g**m	1
Quinn Damerell	q**e@m**m	1
Rajan Chari	r**s@m**m	1
TrevorChristman	1****n	1
and 2 more...

Committer Domains (Top 20 + Academic)

microsoft.com: 8 fernir.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 14
Total pull requests: 115
Average time to close issues: over 2 years
Average time to close pull requests: 7 days
Total issue authors: 9
Total pull request authors: 17
Average comments per issue: 0.71
Average comments per pull request: 0.27
Merged pull requests: 85
Bot issues: 0
Bot pull requests: 2

Past Year

Issues: 0
Pull requests: 5
Average time to close issues: N/A
Average time to close pull requests: 4 days
Issue authors: 0
Pull request authors: 2
Average comments per issue: 0
Average comments per pull request: 0.2
Merged pull requests: 4
Bot issues: 0
Bot pull requests: 1

View more stats

Top Authors

Issue Authors

jackgerrits (3)
kumpera (2)
ataymano (2)
peterychang (1)
adressel (1)
slahabar (1)
TrevorChristman (1)
zwd-ms (1)
rajan-chari (1)

Pull Request Authors

bassmang (34)
jackgerrits (30)
byronxu99 (11)
olgavrou (10)
peterychang (9)
v-jameslongo (7)
homezcx (3)
ataymano (3)
zwd-ms (3)
dependabot[bot] (3)
kumpera (2)
cheng-tan (2)
lokitoth (1)
TrevorChristman (1)
lalo (1)

Top Labels

Issue Labels

bug (5) enhancement (3) documentation (2) good first issue (1)

Pull Request Labels

dependencies (3) github_actions (2)

Packages

Total packages: 2
Total downloads:
- nuget 12,312 total

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 0
(may contain duplicates)
Total versions: 16
Total maintainers: 1

nuget.org: rlnettest

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

Homepage: http://github.com/VowpalWabbit/reinforcement_learning
License: mit
Latest release: 1.0.19360008
published over 3 years ago

Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 12,312 Total

Rankings

Dependent repos count: 10.2%

Average: 13.7%

Dependent packages count: 13.9%

Downloads: 17.0%

Maintainers (1)

johnhuang

Last synced: 6 months ago

nuget.org: rl.net

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

Homepage: https://github.com/VowpalWabbit/reinforcement_learning
License: MIT
Latest release: 0.2.0+dba6ad0
published over 3 years ago

Versions: 9
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent repos count: 15.0%

Average: 17.7%

Dependent packages count: 20.3%

Last synced: 6 months ago

Dependencies

bindings/cs/rl.net/rl.net.csproj nuget

System.Memory 4.5.4

bindings/cs/rl.net.cli/rl.net.cli.csproj nuget

CommandLineParser 2.8.0
Newtonsoft.Json 12.0.3

bindings/cs/rl.net.cli.test/rl.net.cli.test.csproj nuget

MSTest.TestAdapter 1.3.2
MSTest.TestFramework 1.3.2
Microsoft.NET.Test.Sdk 15.8.0

bindings/cs/rl.net.native/packages.config nuget

boost 1.70.0.0
boost_program_options-vc141 1.70.0.0

.github/workflows/asan.yml actions

actions/checkout v2 composite
ilammy/msvc-dev-cmd v1 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/build_docs.yml actions

actions/checkout v2 composite
actions/checkout v1 composite
actions/download-artifact v1 composite
actions/upload-artifact v1 composite
ad-m/github-push-action master composite

.github/workflows/build_nuget.yml actions

actions/checkout v2 composite
actions/download-artifact v1 composite
actions/upload-artifact v1 composite
ilammy/msvc-dev-cmd v1 composite
ilammy/setup-nasm v1 composite
microsoft/setup-msbuild v1.1 composite

.github/workflows/build_python_wheels.yml actions

actions/checkout v1 composite
actions/checkout v2 composite
actions/download-artifact v1 composite
actions/upload-artifact v1 composite

.github/workflows/build_rlclientlib.yml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-dotnet v1 composite
ilammy/msvc-dev-cmd v1 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/build_vw_bp.yml actions

actions/cache v3 composite
actions/checkout v3 composite
ilammy/msvc-dev-cmd v1 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/codeql-analysis.yml actions

actions/cache v3 composite
actions/checkout v3 composite
github/codeql-action/analyze v2 composite
github/codeql-action/autobuild v2 composite
github/codeql-action/init v2 composite
ilammy/msvc-dev-cmd v1 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/daily_integration.yml actions

actions/cache v3 composite
actions/checkout v3 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/dotnet_nugets.yml actions

NuGet/setup-nuget v1.1.1 composite
actions/cache v3 composite
actions/checkout v2 composite
actions/download-artifact v3 composite
actions/setup-dotnet v3 composite
actions/upload-artifact v3 composite
actions/upload-artifact v1 composite
ilammy/msvc-dev-cmd v1 composite
lukka/get-cmake latest composite

.github/workflows/e2e_testing.yml actions

actions/checkout v1 composite

.github/workflows/lint.yml actions

actions/checkout v1 composite
actions/checkout v3 composite
actions/download-artifact v1 composite
actions/setup-python v4 composite
actions/upload-artifact v1 composite

.github/workflows/run_benchmarks.yml actions

actions/cache v3 composite
actions/checkout v3 composite
benchmark-action/github-action-benchmark v1 composite
lukka/get-cmake latest composite
lukka/run-vcpkg v10 composite

.github/workflows/vcpkg_build.yml actions

actions/cache v3 composite
actions/checkout v3 composite
lukka/get-cmake latest composite
lukka/run-cmake v10 composite
lukka/run-vcpkg v10 composite

nuget/dotnet/rl.net.nuspec nuget

nuget/dotnet/test/dotnetcore_nuget_test.csproj nuget

setup.py pypi

test_tools/e2e_testing/requirements.txt pypi

flatbuffers * test
numpy * test
zstd * test

vcpkg.json vcpkg

boost-align *
boost-asio *
boost-date-time *
boost-filesystem *
boost-interprocess *
boost-math *
boost-program-options *
boost-regex *
boost-system *
boost-test *
boost-thread *
boost-uuid *
cpprestsdk *
flatbuffers *
fmt *
openssl *
rapidjson *
spdlog *
zlib *

https://github.com/vowpalwabbit/reinforcement_learning

Science Score: 13.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

RL Client Library

Compiling the library

Getting the source code

Building with Vcpkg + CMake Presets

Requirements

Commands

Alternative system-specific build instructions

Ubuntu

Dependencies

Configure + Build

Configure

Build

Test

MacOS

Dependencies

Configure + Build

Windows

Dependencies

Configure + Build in Visual Studio

Configure with command line + Build in Visual Studio

Troubleshooting

OpenSSL on MacOS

System cpprestsdk on Linux

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

nuget.org: rlnettest

Rankings

Maintainers (1)

nuget.org: rl.net

Rankings

Dependencies