Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of mallm

mallm - v1.0.5

What's Changed

  • Feat/new response generators by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/149
  • vllm fixes by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/150
  • Debate fix by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/151
  • summary renamed to judge by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/152
  • Nicer charts by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/153

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.4...v1.0.5

- Python
Published by jonas-becker 8 months ago

mallm - v1.0.4

What's Changed

  • Aqua rat by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/145
  • fix fstring by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/146
  • toml PEP 621 complicance + ifeval by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/147
  • Judge Agent by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/148

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.3...v1.0.4

- Python
Published by jonas-becker about 1 year ago

mallm - v1.0.3

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.2...v1.0.3

- Python
Published by lkaesberg about 1 year ago

mallm - v1.0.2

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.1...v1.0.2

- Python
Published by lkaesberg about 1 year ago

mallm - v1.0.1

What's Changed

  • ifeval evaluation (instruction following) by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/139
  • no duplicate answer choices by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/143
  • Feat/challenge results by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/144

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.0...v1.0.1

Added release on pypi

- Python
Published by lkaesberg about 1 year ago

mallm - Public Release

What's Changed

  • Create update_readme.yml by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/72
  • add workflow py script by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/73
  • Added QA metric for GPQA, MMLU, etc. by @ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/74
  • Small Code Style Improvements by @ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/77
  • Fix readme updater by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/78
  • feat: added batch executor for mallm by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/81
  • ResponseGenerators: Handling prompts, extraction, and agreements by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/79
  • More robust multichoice metric + fixed datasets by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/82
  • Prompt improvements (+ majority consensus fix) by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/85
  • Fix/no unlimited voting by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/88
  • Extensive evaluation by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/92
  • 1) sort output file after finishing 2) comparable moderator by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/91
  • feat: When there is a dataset issue, print exactly what the issue is by @jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/98
  • Support hf datasets by @jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/93
  • fix: Fix a bug where forgetting trailing slash in memory bucket leads to undesired behaviour by @jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/100
  • Ablation by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/99
  • Refactor/evaluator by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/90
  • fix: Fix a bug where the HF dataset is not sorted properly. by @jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/102
  • Fix/data load and debug output by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/105
  • Feat/plotting by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/106
  • squad metric enhancements by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/107
  • Paraphrase types agent generator by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/103
  • fix out full path by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/109
  • Feat/rich output by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/110
  • fix: unanimity is unreachable because of faulty condition by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/108
  • Prompt changes and minor adjustments by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/114
  • Distinct-N metric by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/116
  • Remove dbm memory by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/115
  • feat: added instruction templates by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/118
  • batch executor refinements by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/113
  • More expressive param names + flexible number of neutral agents by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/119
  • feat: all agents generate a first draft and after that improve by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/120
  • feat: added mmlu pro dataset by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/123
  • feat: add musr dataset by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/125
  • feat: add math lvl 5 dataset downloader by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/126
  • Freely combine persona types + NoPersonaGenerator by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/122
  • feat: added prompts for new datasets by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/127
  • feat: add mallm command line scripts by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/128
  • Feat/evaluator with alterations by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/129
  • feat: added metric that checks if answer is included in response by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/131
  • feat: added discord webhook to batch processing by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/130
  • feat: add mmlu dataset by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/132
  • BBQ, MoCa, MoralExceptQA Datasets + metadata field by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/133
  • Feat/summarize decision protocol by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/124
  • policy feedback agent by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/134
  • Persona diversity index by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/136
  • WinoGrande, ETHICS datasets by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/137
  • Consensus Voting by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/138
  • feat: add new commandline script to execute mallm batch mode by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/140
  • feat: add challenge of final answer to test consistency by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/141
  • update readme by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/142

New Contributors

  • @jpwahle made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/98

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/compare/v0.1.0-alpha...v1.0.0

- Python
Published by jonas-becker over 1 year ago

mallm - v0.1.0-alpha

This is the first release of MALLM (alpha).

What's Changed

  • Fix/setup by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/1
  • memory redesign, more extensive output logs, refactoring, bug fixes by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/2
  • Feat/new build system by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/3
  • refactor: add abstract class for datasets by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/4
  • feat: improve readme by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/6
  • Feat/logging by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/7
  • Feat/formatting by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/8
  • Added GPQA Dataset, cffi dependency for linux by @ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/10
  • Tgi implementation by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/9
  • reorganized files by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/16
  • Multi source datasets by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/19
  • Create unit test by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/21
  • Fix/discussion by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/17
  • Refactor/coordinator by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/18
  • installable package by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/25
  • small fixes by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/26
  • WMT119, paraphrase types, and fixes by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/28
  • Feat/btvote by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/27
  • fix etpc and context by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/35
  • fix json stringify by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/36
  • Feat/decision protocol by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/30
  • feat: added agent history as a chat format by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/38
  • samples left logging and type hints by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/37
  • stream answers to reduce memory usage by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/40
  • Strict types by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/42
  • Refactor/GitHub action by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/45
  • Openai support by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/47
  • feat: added tests for coordinator by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/46
  • Refactor/moderator by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/44
  • Evaluation framework by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/50
  • Feat: fixed number of turns by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/53
  • Improve prompts by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/55
  • Feat baseline by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/56
  • Added IPIP Persona Generator by @ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/59
  • fix missing hftoken, btvote not shuffled, maxsamples by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/57
  • Discussion length by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/58
  • add pytest pre commit hook by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/60
  • Added more decision protocols by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/49
  • Split agree and answer by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/54
  • refactor: introduce config dataclass to remove duplicate code and mak… by @lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/64
  • Improve extraction reliability by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/63
  • failed samples logging by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/65
  • Stability fixes by @jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/66

New Contributors

  • @lkaesberg made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/1
  • @jonas-becker made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/2

Full Changelog: https://github.com/Multi-Agent-LLMs/mallm/commits/v0.1.0-alpha

- Python
Published by jonas-becker over 1 year ago