Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of judy

judy - 2.1.0

What's Changed

  • Updated eval prompts to be optionally configurable in eval config by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/40
  • Fixes judy web app running in debg mode by default by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/40
  • Adds new RAG evaluation scenario by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/40
  • Updated pagination bar in raw results page of web app by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/40

Full Changelog: https://github.com/TNT-Hoopsnake/judy/compare/2.0.0...2.1.0

- Python
Published by tnt-rahul-raghavan about 2 years ago

judy - [2.0.0] - 2024-01-31

What's Changed

  • Adds async concurrent evaluations by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/31
  • Adds evaluation of closed-source models by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/31
  • Adds cost/usage tracking by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/31
  • Changes cli config command to no longer be interactive by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/31
  • Adds cli config list command by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/31
  • Adds validation checks for common config issues by @tnt-rahul-raghavan
  • Fixes error when using openai key specified in run config by @tnt-rahul-raghavan
  • Updates pipeline to only attempt saving successful results by @tnt-rahul-raghavan
  • Updates README with screenshots by @tnt-rahul-raghavan
  • Updates cli to print tracebacks to errors.log file by @tnt-rahul-raghavan

Full Changelog: https://github.com/TNT-Hoopsnake/judy/compare/1.0.1...2.0.0

- Python
Published by tnt-rahul-raghavan about 2 years ago

judy - [1.0.1] - 2024-01-31

What's Changed

  • Bump aiohttp from 3.8.6 to 3.9.0 by @dependabot in https://github.com/TNT-Hoopsnake/judy/pull/30
  • Create LICENSE by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/32
  • Update issue templates by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/33
  • Update issue templates by @tnt-rahul-raghavan in https://github.com/TNT-Hoopsnake/judy/pull/34
  • Bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in https://github.com/TNT-Hoopsnake/judy/pull/35

Full Changelog: https://github.com/TNT-Hoopsnake/judy/compare/1.0.0...1.0.1

- Python
Published by tnt-rahul-raghavan about 2 years ago

judy - [1.0.0] - 2023-12-12

[1.0.0] - 2023-12-12

Added

  • cli app to load data from datasets, construct prompts and run evaluations on model responses
  • web app to display results and view graphs for aggregated results
  • a set of common scenarios to use for evaluation (flask, instruction following, question answering, disinformation)

- Python
Published by tnt-rahul-raghavan about 2 years ago