https://github.com/aksw/llm-kg-bench - 3.0.0 - 2025-03

Release of version 3.0

• changed: moved benchmark tasks from LlmKgBench.bench to LlmKgBench.tasks
• all from 3.0-RC1, especially
  • changed: major update of task api
  • changed: encrypted and updated task data files
  • changed: model connectors in separate files within modelConnectors directory
  • added: vllm model connector
  • added: task RdfSyntaxFixList
  • added: capability compass
  • added: t-test (turtle vs. jsonld)
  • added pytest tests
• updated documentation

- Python
Published by lpmeyer 11 months ago

https://github.com/aksw/llm-kg-bench - 3.0.0-RC1

Release Candidate 1 for version 3.0

• changed: updated task api classes and interface definition
• changed: task data files are encrypted zip files now
• added vllm model connector
• added task RdfSyntaxFixList for turtle, json-ld and ntriples and added support for rdf-xml and ntriples for RdfConnectionExplainStatic and RdfFriendCount task
• added capability compass diagram on aggregated task scores

for more see CHANGELOG.md

- Python
Published by lpmeyer about 1 year ago

https://github.com/aksw/llm-kg-bench - 3.0.0 RC1

Release Candidate 1 for version 3.0

• changed: updated task api classes and interface definition
• changed: task data files are encrypted zip files now
• added vllm model connector
• added task RdfSyntaxFixList for turtle, json-ld and ntriples and added support for rdf-xml and ntriples for RdfConnectionExplainStatic and RdfFriendCount task
• added capability compass diagram on aggregated task scores

for more see CHANGELOG.md

- Python
Published by lpmeyer about 1 year ago

https://github.com/aksw/llm-kg-bench - 2.0.0

Release of version 2.0 * changed: some task class names * changed: renamed config section test to tasks * added: support for task configuration * added: support for nested score and info entries * added: SPARQL evaluation tasks

for more see CHANGELOG.md

- Python
Published by lpmeyer over 1 year ago

https://github.com/aksw/llm-kg-bench - 1.2.0

release for ESWC 2024 paper submission

- Python
Published by lpmeyer about 2 years ago

https://github.com/aksw/llm-kg-bench - 1.1.0

release for DL4KG 2023 Turtle KG evaluation paper submission

- Python
Published by JJ-Author over 2 years ago

https://github.com/aksw/llm-kg-bench - 1.0.0

initial release for zenodo archiving

- Python
Published by lpmeyer over 2 years ago