Data

Tools

Indexes

Applications

Experiments

Open Source Science

agentlego

Enhance LLM agents with rich tool APIs

https://github.com/internlm/agentlego

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 9 committers (11.1%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.5%) to scientific vocabulary

Keywords

large-language-models llm llm-agents

Keywords from Contributors

beit clip constrastive-learning convnext mae masked-image-modeling mobilenet moco multimodal pretrained-models
Last synced: 6 months ago · JSON representation ·

Repository

Enhance LLM agents with rich tool APIs

Basic Info
  • Host: GitHub
  • Owner: InternLM
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 4.44 MB
Statistics
  • Stars: 397
  • Watchers: 10
  • Forks: 34
  • Open Issues: 7
  • Releases: 2
Topics
large-language-models llm llm-agents
Created almost 3 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation

README.md

[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg)](https://openxlab.org.cn/apps/detail/mzr1996/AgentLego) [![docs](https://img.shields.io/badge/docs-latest-blue)](https://agentlego.readthedocs.io/en/latest/) [![PyPI](https://img.shields.io/pypi/v/agentlego)](https://pypi.org/project/agentlego) [![license](https://img.shields.io/github/license/InternLM/agentlego.svg)](https://github.com/InternLM/agentlego/tree/main/LICENSE) English | [简体中文](./README_zh-CN.md)

Introduction

AgentLego is an open-source library of versatile tool APIs to extend and enhance large language model (LLM) based agents, with the following highlight features:

  • Rich set of tools for multimodal extensions of LLM agents including visual perception, image generation and editing, speech processing and visual-language reasoning, etc.
  • Flexible tool interface that allows users to easily extend custom tools with arbitrary types of arguments and outputs.
  • Easy integration with LLM-based agent frameworks like LangChain, Transformers Agents, Lagent.
  • Support tool serving and remote accessing, which is especially useful for tools with heavy ML models (e.g. ViT) or special environment requirements (e.g. GPU and CUDA).

https://github-production-user-asset-6210df.s3.amazonaws.com/26739999/289006700-2140015c-b5e0-4102-bc54-9a1b4e3db9ec.mp4

Quick Starts

Installation

Install the AgentLego package

shell pip install agentlego

Install tool-specific dependencies

Some tools requires extra packages, please check the readme file of the tool, and confirm all requirements are satisfied.

For example, if we want to use the ImageDescription tool. We need to check the Set up section of readme and install the requirements.

bash pip install -U openmim mim install -U mmpretrain

Use tools directly

```Python from agentlego import listtools, loadtool

print(list_tools()) # list tools in AgentLego

imagecaptiontool = loadtool('ImageDescription', device='cuda') print(imagecaptiontool.description) image = './examples/demo.png' caption = imagecaption_tool(image) ```

Integrated into agent frameworks

Supported Tools

General ability

Speech related

Image-processing related

  • ImageDescription: Describe the input image.
  • OCR: Recognize the text from a photo.
  • VQA: Answer the question according to the image.
  • HumanBodyPose: Estimate the pose or keypoints of human in an image.
  • HumanFaceLandmark: Estimate the landmark or keypoints of human faces in an image.
  • ImageToCanny: Extract the edge image from an image.
  • ImageToDepth: Generate the depth image of an image.
  • ImageToScribble: Generate a sketch scribble of an image.
  • ObjectDetection: Detect all objects in the image.
  • TextToBbox: Detect specific objects described by the given text in the image.
  • Segment Anything series
    • SegmentAnything: Segment all items in the image.
    • SegmentObject: Segment the certain objects in the image according to the given object name.

AIGC related

Licence

This project is released under the Apache 2.0 license. Users should also ensure compliance with the licenses governing the models used in this project.

Owner

  • Name: InternLM
  • Login: InternLM
  • Kind: organization
  • Email: internlm@pjlab.org.cn
  • Location: China

Citation (CITATION.cff) 

cff-version: 1.3.1
message: "If you use this software, please cite it as below."
authors:
  - name: "AgentLego Contributors"
title: "AgentLego: Open-source tool API library to extend and enhance LLM agents"
date-released: 2023-12-31
url: "https://github.com/InternLM/agentlego"
license: Apache-2.0

GitHub Events

Total
  • Issues event: 1
  • Watch event: 58
  • Fork event: 7
Last Year
  • Issues event: 1
  • Watch event: 58
  • Fork event: 7

Committers

Last synced: 10 months ago

All Time
  • Total Commits: 119
  • Total Committers: 9
  • Avg Commits per committer: 13.222
  • Development Distribution Score (DDS): 0.437
Past Year
  • Commits: 1
  • Committers: 1
  • Avg Commits per committer: 1.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
mzr1996 m****6@1****m 67
Yining Li l****2@g****m 12
Mashiro 5****E 10
zhouzaida z****a@1****m 10
Tau t****g@o****m 9
Haian Huang(深度眸) h****n@s****m 5
YirongYan y****g@s****n 3
fanqiNO1 7****1 2
tpoisonooo k****n@a****m 1
Committer Domains (Top 20 + Academic)
163.com: 2 aliyun.com: 1 sjtu.edu.cn: 1 sensetime.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 7
  • Total pull requests: 65
  • Average time to close issues: 2 days
  • Average time to close pull requests: 7 days
  • Total issue authors: 7
  • Total pull request authors: 11
  • Average comments per issue: 0.43
  • Average comments per pull request: 0.35
  • Merged pull requests: 60
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 3
  • Pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: less than a minute
  • Issue authors: 3
  • Pull request authors: 1
  • Average comments per issue: 0.33
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • uniyushu (1)
  • Hank10032 (1)
  • Shiyao-Huang (1)
  • GavinZhang0228 (1)
  • ziky168 (1)
  • vansin (1)
  • xjspace (1)
Pull Request Authors
  • mzr1996 (12)
  • ly015 (12)
  • HAOCHENYE (10)
  • Tau-J (8)
  • hhaAndroid (5)
  • Jize-W (4)
  • fanqiNO1 (4)
  • Yanyirong (4)
  • zhouzaida (3)
  • JackWoo0831 (2)
  • tpoisonooo (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 3
  • Total downloads:
    • pypi 84 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 12
  • Total maintainers: 1
proxy.golang.org: github.com/InternLM/agentlego
  • Versions: 4
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 6.5%
Average: 6.7%
Dependent repos count: 7.0%
Last synced: 6 months ago
proxy.golang.org: github.com/internlm/agentlego
  • Versions: 4
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 6.5%
Average: 6.7%
Dependent repos count: 7.0%
Last synced: 6 months ago
pypi.org: agentlego

AgentLego is a versatile tool library for enhancing LLM-based agents.

  • Versions: 4
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 84 Last month
Rankings
Dependent packages count: 10.1%
Stargazers count: 16.1%
Forks count: 22.7%
Average: 29.0%
Dependent repos count: 67.2%
Maintainers (1)
openmmlab
Last synced: 6 months ago

Dependencies

.github/workflows/publish-to-pypi.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v1 composite
examples/visual_chatgpt/requirements.txt pypi
  • accelerate *
  • addict *
  • albumentations *
  • einops *
  • gradio *
  • imageio *
  • imageio-ffmpeg *
  • invisible-watermark *
  • langchain ==0.0.101
  • numpy *
  • omegaconf *
  • open_clip_torch *
  • openai *
  • opencv-python *
  • openmim *
  • prettytable *
  • safetensors *
  • streamlit *
  • test-tube *
  • torchmetrics *
  • transformers *
  • webdataset *
  • wget ==3.2
  • yapf *
pyproject.toml pypi
requirements/docs.txt pypi
  • docutils ==0.18.1
  • modelindex *
  • myst-parser *
  • pytorch_sphinx_theme *
  • sphinx ==6.1.3
  • sphinx-copybutton *
  • sphinx-notfound-page *
  • sphinx-tabs *
  • sphinxcontrib-jquery *
  • tabulate *
requirements/optional.txt pypi
  • fastapi *
  • openai *
  • python-multipart *
  • sentence-transformers *
  • torch *
  • torchaudio *
  • torchvision *
  • typing-extensions *
  • uvicorn *
requirements/runtime.txt pypi
  • Pillow *
  • func_timeout *
  • mmengine >=0.8
  • numpy *
  • opencv-python *
  • packaging *
  • prompt_toolkit *
  • requests *
  • thefuzz *
  • tqdm *
requirements/server.txt pypi
  • fastapi *
  • python-multipart *
  • typing-extensions *
  • uvicorn *
requirements.txt pypi