Data

Tools

Indexes

Applications

Experiments

Open Source Science

kor

LLM(😽)

https://github.com/eyurtsev/kor

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.6%) to scientific vocabulary

Keywords

information-extraction llm natural-language natural-language-processing natural-language-understanding

Keywords from Contributors

agents cryptocurrencies transformers gemini interactive optim anthropic embedded multi-agents langchain
Last synced: 6 months ago · JSON representation

Repository

LLM(😽)

Basic Info
Statistics
  • Stars: 1,688
  • Watchers: 15
  • Forks: 95
  • Open Issues: 27
  • Releases: 20
Topics
information-extraction llm natural-language natural-language-processing natural-language-understanding
Created about 3 years ago · Last pushed about 1 year ago
Metadata Files
Readme Contributing License

README.md

Unit Tests Test Docs Release Notes Downloads Open Issues

[!WARNING] If you're using a chat model that supports a tool calling API, you should probably be using the chat models' tool calling API instead of Kor!

Kor is best suited for old style LLMs that did not have a chat interface and did not support tool calling.

Read the tool calling guide in LangChain for more details.

Please refer to the chat model integration table for a list of chat models that support native tool calling.

Kor

This is a half-baked prototype that "helps" you extract structured data from text using LLMs 🧩.

Specify the schema of what should be extracted and provide some examples.

Kor will generate a prompt, send it to the specified LLM and parse out the output.

You might even get results back.

So yes – it’s just another wrapper on top of LLMs with its own flavor of abstractions. 😸

See documentation.

Integrated with the LangChain framework 😽💗 🦜🔗.

Kor vs. LangChain

There are 3 different approaches for extracting information using LLMs:

  1. prompt based/parsing
  2. function/tool calling
  3. JSON mode

Please see the LangChain extraction use case docs for an overview.

Kor has a pretty good implementation of the parsing approach. The approach works with all good-enough LLMs regardless of whether they support function/tool calling or JSON modes.

Extraction quality is principally driven by providing good reference examples and good schema documentation.

Please see guidelines here and here.

Version 1.0.0 Release

  • kor compatible with both pydantic v2 and v1.
  • pydantic v2 had significant breaking changes w/ respect to v1, kor major version bump was used as a precaution.

Main things to watch out for:

  1. Use a default value for any Optional fields if using pydantic v2 for validation.

python class MusicRequest(BaseModel): song: Optional[List[str]] = Field( default=None, description="The song(s) that the user would like to be played." )

  1. Kor schema is typed checked using pydantic. Pydantic v2 is stricter, and may catch issues that were hiding in existing user code that was using the kor library.

  2. Serialization has not yet been implemented with pydantic v2.

Kor style schema

```python from langchain.chatmodels import ChatOpenAI from kor import createextraction_chain, Object, Text

llm = ChatOpenAI( modelname="gpt-3.5-turbo", temperature=0, maxtokens=2000, modelkwargs = { 'frequencypenalty':0, 'presencepenalty':0, 'topp':1.0 } )

schema = Object( id="player", description=( "User is controlling a music player to select songs, pause or start them or play" " music by a particular artist." ), attributes=[ Text( id="song", description="User wants to play this song", examples=[], many=True, ), Text( id="album", description="User wants to play this album", examples=[], many=True, ), Text( id="artist", description="Music by the given artist", examples=[("Songs by paul simon", "paul simon")], many=True, ), Text( id="action", description="Action to take one of: play, stop, next, previous.", examples=[ ("Please stop the music", "stop"), ("play something", "play"), ("play a song", "play"), ("next song", "next"), ], ), ], many=False, )

chain = createextractionchain(llm, schema, encoderorencoder_class='json') chain.invoke("play songs by paul simon and led zeppelin and the doors")['data'] ```

python {'player': {'artist': ['paul simon', 'led zeppelin', 'the doors']}}

Pydantic style schema

```python class Action(enum.Enum): play = "play" stop = "stop" previous = "previous" next_ = "next"

class MusicRequest(BaseModel): song: Optional[List[str]] = Field( default=None, description="The song(s) that the user would like to be played." ) album: Optional[List[str]] = Field( default=None, description="The album(s) that the user would like to be played." ) artist: Optional[List[str]] = Field( default=None, description="The artist(s) whose music the user would like to hear.", examples=[("Songs by paul simon", "paul simon")], ) action: Optional[Action] = Field( default=None, description="The action that should be taken; one of play, stop, next, previous", examples=[ ("Please stop the music", "stop"), ("play something", "play"), ("play a song", "play"), ("next song", "next"), ], )

schema, validator = frompydantic(MusicRequest)
chain = createextractionchain( llm, schema, encoderorencoderclass="json", validator=validator ) chain.invoke("stop the music now")["validated_data"] ```

python MusicRequest(song=None, album=None, artist=None, action=<Action.stop: 'stop'>)

Compatibility

Kor is tested against python 3.8, 3.9, 3.10, 3.11.

Installation

sh pip install kor

💡 Ideas

Ideas of some things that could be done with Kor.

  • Extract data from text that matches an extraction schema.
  • Power an AI assistant with skills by precisely understanding a user request.
  • Provide natural language access to an existing API.

🚧 Prototype

Prototype! So the API is not expected to be stable!

✨ What does Kor excel at? 🌟

  • Making mistakes! Plenty of them!
  • Slow! It uses large prompts with examples, and works best with the larger slower LLMs.
  • Crashing for long enough pieces of text! Context length window could become limiting when working with large forms or long text inputs.

The expectation is that as LLMs improve some of these issues will be mitigated.

Limitations

Kor has no limitations. (Just kidding.)

Take a look at the section above and at the compatibility section.

Got Ideas?

Open an issue, and let's discuss!

🎶 Why the name?

Fast to type and sufficiently unique.

Contributing

If you have any ideas or feature requests, please open an issue and share!

See CONTRIBUTING.md for more information.

Other packages

Probabilistically speaking this package is unlikely to work for your use case.

So here are some great alternatives:

Owner

  • Name: Eugene Yurtsev
  • Login: eyurtsev
  • Kind: user
  • Location: Boston
  • Company: @langchain-ai

SWE | ML | Chief Data Kitten

GitHub Events

Total
  • Issues event: 2
  • Watch event: 72
  • Delete event: 1
  • Issue comment event: 5
  • Push event: 1
  • Pull request event: 6
  • Fork event: 9
  • Create event: 4
Last Year
  • Issues event: 2
  • Watch event: 72
  • Delete event: 1
  • Issue comment event: 5
  • Push event: 1
  • Pull request event: 6
  • Fork event: 9
  • Create event: 4

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 231
  • Total Committers: 12
  • Avg Commits per committer: 19.25
  • Development Distribution Score (DDS): 0.074
Past Year
  • Commits: 11
  • Committers: 2
  • Avg Commits per committer: 5.5
  • Development Distribution Score (DDS): 0.091
Top Committers
Name Email Commits
Eugene Yurtsev e****v@g****m 214
dependabot[bot] 4****] 5
Boris Wilhelms m****l@w****e 3
Vadym Barda v****a@g****m 1
Tom Dyson t****m@t****m 1
Stephen Witkowski s****i@p****m 1
Seb0 S****0 1
Sachin Bhat s****7@g****m 1
Rishabh Jain r****8@g****m 1
Ikko Eltociear Ashimine e****r@g****m 1
Harrison Chase h****7@g****m 1
Chaim Turkel c****l@g****m 1
Committer Domains (Top 20 + Academic)
torchbox.com: 1 wille-zone.de: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 81
  • Total pull requests: 160
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 14 days
  • Total issue authors: 60
  • Total pull request authors: 14
  • Average comments per issue: 3.05
  • Average comments per pull request: 0.46
  • Merged pull requests: 119
  • Bot issues: 0
  • Bot pull requests: 33
Past Year
  • Issues: 5
  • Pull requests: 7
  • Average time to close issues: about 7 hours
  • Average time to close pull requests: 35 minutes
  • Issue authors: 5
  • Pull request authors: 2
  • Average comments per issue: 1.8
  • Average comments per pull request: 0.14
  • Merged pull requests: 3
  • Bot issues: 0
  • Bot pull requests: 3
View more stats
Top Authors
Issue Authors
  • eyurtsev (7)
  • hitsense (3)
  • MauroLondon (2)
  • Feya (2)
  • nans-here (2)
  • shatealaboxiaowang (2)
  • mann2107 (2)
  • smwitkowski (2)
  • Sachin-Bhat (2)
  • BorisWilhelms (2)
  • DavideBFerri (2)
  • oliverbj (2)
  • rlancemartin (1)
  • ihorizons2022 (1)
  • NicoSzela (1)
Pull Request Authors
  • eyurtsev (124)
  • dependabot[bot] (51)
  • sshussain270 (2)
  • chaimt (2)
  • PrasannaIITM (2)
  • BorisWilhelms (2)
  • Sachin-Bhat (2)
  • smwitkowski (1)
  • nfcampos (1)
  • tomdyson (1)
  • eltociear (1)
  • Seb0 (1)
  • rishabhjain1198 (1)
  • Zakharyan9889 (1)
Top Labels
Issue Labels
enhancement (9) bug (2) good first issue (2) pydantic compatibility (1)
Pull Request Labels
dependencies (51) release (18) doc-publish (8)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 39,691 last-month
  • Total docker downloads: 4,564
  • Total dependent packages: 10
    (may contain duplicates)
  • Total dependent repositories: 53
    (may contain duplicates)
  • Total versions: 36
  • Total maintainers: 1
pypi.org: fcsparser

A python package for reading raw fcs files

  • Versions: 14
  • Dependent Packages: 9
  • Dependent Repositories: 45
  • Downloads: 13,552 Last month
  • Docker Downloads: 4,564
Rankings
Dependent packages count: 1.1%
Stargazers count: 1.8%
Docker downloads count: 1.9%
Dependent repos count: 2.2%
Average: 2.5%
Downloads: 2.9%
Forks count: 5.2%
Maintainers (1)
eyurtsev
Last synced: 6 months ago
pypi.org: kor

Extract information with LLMs from text

  • Versions: 22
  • Dependent Packages: 1
  • Dependent Repositories: 8
  • Downloads: 26,139 Last month
Rankings
Stargazers count: 1.8%
Downloads: 1.9%
Average: 4.8%
Dependent repos count: 5.2%
Forks count: 5.2%
Dependent packages count: 10.1%
Maintainers (1)
eyurtsev
Last synced: 6 months ago

Dependencies

.github/workflows/doc_publish.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • peaceiris/actions-gh-pages v3 composite
.github/workflows/doc_test.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/test.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
poetry.lock pypi
  • 171 dependencies
pyproject.toml pypi
  • langchain >=0.0.110
  • markdownify ^0.11.6
  • openai ^0.27
  • pandas ^1.5.3
  • python ^3.8.1