citationfinder
The project aims to automate the process of generating relevant citations for a given text and conducting literature review for researchers.
Science Score: 31.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.0%) to scientific vocabulary
Last synced: 11 months ago
·
JSON representation
·
Repository
The project aims to automate the process of generating relevant citations for a given text and conducting literature review for researchers.
Basic Info
- Host: GitHub
- Owner: bhaskatripathi
- Language: Jupyter Notebook
- Default Branch: main
- Size: 47.9 KB
Statistics
- Stars: 3
- Watchers: 2
- Forks: 1
- Open Issues: 0
- Releases: 0
Created over 3 years ago
· Last pushed over 3 years ago
Metadata Files
Readme
Citation
README.md
CitationFinder
The project aims to automate the process of generating relevant citations for a given text and conducting literature review for researchers.
Research Problem:
As a researcher, it is often necessary to substantiate one's statements with existing literature by providing relevant citations. This can be a time-consuming and labor-intensive process, particularly when searching for citations that are both relevant and up-to-date. Moreover, literature review can be slow, making it difficult to write and publish research papers in a timely manner. Additionally, while researchers may have past knowledge on a topic, linking it to the existing literature can be a challenging task, especially when searching for the relevant citations manually.
The Solution:
To solve these problems, we propose a tool that uses both semantic and AI search techniques to generate relevant citations for a given text. The tool searches the Crossref database using a combination of text-based queries and extracted keywords to identify relevant literature. The search results are then ranked based on their relevance score, which is generated using AI techniques. The most relevant citations are presented to the user in a user-friendly format, allowing them to quickly and easily review the literature and add relevant citations to their research paper. With this tool, researchers can save time and effort by quickly finding and selecting relevant citations, improving their research process, and enabling them to publish their work more efficiently. Additionally, the tool can aid researchers in identifying new research areas by surfacing relevant literature, helping them stay up-to-date with the latest developments in their field.
Demo:
UML:
sequenceDiagram participant User participant API participant Crossref participant OpenAI participant Pandas
User->>+API: Define Query Text
API->>+Crossref: Send Query Text
Crossref->>-API: Send Data
API->>+Pandas: Create DataFrame
Pandas->>-API: Send DataFrame
API->>+OpenAI: Generate Keywords
OpenAI->>-API: Send Keywords
API->>+Crossref: Send Query with Keywords
Crossref->>-API: Send Data
API->>+OpenAI: Extract Relevance Score
OpenAI->>-API: Send Score
API->>+Pandas: Sort DataFrame by Score
Pandas->>-API: Send Sorted DataFrame
User->>+API: Get Citations
API->>+User: Display Citations
Owner
- Name: Bhaskar Tripathi
- Login: bhaskatripathi
- Kind: user
- Location: India
- Website: www.bhaskartripathi.com
- Twitter: bt_veeru
- Repositories: 57
- Profile: https://github.com/bhaskatripathi
I like to build stuff (fast, valuable and not built earlier)
Citation (CitationFinder.ipynb)
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/bhaskatripathi/CitationFinder/blob/main/CitationFinder.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "VSVbRvFhOMhU",
"outputId": "68c6df6b-6435-43eb-ea58-f4d7e1f5b2ff"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting pybtex\n",
" Downloading pybtex-0.24.0-py2.py3-none-any.whl (561 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m561.4/561.4 KB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pybtex-apa-style\n",
" Downloading pybtex_apa_style-1.3-py3-none-any.whl (6.4 kB)\n",
"Collecting scholarly\n",
" Downloading scholarly-1.7.11-py3-none-any.whl (39 kB)\n",
"Collecting openai\n",
" Downloading openai-0.27.2-py3-none-any.whl (70 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.1/70.1 KB\u001b[0m \u001b[31m13.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.9/dist-packages (from pybtex) (1.16.0)\n",
"Requirement already satisfied: PyYAML>=3.01 in /usr/local/lib/python3.9/dist-packages (from pybtex) (6.0)\n",
"Collecting latexcodec>=1.0.4\n",
" Downloading latexcodec-2.0.1-py2.py3-none-any.whl (18 kB)\n",
"Collecting sphinx-rtd-theme\n",
" Downloading sphinx_rtd_theme-1.2.0-py2.py3-none-any.whl (2.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.8/2.8 MB\u001b[0m \u001b[31m39.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.9/dist-packages (from scholarly) (4.11.2)\n",
"Requirement already satisfied: requests[socks] in /usr/local/lib/python3.9/dist-packages (from scholarly) (2.27.1)\n",
"Collecting free-proxy\n",
" Downloading free_proxy-1.1.1.tar.gz (5.1 kB)\n",
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from scholarly) (4.5.0)\n",
"Collecting python-dotenv\n",
" Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)\n",
"Collecting bibtexparser\n",
" Downloading bibtexparser-1.4.0.tar.gz (51 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m51.9/51.9 KB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting selenium\n",
" Downloading selenium-4.8.3-py3-none-any.whl (6.5 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 MB\u001b[0m \u001b[31m79.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting fake-useragent\n",
" Downloading fake_useragent-1.1.3-py3-none-any.whl (50 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 KB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting httpx\n",
" Downloading httpx-0.23.3-py3-none-any.whl (71 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.5/71.5 KB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting deprecated\n",
" Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)\n",
"Collecting arrow\n",
" Downloading arrow-1.2.3-py3-none-any.whl (66 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.4/66.4 KB\u001b[0m \u001b[31m11.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting aiohttp\n",
" Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m15.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from openai) (4.65.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests[socks]->scholarly) (2022.12.7)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests[socks]->scholarly) (3.4)\n",
"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests[socks]->scholarly) (2.0.12)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests[socks]->scholarly) (1.26.15)\n",
"Collecting async-timeout<5.0,>=4.0.0a3\n",
" Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
"Collecting yarl<2.0,>=1.0\n",
" Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m264.6/264.6 KB\u001b[0m \u001b[31m35.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (22.2.0)\n",
"Collecting aiosignal>=1.1.2\n",
" Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
"Collecting multidict<7.0,>=4.5\n",
" Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.2/114.2 KB\u001b[0m \u001b[31m16.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting frozenlist>=1.1.1\n",
" Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m158.8/158.8 KB\u001b[0m \u001b[31m23.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: python-dateutil>=2.7.0 in /usr/local/lib/python3.9/dist-packages (from arrow->scholarly) (2.8.2)\n",
"Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.9/dist-packages (from beautifulsoup4->scholarly) (2.4)\n",
"Requirement already satisfied: pyparsing>=2.0.3 in /usr/local/lib/python3.9/dist-packages (from bibtexparser->scholarly) (3.0.9)\n",
"Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.9/dist-packages (from deprecated->scholarly) (1.14.1)\n",
"Requirement already satisfied: importlib-resources>=5.0 in /usr/local/lib/python3.9/dist-packages (from fake-useragent->scholarly) (5.12.0)\n",
"Requirement already satisfied: lxml in /usr/local/lib/python3.9/dist-packages (from free-proxy->scholarly) (4.9.2)\n",
"Collecting rfc3986[idna2008]<2,>=1.3\n",
" Downloading rfc3986-1.5.0-py2.py3-none-any.whl (31 kB)\n",
"Collecting sniffio\n",
" Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)\n",
"Collecting httpcore<0.17.0,>=0.15.0\n",
" Downloading httpcore-0.16.3-py3-none-any.whl (69 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m69.6/69.6 KB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.9/dist-packages (from requests[socks]->scholarly) (1.7.1)\n",
"Collecting trio~=0.17\n",
" Downloading trio-0.22.0-py3-none-any.whl (384 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m384.9/384.9 KB\u001b[0m \u001b[31m39.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting trio-websocket~=0.9\n",
" Downloading trio_websocket-0.10.2-py3-none-any.whl (17 kB)\n",
"Collecting sphinxcontrib-jquery!=3.0.0,>=2.0.0\n",
" Downloading sphinxcontrib_jquery-4.1-py2.py3-none-any.whl (121 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.1/121.1 KB\u001b[0m \u001b[31m17.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: sphinx<7,>=1.6 in /usr/local/lib/python3.9/dist-packages (from sphinx-rtd-theme->scholarly) (3.5.4)\n",
"Requirement already satisfied: docutils<0.19 in /usr/local/lib/python3.9/dist-packages (from sphinx-rtd-theme->scholarly) (0.16)\n",
"Collecting h11<0.15,>=0.13\n",
" Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 KB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting anyio<5.0,>=3.0\n",
" Downloading anyio-3.6.2-py3-none-any.whl (80 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m80.6/80.6 KB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.9/dist-packages (from importlib-resources>=5.0->fake-useragent->scholarly) (3.15.0)\n",
"Requirement already satisfied: sphinxcontrib-htmlhelp in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (2.0.1)\n",
"Requirement already satisfied: babel>=1.3 in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (2.12.1)\n",
"Requirement already satisfied: sphinxcontrib-serializinghtml in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.1.5)\n",
"Requirement already satisfied: setuptools in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (67.6.1)\n",
"Requirement already satisfied: sphinxcontrib-qthelp in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.0.3)\n",
"Requirement already satisfied: sphinxcontrib-applehelp in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.0.4)\n",
"Requirement already satisfied: snowballstemmer>=1.1 in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (2.2.0)\n",
"Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (23.0)\n",
"Requirement already satisfied: Pygments>=2.0 in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (2.14.0)\n",
"Requirement already satisfied: Jinja2>=2.3 in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (3.1.2)\n",
"Requirement already satisfied: imagesize in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.4.1)\n",
"Requirement already satisfied: alabaster<0.8,>=0.7 in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (0.7.13)\n",
"Requirement already satisfied: sphinxcontrib-devhelp in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.0.2)\n",
"Requirement already satisfied: sphinxcontrib-jsmath in /usr/local/lib/python3.9/dist-packages (from sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (1.0.1)\n",
"Collecting outcome\n",
" Downloading outcome-1.2.0-py2.py3-none-any.whl (9.7 kB)\n",
"Collecting async-generator>=1.9\n",
" Downloading async_generator-1.10-py3-none-any.whl (18 kB)\n",
"Requirement already satisfied: exceptiongroup>=1.0.0rc9 in /usr/local/lib/python3.9/dist-packages (from trio~=0.17->selenium->scholarly) (1.1.1)\n",
"Requirement already satisfied: sortedcontainers in /usr/local/lib/python3.9/dist-packages (from trio~=0.17->selenium->scholarly) (2.4.0)\n",
"Collecting wsproto>=0.14\n",
" Downloading wsproto-1.2.0-py3-none-any.whl (24 kB)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from Jinja2>=2.3->sphinx<7,>=1.6->sphinx-rtd-theme->scholarly) (2.1.2)\n",
"Building wheels for collected packages: bibtexparser, free-proxy\n",
" Building wheel for bibtexparser (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for bibtexparser: filename=bibtexparser-1.4.0-py3-none-any.whl size=42444 sha256=3d610f3e8c60ce94bc06a1cbe5249dabb921c9225be79840b7d794707f7e3354\n",
" Stored in directory: /root/.cache/pip/wheels/83/e1/e3/2311be27728119eefd014e0a6039eee58470560d8ab31fd1fa\n",
" Building wheel for free-proxy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for free-proxy: filename=free_proxy-1.1.1-py3-none-any.whl size=5652 sha256=1f3d01418bc9944744a4868477d084dae8edf4c322e27c0cf22f37f778361bc6\n",
" Stored in directory: /root/.cache/pip/wheels/a9/38/db/3efef5f071876e3a6888f9933d636c6b764655a96a32d78f4e\n",
"Successfully built bibtexparser free-proxy\n",
"Installing collected packages: rfc3986, pybtex-apa-style, sniffio, python-dotenv, outcome, multidict, latexcodec, h11, frozenlist, deprecated, bibtexparser, async-timeout, async-generator, yarl, wsproto, trio, pybtex, free-proxy, fake-useragent, arrow, anyio, aiosignal, trio-websocket, sphinxcontrib-jquery, httpcore, aiohttp, sphinx-rtd-theme, selenium, openai, httpx, scholarly\n",
"Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 anyio-3.6.2 arrow-1.2.3 async-generator-1.10 async-timeout-4.0.2 bibtexparser-1.4.0 deprecated-1.2.13 fake-useragent-1.1.3 free-proxy-1.1.1 frozenlist-1.3.3 h11-0.14.0 httpcore-0.16.3 httpx-0.23.3 latexcodec-2.0.1 multidict-6.0.4 openai-0.27.2 outcome-1.2.0 pybtex-0.24.0 pybtex-apa-style-1.3 python-dotenv-1.0.0 rfc3986-1.5.0 scholarly-1.7.11 selenium-4.8.3 sniffio-1.3.0 sphinx-rtd-theme-1.2.0 sphinxcontrib-jquery-4.1 trio-0.22.0 trio-websocket-0.10.2 wsproto-1.2.0 yarl-1.8.2\n"
]
}
],
"source": [
"pip install pybtex pybtex-apa-style scholarly openai "
]
},
{
"cell_type": "code",
"source": [
"\n",
"import requests\n",
"import pandas as pd\n",
"import re\n",
"import openai\n",
"from pybtex.database import BibliographyData, Entry\n",
"from pybtex.style.formatting import plain\n",
"\n",
"# Define the text you want to fetch citations for\n",
"#query_text = \"Bitcoin liquidity\"\n",
"query_text =\"Liquidity, unlike other trade analysis indicators, has no fixed value. As a result, calculating the exact liquidity of the exchange or market is difficult. However, there are other signs that can be used as proxies for liquidity in cryptocurrencies.\"\n",
"\n",
"# Use OpenAI API to extract relevant keywords from the query text\n",
"openai_api_key = \"Your_API_Key_Here\"\n",
"openai.api_key = openai_api_key\n",
"openai_model = \"text-davinci-003\"\n",
"openai_response = openai.Completion.create(\n",
" engine=openai_model,\n",
" prompt=f\"Generate keywords for the following text:\\n\\n{query_text}\\n\\nKeywords:\",\n",
" max_tokens=200,\n",
" n=1,\n",
" stop=None,\n",
" temperature=0.7,\n",
")\n",
"keywords = openai_response.choices[0].text.strip().split(\"\\n\")\n",
"\n",
"# Combine query text and extracted keywords to form a new query\n",
"new_query_text = f\"{query_text} {' '.join(keywords)}\"\n",
"#print(new_query_text)\n",
"\n",
"\n",
"# Define the Crossref API URL\n",
"crossref_url = \"https://api.crossref.org/works?query=\"\n",
"\n",
"# Fetch data from Crossref API\n",
"response = requests.get(crossref_url + query_text).json()\n",
"\n",
"# Extract desired information and create dataframe\n",
"df = pd.DataFrame({\n",
" 'Title': [item.get('title', '') for item in response.get('message', {}).get('items', [])],\n",
" 'Author(s)': [', '.join([author.get('given', '') + ' ' + author.get('family', '') for author in item.get('author', [])]) for item in response.get('message', {}).get('items', [])],\n",
" 'Year': [item.get('created', {}).get('date-parts', [[None]])[0][0] for item in response.get('message', {}).get('items', [])],\n",
" 'Journal': [item.get('container-title', [''])[0] for item in response.get('message', {}).get('items', [])],\n",
" 'Volume': [item.get('volume', '') for item in response.get('message', {}).get('items', [])],\n",
" 'Issue': [item.get('issue', '') for item in response.get('message', {}).get('items', [])],\n",
" 'Page': [item.get('page', '') for item in response.get('message', {}).get('items', [])],\n",
" 'DOI': [item.get('DOI', '') for item in response.get('message', {}).get('items', [])],\n",
" 'Abstract': [item.get('abstract', '') for item in response.get('message', {}).get('items', [])],\n",
"})\n",
"# Prefix https:// to DOIs\n",
"df['DOI'] = df['DOI'].apply(lambda doi: 'https://doi.org/' + doi if doi else '')\n",
"\n",
"# Sort the DataFrame by the latest year to the earliest year\n",
"df = df.sort_values(by='Year', ascending=False)\n",
"\n",
"\n",
"# Use OpenAI API to match the query text with the titles and abstracts of the papers and generate relevance score\n",
"relevant_citations = []\n",
"for index, row in df.iterrows():\n",
" title = row['Title']\n",
" abstract = row['Abstract']\n",
" year = row['Year']\n",
" journal = row['Journal']\n",
" doi = row['DOI']\n",
" if not abstract:\n",
" continue\n",
" # Combine the title and abstract\n",
" text = f\"{title}.{abstract}\"\n",
" # Use OpenAI API to generate relevance score for the text and query\n",
" response = openai.Completion.create(\n",
" engine=\"text-davinci-003\",\n",
" prompt=f\"Match the following query with the text and return the relevance score:\\n\\nQuery: {query_text} [({title}, {year}, {journal}, {doi})]\\n\\nText: {text}\\n\\nRelevance Score:\",\n",
" max_tokens=1,\n",
" n=1,\n",
" stop=None,\n",
" temperature=0.5,\n",
" )\n",
" relevance_score_str = response.choices[0].text.strip()\n",
" if relevance_score_str:\n",
" relevance_score = float(relevance_score_str)\n",
" # Add the title, relevance score, and DOI to the relevant citations list\n",
" relevant_citations.append((title, relevance_score, year, journal, doi))\n",
"\n",
"# Sort the relevant citations by relevance score in descending order\n",
"relevant_citations.sort(key=lambda x: x[1], reverse=True)\n",
"#print(relevant_citations)\n",
"\n",
"# Generate bibliography for the most relevant citations\n",
"bibliography = \"\"\n",
"for citation in relevant_citations[:5]:\n",
" title = citation[0]\n",
" year = citation[2]\n",
" journal = citation[3]\n",
" doi = citation[4]\n",
" bibliography += f\"{title}, {year}, {journal}, {doi}\\n\"\n",
"\n",
"# Print the original query and the most relevant citations in APA style\n",
"print(\"Query: \", query_text)\n",
"print(\"Citations:\")\n",
"# Create a DataFrame from relevant_citations\n",
"df1 = pd.DataFrame(relevant_citations, columns=['Title', 'Relevance Score', 'Year', 'Journal', 'DOI'])\n",
"\n",
"# Sort the DataFrame by relevance score in descending order\n",
"df1 = df1.sort_values(by='Year', ascending=False)\n",
"# Print the DataFrame in a tabular format\n",
"#print(df1.to_markdown(index=False))\n",
"print(\"Execution complete !\")"
],
"metadata": {
"id": "kf2gp_39ILmR",
"outputId": "c0948141-dd9b-4b06-b425-29df7f38ea89",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Query: Liquidity, unlike other trade analysis indicators, has no fixed value. As a result, calculating the exact liquidity of the exchange or market is difficult. However, there are other signs that can be used as proxies for liquidity in cryptocurrencies.\n",
"Citations:\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"print(\"******************************** Display Results ********************************\")\n",
"print(\"Original Text:\")\n",
"display(query_text)\n",
"print(\"\")\n",
"print(\"Keywords:\")\n",
"print(keywords)\n",
"print(\"\")\n",
"print(\"Citations for the given text based on title and keywords using Semantic Search:\")\n",
"print(\"*********************************************************************************************************************\")\n",
"print(df[['Title', 'Author(s)', 'Year', 'Journal', 'DOI']].head().to_markdown(index=False))\n",
"print(\"\")\n",
"print(\"*********************************************************************************************************************\")\n",
"print(\"\")\n",
"print(\"\")\n",
"print(\"*****Most relevant citations based on AI search******:\")\n",
"if len(relevant_citations) == 0:\n",
" print(\"No relevant citations found.\")\n",
"else:\n",
" print(df1[['Title', 'Year', 'Journal', 'DOI']].to_markdown(index=False))\n",
"print(\"\")"
],
"metadata": {
"id": "eIiVUXhxI1m4",
"outputId": "b2b0284d-6630-4bd1-a68f-e93724f9922b",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 489
}
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"******************************** Display Results ********************************\n",
"Original Text:\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"'Liquidity, unlike other trade analysis indicators, has no fixed value. As a result, calculating the exact liquidity of the exchange or market is difficult. However, there are other signs that can be used as proxies for liquidity in cryptocurrencies.'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
"Keywords:\n",
"['Liquidity, Trade Analysis, Indicators, Fixed Value, Calculating, Exchange, Market, Cryptocurrencies, Proxies.']\n",
"\n",
"Citations for the given text based on title and keywords using Semantic Search:\n",
"*********************************************************************************************************************\n",
"| Title | Author(s) | Year | Journal | DOI |\n",
"|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------|-------:|:------------------------------------------------|:------------------------------------------------|\n",
"| ['How much liquidity would a liquidity-saving mechanism save if a liquidity-saving mechanism could save liquidity? A simulation approach for Canada’s large-value payment system'] | Ronald Heijmans, Shaun Byck | 2020 | The Journal of Financial Market Infrastructures | https://doi.org/10.21314/jfmi.2019.120 |\n",
"| ['Robust and Interpretable Liquidity Proxies for Market and Funding Liquidity'] | Riccardo Rebonato, Hong Sherwin | 2020 | The Journal of Fixed Income | https://doi.org/10.3905/jfi.2020.1.103 |\n",
"| ['Figure 1.15. Austria has the lowest stock market capitalisation in the OECD and the market lacks liquidity'] | | 2019 | | https://doi.org/10.1787/888934025917 |\n",
"| ['Trade Intensity and Liquidity'] | Matt Brigida, William Pratt | 2019 | Market Microstructure and Liquidity | https://doi.org/10.1142/s2382626619500023 |\n",
"| ['Exchange Rates, Nominal Bonds, and Open Market Operations'] | | 2018 | Money, Payments, and Liquidity | https://doi.org/10.7551/mitpress/10518.003.0015 |\n",
"\n",
"*********************************************************************************************************************\n",
"\n",
"\n",
"*****Most relevant citations based on AI search******:\n",
"| Title | Year | Journal | DOI |\n",
"|:----------------------------------|-------:|:------------------------------------|:------------------------------------------|\n",
"| ['Trade Intensity and Liquidity'] | 2019 | Market Microstructure and Liquidity | https://doi.org/10.1142/s2382626619500023 |\n",
"\n"
]
}
]
}
]
}
GitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Bhaskar Tripathi | 3****i | 9 |
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0