science-portal-seeding
For seeding the Science Portal MongoDB from annual review
Science Score: 31.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.8%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
·
Repository
For seeding the Science Portal MongoDB from annual review
Basic Info
- Host: GitHub
- Owner: mattbocc
- Language: Python
- Default Branch: main
- Size: 34.2 KB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Created almost 3 years ago
· Last pushed about 2 years ago
Metadata Files
Readme
Citation
README.md
science-portal-seeding
This script will parse through the UHN publication excel spreadsheet and extract publications with article types: research-articles, journal articles, and articles. The script then compiles this data into a JSON with required fields that is directly added to a mongo collection of choice defined in your environment file.
pip dependencies needed are python-dotenv, pandas, pymongo, and openpyxl(dependency for another package):
pip install python-dotenv pandas pymongo openpyxl
Owner
- Login: mattbocc
- Kind: user
- Location: Toronto, Ontario
- Repositories: 1
- Profile: https://github.com/mattbocc
Citation (citations.py)
import os
from dotenv import load_dotenv
from scholarly import scholarly
import certifi
import pymongo
import random
import time
import json
load_dotenv()
# MongoDB connection parameters
client = pymongo.MongoClient(os.getenv("MONGO_URI"), tlsCAFile=certifi.where())
db_name = client[os.getenv("DB_NAME")]
collection = db_name[os.getenv("COLLECTION_NAME")]
no_scholar_list = []
# Iterate through each document in the collection
for document in collection.find():
name = document.get('name', '')
doi = document.get('doi', '')
try:
# Search for the publication on Google Scholar by doi
search_publication = scholarly.search_single_pub(doi)
if search_publication is not None:
# Get the number of citations
num_citations = int(search_publication.get('num_citations', 0))
if num_citations != 0:
# Update the 'citations' field in the document
collection.update_one(
{"_id": document["_id"]},
{"$set": {"citations": num_citations}}
)
print(f'{name} entered {num_citations}')
else:
print(f"No citations found for publication '{name}'.")
else:
print(f"Publication '{name}' not found on Google Scholar.")
except:
no_scholar_list.append(name)
print(f'Cannot get citations for {name} publication')
time.sleep(random.randint(8, 20))
client.close()
# Write no_scholar_list to a JSON file
with open('no_scholar_list.json', 'w') as json_file:
json.dump(no_scholar_list, json_file)