https://github.com/alan-turing-institute/ati_tm

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
○
.zenodo.json file
○
DOI references
○
Academic publication links
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (9.8%) to scientific vocabulary

Keywords

hut23 hut23-592

Last synced: 9 months ago · JSON representation

Repository

Basic Info

Host: GitHub
Owner: alan-turing-institute
License: mit
Language: Jupyter Notebook
Default Branch: master
Size: 30.5 MB

Statistics

Stars: 2
Watchers: 3
Forks: 1
Open Issues: 0
Releases: 0

Topics

hut23 hut23-592

Created almost 9 years ago · Last pushed almost 8 years ago

Metadata Files

Readme

Topic modeling (Latent Dirichlet Allocation) of Turing Institute publications

Aim:

Identify topics researched at the Alan Turing Institute

Overview:

For more details see the blog post.

In summary, for each Turing fellow we downloaded up to 50 articles they published since 2012 (inclusive) using open access routes. The downloaded articles were converted to text files and cleaned. We analysed the text data using Latent Dirichlet Allocation (LDA) which identified 25 separate research topics in the dataset. The result is a visualisation of each Turing fellow and the extent to which their work falls under any of the identified research topics.

Procedure:

1: Identifying Turing fellows' publications

We used Microsoft's Academic Knowledge (AK) database to extract publications records for each Turing fellow. We first manually extracted a list of unique AK IDs for all fellows to get around entity matching issues to ensure we are targeting the right researchers in the database. We then used these IDs to retrieve a list of article titles, DOIs and source URLs.

Method: 1generatepublications_list.ipynb
Results: publications_eng.csv

2: Publications download

(i) We built a web scraper to download each of the articles identified in step 1 (relying on open access routes)
Method: 2aarticlescraper.ipynb

(ii) The downloaded PDFs were converted to .txt files (using Contentmine's NORMA tool)
see 2bpdfto_txt.md for description

3: Data pre-processing

All data was cleaned and combined into final analysis-ready csv file
The final csv consists of 1 article per row and contains: fellow name, ID and affiliation, article title and ID, article full text and length, URLs to versions of the article available online and AK provided keywords associated with the article.
Method: 3datapreprocessing.ipynb
Data in 2 parts: finaldataset1.csv and finaldataset2.csv

4: Analysis

The converted texts were analysed using LDA
Method: 4_LDA.ipynb
Results (topic proportions by researcher, used for visualisation purposes): data_original.csv

5: Visualisation

A visualisation of the final topics was built using an adaptation of the aster plot (see visualisation folder)

Owner

Name: The Alan Turing Institute
Login: alan-turing-institute
Kind: organization
Email: info@turing.ac.uk

Website: https://turing.ac.uk
Repositories: 477
Profile: https://github.com/alan-turing-institute

The UK's national institute for data science and artificial intelligence.

GitHub Events

Total

Last Year

Issues and Pull Requests

Last synced: 12 months ago

All Time

Total issues: 0
Total pull requests: 1
Average time to close issues: N/A
Average time to close pull requests: about 1 hour
Total issue authors: 0
Total pull request authors: 1
Average comments per issue: 0
Average comments per pull request: 0.0
Merged pull requests: 1
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science