NeuroScape: Mapping the Neuroscience Research Landscape

Codebase accompanying the article:

Senden, M. (n.d.). AI-Driven Analysis of 461,316 Articles (1999-2023) Reveals Neuroscience’s Shifting Research Landscape. Manuscript submitted.

BioRxiv preprint

This repository contains scripts and notebooks for analyzing and visualizing neuroscience research through large-scale data collection, filtering, clustering, and semantic analysis of scientific articles.

Abstract

Neuroscience emerged as a distinct academic discipline during the 20th century and has undergone rapid expansion since then. To retain an overview of the structural organization of contemporary neuroscience, this study leverages text-embedding and clustering techniques together with large language models for analyzing 461,316 articles published between 1999 and 2023. The field exhibits a strong experimental focus and a growing emphasis on applied research while fundamental research is at the risk of decline. Furthermore, neuroscience currently lacks theory-centric clusters. Inter-cluster citation analysis uncovers a surprisingly high level of cross-fertilization and key intellectual hubs that shape the broader landscape. Yet, cross-scale integration remains limited. This study provides a framework for understanding neuroscience’s trajectory and identifies potential avenues for strengthening the field. Publicly available data and code provide a reusable resource for researchers, funders, and policy analysts.

Repository Structure

. ├── config/ # TOML config files (scraping, clustering, analysis, etc.) ├── notebooks/ # Jupyter notebooks for exploration and results ├── scripts/ # Main processing scripts organized by function │ ├── ingestion/ # Data collection and cleaning │ ├── preprocessing/ # Filtering and classification │ ├── domain_embedding/ # Training & applying domain-specific embeddings │ ├── clustering/ # Building semantic graphs & community detection │ ├── graph_analysis/ # Citation/network density analysis │ └── semantic_analysis/ # Dimension analysis, cluster characterization, trends, etc. ├── src/ │ ├── classes/ # Python classes for data structures and model architectures │ └── utils/ # Utility modules (parsing, data loading, plotting, etc.) └── README.md

Data

This repository has been used to collect, curate, and analyze the following dataset:

Senden, M. (2025). NeuroScape (1.0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.14865161

Workflow Overview

1. Scrape Data

   • scripts/ingestion/scraping.py
     
   • Query PubMed for relevant articles. Neuroscience, multidisciplinary and other (for training discipline classifier).

2. Merge and Clean

   • scripts/ingestion/mergeandclean.py
     
   • Consolidate scraped data, remove duplicates, and clean metadata.

3. Initial Embedding

   • scripts/ingestion/initial_embedding.py
     
   • Generate general-purpose text embeddings (via Voyage AI) for each abstract.

4. Prepare Classifier Training Data

   • scripts/preprocessing/prepareclassifiertraining_data.py
     
   • Create labeled samples for discipline classification (to distinguish neuroscience from other fields).

5. Train Discipline Classifier

   • scripts/preprocessing/traindisciplineclassifier.py
     
   • Train a neural network to identify neuroscientific articles.

6. Filter Data

   • scripts/preprocessing/filter_disciplines.py
     
   • Retain only articles classified as neuroscience with high confidence.

7. Build Adjacency Matrix

   • scripts/ingestion/build_adjacencies.py
     
   • Obtains citing and cited articles for each article in the dataset.

8. Train Domain Embedding Model

   • scripts/domainembedding/trainembedding_model.py
   • Trains a domain-specific embedding model on top of the initial embeddings (through contrastive learning).

9. Domain Embedding

   • scripts/domainembedding/embedabstracts.py
   • Re-embeds abstracts in a lower-dimensional, neuroscience-focused space for semantic clustering.

10. Build the Semantic Graph

    • scripts/clustering/graph_construction.py
    • Uses the domain-specific embeddings to construct a similarity graph (e.g., KNN) needed for community detection.

11. Community Detection

    • scripts/clustering/community_detection.py
      
    • Perform clustering (e.g., Leiden community detection) on the network.

12. Cluster Definition

    • scripts/semanticanalysis/clusterdefinition.py
      
    • Generate descriptive titles, keywords, and descriptions for each cluster.

13. Cluster Distinction

    • scripts/semanticanalysis/clusterdistinction.py
      
    • Identify key differences between similar clusters.

14. Dimensions Extraction

    • scripts/semanticanalysis/assessdimensions.py
      
    • Analyze each cluster across multiple research dimensions (e.g., appliedness, modality).

15. Dimension Categorization

    • scripts/semanticanalysis/assessdimension_categories.py
      
    • Categorize clusters along specific sub-dimensions (e.g., fundamental vs clinical research within appliedness).

16. Open Questions

    • scripts/semanticanalysis/extractopen_questions.py
      
    • Identify important open research questions from recent review articles.

17. Trends Extraction

    • scripts/semanticanalysis/extracttrends.py
      
    • Compare older vs. recent publications to reveal emerging and declining trends.

18. Density Graph

    • scripts/graphanalysis/clusterdensity.py
      
    • Assess citation density and connections between clusters.

Optional - scripts/preprocessing/update_embedding.py allows updating the general embeddings with a newer Voyage AI model if desired.

Notebooks

Notebooks handle creation of Figures and Tables for the manuscript, including additional analyses.

Setup Instructions

1. Create a Conda Environment

First, create and activate a Conda environment with Python 3.12: bash conda create --name neuroscape_env python==3.12 conda activate neuroscape_env

2. Install PyTorch

Install PyTorch before installing other dependencies. Follow the official instructions based on your system: - Visit: PyTorch Installation Guide

3. Install Other Dependencies

Once PyTorch is installed, install the remaining dependencies from requirements.txt: bash pip install -r requirements.txt

Disclaimer

This repository provides the scripts and workflow used in the study and is intended for research and educational purposes. While I encourage other researchers to use and build upon this work, I am primarily a researcher, not a full-time software developer. As such:

• I welcome issues and pull requests and will try to address them as time permits.
• However, active maintenance is not guaranteed. Users should not expect frequent updates or extensive support.
• The code is provided as is, without warranties regarding performance, correctness, or long-term compatibility.

If you use this repository for your research, I would appreciate a citation to the accompanying manuscript.