Data

Tools

Indexes

Applications

Experiments

Open Source Science

dblp

A scraper that downloads search results from DBLP.

https://github.com/sebastiangehrmann/dblp-pub

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: acm.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.7%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

A scraper that downloads search results from DBLP.

Basic Info
  • Host: GitHub
  • Owner: sebastianGehrmann
  • License: mit
  • Language: Python
  • Default Branch: master
  • Size: 10.7 KB
Statistics
  • Stars: 12
  • Watchers: 1
  • Forks: 5
  • Open Issues: 1
  • Releases: 0
Created almost 11 years ago · Last pushed about 7 years ago
Metadata Files
Readme License

README.md

dblp publication scraper

A scraper that downloads search results from DBLP. You can search for any list of keywords.

Scraping dblp is allowed by them and they even offer an API for it. The API, however, is restricted to the search for authors. This is why I wrote this scraper, which allows you to also search for publications.

Installation

Just type "pip install dblp" in your terminal and it will get installed. You can also clone this repository and type "python setup.py install" after switching to the downloaded folder.

How to use it?

After importing it, you can use the search()-function to search in dblp for any array of key words. The result will be a pandas DataFrame.

Example

Imagine you are interested research in Collaborative Writing and want to evaluate which conferences people who write about it publish to. With this library you get a table for every search request, so you can easily compare results. Below is an example for one of these tables and the lookup for the search phrase "Collaborative Writing".

```python import dblp

results = dblp.search(["Collaborative Writing"])

results.head() ``` The Code above yields the following results:

|Authors | Link | Title |Type | Where |Year | |------------------------|:--------------------:|:--------------------------------:|:-----------:|:--------:|----:| | [Lekha Limbu, Lina M...| http://dx.doi.org/...|How do learners experience join...|article |Compute...|2015 | | [Sebastian Gehrmann,...| http://dx.acm.org/...|Deploying AI Methods to Support...|inproceedings|CHI Ext...|2015 | | [Dakuo Wang, Judith ...| http://dx.acm.org/...|DocuViz: Visualizing Collaborat...|inproceedings|CHI |2015 | | [Oluwabunmi Adewoyin...| http://dx.doi.org/...|Exploiting the Use of Wikis to ...|inproceedings|CRIWG |2015 | | [Menghui Li, Young M...| http://dx.doi.org/...|An Exploration of Mobile Collab...|inproceedings|HCI (24...|2015 |

Now imagine, you want to look for the papers with "AI" in the title. You can easily do that using pandas:

python results[results['Title'].str.contains("AI")]

Given the search query above, this yield one paper with the following link: http://doi.acm.org/10.1145/2702613.2732705

Licence

This code is published under the MIT licence. Feel free to improve and redistribute my code. I hope I made your life a little bit easier.

Owner

  • Name: Sebastian Gehrmann
  • Login: sebastianGehrmann
  • Kind: user
  • Location: New York City
  • Company: Google

Head of NLP in the CTO office at Bloomberg. Formerly researcher at Google.

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Committers

Last synced: almost 3 years ago

All Time
  • Total Commits: 18
  • Total Committers: 2
  • Avg Commits per committer: 9.0
  • Development Distribution Score (DDS): 0.056
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Sebastian Gehrmann s****n@o****m 17
Martin Thoma i****o@m****e 1
Committer Domains (Top 20 + Academic)
martin-thoma.de: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 1
  • Total pull requests: 1
  • Average time to close issues: N/A
  • Average time to close pull requests: 1 minute
  • Total issue authors: 1
  • Total pull request authors: 1
  • Average comments per issue: 1.0
  • Average comments per pull request: 3.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • jinfenglin (1)
Pull Request Authors
  • MartinThoma (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 38 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 2
  • Total versions: 1
  • Total maintainers: 1
pypi.org: dblp

Downloads and formats search results from dblp

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 2
  • Downloads: 38 Last month
Rankings
Dependent packages count: 10.0%
Dependent repos count: 11.6%
Forks count: 14.2%
Stargazers count: 16.5%
Average: 18.2%
Downloads: 38.9%
Maintainers (1)
gehrmann
Last synced: 11 months ago

Dependencies

requirements.txt pypi
  • beautifulsoup4 >=4.3.2
  • pandas >=0.16.2
  • requests >=2.7.0
setup.py pypi
  • beautifulsoup4 >=4.3.2
  • pandas >=0.16.2
  • requests >=2.7.0