Projects

Updated 9 months ago

trafilatura • Rank 26.3 • Science 77%

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

article-extractor corpus-builder corpus-tools crawler html-to-markdown html2text llm news-aggregator news-crawler nlp rag readability rss-feed scraping tei text-cleaning text-extraction text-mining text-preprocessing web-scraping

Updated 9 months ago

dhscrapers • Rank 3.2 • Science 44%

A unified interface for scrapers for Digital Humanities resources

digtial-humanities python scraping

Updated 9 months ago

https://github.com/citiususc/pyplexity • Rank 16.8 • Science 13%

Cleaning tool for web scraped text

information-retrieval nlp python scraping tag-cleaning

Updated 9 months ago

Iceberg-locations • Rank 2.5 • Science 26%

Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

beautifulsoup4 climate-change git-scraping iceberg python scraping

Updated 9 months ago

worldometer • Rank 10.1 • Science 13%

Get live, population, geography, projected, and historical data from around the world 🌍

api data historical historical-data live livedata metrics mit-license projected pypi python scraping world worldometer worldometer-api worldometer-scraping worldometers

Updated 9 months ago

https://github.com/claromes/volleystats • Rank 2.9 • Science 13%

🏐 Command-line tool to scrape volleyball statistics from Data Project Web Competition websites

data-project data-volley python scraping scrapy sports-data volleyball

Updated 9 months ago

foot • Rank 0.0 • Science 13%

Foot is a library that fetches a list of URLs and silly walks through each site to gather information.

bugbounty crawler scraping

Updated 9 months ago

semantic-outlier-removal • Science 54%

Code and data for SORE (ACL 2025), a semantic boilerplate remover.

article-extractor crawler embedding html-to-text html2text llm nlp outlier-removal preprocessing readability scraping text-extraction text-mining web-scraping

Updated 9 months ago

resultatselection • Science 44%

Scraped data from https://resultatselection.belgium.be

belgium dataset election opendata politics python scraping selenium selenium-webdriver

Updated 9 months ago

https://github.com/cumbof/galaxy-scraper-extension • Science 26%

Chrome extension that automatically scrape and send data from a webpage to the current history of a Galaxy instance.

chrome-extension galaxy-project scraping

Updated 9 months ago

rat-software • Science 67%

Streamline your search engine research. With the Result Assessment Tool (RAT) you can easily collect results from different search engines, let participants evaluate the results and analyse your findings.