extract-urls
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
6 of 9 committers (66.7%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.2%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Basic Info
- Host: GitHub
- Owner: oduwsdl
- Default Branch: main
- Size: 1.84 GB
Statistics
- Stars: 6
- Watchers: 2
- Forks: 7
- Open Issues: 1
- Releases: 0
Created over 4 years ago
· Last pushed almost 3 years ago
Metadata Files
Readme
Citation
README.md
Extract-URLs
This project is currently set up to extract URLs from PDf publications in the arXiv corpus and run numerous data analysis programs to understand the use of URLs in the arXiv corpus.
Order of Programs
Some of the programs/scripts require the files from other programs/scripts. The correct order is shown below:
- extract_urls.py (creates a JSON file of extracted URLs for each directory)
- getrepourls.py (creates repourls.json (only files with a repo URL) and allfile_urls.json (all files))
- dataprocessing/reporepeats.sh (outputs the frequency of each URL for each repo)
- dataprocessing/dirurls.py (aggregates the number of URLs per month per repo and creates dirurls.csv and filecount.csv)
- dataprocessing/fileurlcounts.py (creates fileurlcounts.csv, ecdfnumbers.csv, and ccdf_numbers.csv)
- dataprocessing/urlfreq.sh (creates URL_frequency.csv)
- dataprocessing/multiplerepos.sh (finds files that contain URLs to more than 1 repo and creates multiple_repos.csv)
- dataprocessing/multiplerepos_table.py (creates a table of the file and which repos it links to)
- dataprocessing/pdfstats.py (calculates mean, median, and quartiles for the number of repo URLs in a PDF and creates pdf_stats.csv)
- dataprocessing/repourlcounts.py (creates repourl_counts.csv which contains the URL count for each directory and repo)
- dataprocessing/urlsperfile.py (creates urlsper_file.csv which contains the average number of URLs per mopnth for each repo)
Owner
- Name: ODU WS-DL
- Login: oduwsdl
- Kind: organization
- Location: Norfolk, Virginia, USA
- Website: https://oduwsdl.github.io/
- Twitter: WebSciDL
- Repositories: 54
- Profile: https://github.com/oduwsdl
Web Science and Digital Libraries Research Group at Old Dominion University
GitHub Events
Total
Last Year
Committers
Last synced: over 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Emily Vogt | e****t@l****u | 212 |
| Emily Escamilla | v****m@g****m | 35 |
| cs_evogt001 | c****1@t****u | 11 |
| cs_evogt001 | c****1@g****u | 3 |
| cs_evogt001 | c****1@c****u | 3 |
| mara004 | 6****4 | 2 |
| Stephan Druskat | s****t | 1 |
| cs_evogt001 | c****1@w****u | 1 |
| cs_evogt001 | c****1@e****u | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 1
- Total pull requests: 3
- Average time to close issues: N/A
- Average time to close pull requests: 2 months
- Total issue authors: 1
- Total pull request authors: 2
- Average comments per issue: 1.0
- Average comments per pull request: 0.33
- Merged pull requests: 2
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- sdruskat (1)
Pull Request Authors
- mara004 (2)
- sdruskat (1)