Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
1 of 2 committers (50.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (12.9%) to scientific vocabulary
Repository
R-Package for parsing Excel files
Basic Info
- Host: GitHub
- Owner: fhenz
- License: other
- Language: C++
- Default Branch: master
- Size: 332 KB
Statistics
- Stars: 38
- Watchers: 1
- Forks: 2
- Open Issues: 1
- Releases: 0
Metadata Files
README.md
SheetReader
SheetReader provides functionality to read tabular data from Excel OOXML (.xlsx) files.
This repository integrates SheetReader into a R package, with Rcpp serving as the interface for the parsing code written in C++.
SheetReader is available via CRAN:
r
install.packages("SheetReader")
Overview
SheetReader uses incremental decompression with buffer recycling to keep memory usage minimal, while employing multithreading where possible to speed up the parsing process.
While there are existing packages for parsing Excel files into R (notably readxl and openxlsx), I was not satisfied with their performance when processing large files.
readxl uses the RapidXML library to completely parse the Excel documents into full XML/DOM trees, which can get prohibitively expensive (dozens of gigabytes for files upwards of 200,000 rows by 100 columns).
openxlsx is more efficient in terms of memory usage but generally slower than readxl.
Benchmarks on files of roughly these sizes (200,000 rows by 100 columns) indicate SheetReader to be about 3x faster with 20x less memory usage than readxl, and 15x faster with 10x less memory than openxlsx.
Be aware that these are relatively old benchmarking results (early 2021) and obviously depend on the file being parsed (SheetReader takes advantage of some optional, but usually present, features of the OOXML format to improve performance).
Having said that, this package is very bare-bones and if you require anything other than parsing tabular data (e.g. retrieving sheet names, writing Excel files) then you should have a look at the other mentioned packages.
Additionally, the transformation to R dataframe currently assumes homogenous columns.
If cell types in a column don't match the first non-blank cell, they are returned as NA.
SheetReader Paper
SheetReader was presented at DOLAP 2022 (colocated with EDBT). The paper is open access and can be found here.
If you use SheetReader in your academic project, please cite it with the following bibtex entry:
@inproceedings{DBLP:conf/dolap/HenzeGZM22,
author = {Felix Henze and
Haralampos Gavriilidis and
Eleni Tzirita Zacharatou and
Volker Markl},
editor = {Kostas Stefanidis and
Lukasz Golab},
title = {Efficient Specialized Spreadsheet Parsing for Data Science},
booktitle = {Proceedings of the 24th International Workshop on Design, Optimization,
Languages and Analytical Processing of Big Data {(DOLAP)} co-located
with the 25th International Conference on Extending Database Technology
and the 25th International Conference on Database Theory {(EDBT/ICDT}
2022), Edinburgh, UK, March 29, 2022},
series = {{CEUR} Workshop Proceedings},
volume = {3130},
pages = {41--50},
publisher = {CEUR-WS.org},
year = {2022},
url = {http://ceur-ws.org/Vol-3130/paper5.pdf},
timestamp = {Thu, 23 Jun 2022 19:58:25 +0200},
biburl = {https://dblp.org/rec/conf/dolap/HenzeGZM22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
Acknowledgements
SheetReader includes and uses the following C/C++ libraries:
- miniz for ZIP archive operations and decompression
- libdeflate for optimized full-buffer decompression
- fastdoubleparser for optimized number parsing
GitHub Events
Total
- Issues event: 2
- Watch event: 3
- Fork event: 1
Last Year
- Issues event: 2
- Watch event: 3
- Fork event: 1
Committers
Last synced: over 2 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| fhenz | 1****z | 14 |
| harry | g****s@t****e | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: over 2 years ago
All Time
- Total issues: 2
- Total pull requests: 1
- Average time to close issues: less than a minute
- Average time to close pull requests: 1 day
- Total issue authors: 2
- Total pull request authors: 1
- Average comments per issue: 1.5
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 2
- Pull requests: 1
- Average time to close issues: less than a minute
- Average time to close pull requests: 1 day
- Issue authors: 2
- Pull request authors: 1
- Average comments per issue: 1.5
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- zergar (2)
- freddie-freeloader (1)
- coforfe (1)
- zoushucai (1)
Pull Request Authors
- harrygav (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- cran 301 last-month
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 5
- Total maintainers: 1
cran.r-project.org: SheetReader
Parse xlsx Files
- Homepage: https://github.com/fhenz/SheetReader-r
- Documentation: http://cran.r-project.org/web/packages/SheetReader/SheetReader.pdf
- License: MIT + file LICENSE
-
Latest release: 1.2.1
published about 2 years ago
Rankings
Maintainers (1)
Dependencies
- Rcpp >= 1.0.5 imports