https://github.com/queryverse/parquetfiles.jl
FileIO.jl integration for Parquet files
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
1 of 6 committers (16.7%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.3%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
FileIO.jl integration for Parquet files
Basic Info
- Host: GitHub
- Owner: queryverse
- License: other
- Language: Julia
- Default Branch: master
- Size: 147 KB
Statistics
- Stars: 19
- Watchers: 2
- Forks: 10
- Open Issues: 13
- Releases: 0
Topics
Metadata Files
README.md
ParquetFiles
Overview
This package provides load support for Parquet files under the FileIO.jl package.
Installation
Use ] add ParquetFiles in Julia to install ParquetFiles and its dependencies.
Usage
Load a Parquet file
To read a Parquet file into a DataFrame, use the following julia code:
````julia using ParquetFiles, DataFrames
df = DataFrame(load("data.parquet")) ````
The call to load returns a struct that is an IterableTable.jl, so it can be passed to any function that can handle iterable tables, i.e. all the sinks in IterableTable.jl. Here are some examples of materializing a Parquet file into data structures that are not a DataFrame:
````julia using ParquetFiles, IndexedTables, TimeSeries, Temporal, VegaLite
Load into an IndexedTable
it = IndexedTable(load("data.parquet"))
Load into a TimeArray
ta = TimeArray(load("data.parquet"))
Load into a TS
ts = TS(load("data.parquet"))
Plot directly with Gadfly
@vlplot(:point, data=load("data.parquet"), x=:a, y=:b) ````
Using the pipe syntax
load also support the pipe syntax. For example, to load a Parquet file into a DataFrame, one can use the following code:
````julia using ParquetFiles, DataFrame
df = load("data.parquet") |> DataFrame ````
The pipe syntax is especially useful when combining it with Query.jl queries, for example one can easily load a Parquet file, pipe it into a query, then pipe it to the save function to store the results in a new file.
Owner
- Name: Queryverse
- Login: queryverse
- Kind: organization
- Website: http://www.queryverse.org/
- Repositories: 37
- Profile: https://github.com/queryverse
GitHub Events
Total
Last Year
Committers
Last synced: over 3 years ago
All Time
- Total Commits: 48
- Total Committers: 6
- Avg Commits per committer: 8.0
- Development Distribution Score (DDS): 0.292
Top Committers
| Name | Commits | |
|---|---|---|
| David Anthoff | a****f@b****u | 34 |
| Julia Package Butler | 8 | |
| github-actions[bot] | 4****]@u****m | 2 |
| evalparse | z****i@g****m | 2 |
| [Yijia Chen] | y****n@g****m | 1 |
| scls19fr | s****r@u****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 16
- Total pull requests: 24
- Average time to close issues: 5 months
- Average time to close pull requests: 20 days
- Total issue authors: 10
- Total pull request authors: 8
- Average comments per issue: 0.75
- Average comments per pull request: 0.58
- Merged pull requests: 15
- Bot issues: 0
- Bot pull requests: 15
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- xiaodaigh (5)
- davidanthoff (3)
- Gyslain (1)
- tpoisot (1)
- xianwenchen (1)
- blairn (1)
- tclements (1)
- Deduction42 (1)
- calebwin (1)
- MichaelDS (1)
Pull Request Authors
- github-actions[bot] (15)
- davidanthoff (3)
- ruslanasa (1)
- xiaodaigh (1)
- tanmaykm (1)
- Yijia-Chen (1)
- morris25 (1)
- scls19fr (1)