Science Score: 77.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 4 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
✓Committers with academic emails
7 of 56 committers (12.5%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.2%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
Manipulate JSON-like data with NumPy-like idioms.
Basic Info
- Host: GitHub
- Owner: scikit-hep
- License: bsd-3-clause
- Language: Python
- Default Branch: main
- Homepage: https://awkward-array.org
- Size: 26.4 MB
Statistics
- Stars: 900
- Watchers: 20
- Forks: 95
- Open Issues: 149
- Releases: 270
Topics
Metadata Files
README.md
<!-- end-logo -->
Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms.
Arrays are dynamically typed, but operations on them are compiled and fast. Their behavior coincides with NumPy when array dimensions are regular and generalizes when they're not.
Motivating example
Given an array of lists of objects with x, y fields (with nested lists in the y field),
```python import awkward as ak
array = ak.Array([ [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}], [], [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}] ]) ```
the following slices out the y values, drops the first element from each inner list, and runs NumPy's np.square function on everything that is left:
python
output = np.square(array["y", ..., 1:])
The result is
python
[
[[], [4], [4, 9]],
[],
[[4, 9, 16], [4, 9, 16, 25]]
]
The equivalent using only Python is
python
output = []
for sublist in array:
tmp1 = []
for record in sublist:
tmp2 = []
for number in record["y"][1:]:
tmp2.append(np.square(number))
tmp1.append(tmp2)
output.append(tmp1)
The expression using Awkward Arrays is more concise, using idioms familiar from NumPy, and it also has NumPy-like performance. For a similar problem 10 million times larger than the one above (single-threaded on a 2.2 GHz processor),
- the Awkward Array one-liner takes 1.5 seconds to run and uses 2.1 GB of memory,
- the equivalent using Python lists and dicts takes 140 seconds to run and uses 22 GB of memory.
Awkward Array is even faster when used in Numba's JIT-compiled functions.
See the Getting started documentation on awkward-array.org for an introduction, including a no-install demo you can try in your web browser.
Getting help
- View the documentation on awkward-array.org.
- Report bugs, request features, and ask for additional documentation on GitHub Issues.
- If you have a "How do I...?" question, start a GitHub Discussion with category "Q&A".
- Alternatively, ask about it on StackOverflow with the [awkward-array] tag. Be sure to include tags for any other libraries that you use, such as Pandas or PyTorch.
- To ask questions in real time, try the Gitter Scikit-HEP/awkward-array chat room.
Installation
Awkward Array can be installed from PyPI using pip:
bash
pip install awkward
The awkward package is pure Python, and it will download the awkward-cpp compiled components as a dependency. If there is no awkward-cpp binary package (wheel) for your platform and Python version, pip will attempt to compile it from source (which has additional dependencies, such as a C++ compiler).
Awkward Array is also available on conda-forge:
bash
conda install -c conda-forge awkward
Because of the two packages (awkward-cpp may be updated in GitHub but not on PyPI), pip install through git (pip install git+https://...) will not work. Instead, use the Installation for developers section below.
Installation for developers
Clone this repository recursively to get the header-only C++ dependencies, then generate sources with nox, compile and install awkward-cpp, and finally install awkward as an editable installation:
```bash git clone --recursive https://github.com/scikit-hep/awkward.git cd awkward
nox -s prepare python -m pip install -v ./awkward-cpp python -m pip install -e . ```
Tests can be run in parallel with pytest:
bash
python -m pytest -n auto tests
For more details, see CONTRIBUTING.md, or one of the links below.
- Continuous integration and continuous deployment are hosted by GitHub Actions.
- Code of conduct for how we work together.
- The LICENSE is BSD-3.
Documentation, Release notes, Roadmap, Citations
The documentation is on awkward-array.org, including
- Getting started
- User guide
- API reference
- Tutorials (with videos)
- Papers and talks about Awkward Array
The Release notes for each version are in the GitHub Releases tab.
The Roadmap, Plans, and Deprecation Schedule are in the GitHub Wiki.
To cite Awkward Array in a paper, see the "Cite this repository" drop-down menu on the top-right of the GitHub front page. The BibTeX is
bibtex
@software{Pivarski_Awkward_Array_2018,
author = {Pivarski, Jim and Osborne, Ianna and Ifrim, Ioana and Schreiner, Henry and Hollands, Angus and Biswas, Anish and Das, Pratyush and Roy Choudhury, Santam and Smith, Nicholas and Goyal, Manasvi and Fackeldey, Peter and Krommydas, Iason},
doi = {10.5281/zenodo.4341376},
month = {10},
title = {{Awkward Array}},
year = {2018}
}
Acknowledgements
Support for this work was provided by NSF cooperative agreement OAC-1836650 (IRIS-HEP 1), PHY-2323298 (IRIS-HEP 2), grant OAC-1450377 (DIANA/HEP), PHY-1520942 and PHY-2121686 (US-CMS LHC Ops), and OAC-2103945 (Awkward Array).
We also thank Erez Shinan and the developers of the Lark standalone parser, which is used to parse type strings as type objects.
Thanks especially to the gracious help of Awkward Array contributors (including the original repository).
💻: code, 📖: documentation, 🚇: infrastructure, 🚧: maintenance, ⚠: tests and feedback, 🤔: foundational ideas.
Owner
- Name: Scikit-HEP Project
- Login: scikit-hep
- Kind: organization
- Email: scikit-hep-forum@googlegroups.com
- Website: https://scikit-hep.org
- Repositories: 46
- Profile: https://github.com/scikit-hep
A community project for High Energy Physics data analysis in Python
Citation (CITATION.cff)
cff-version: 1.2.0 title: "Awkward Array" message: "If you use this software, please cite it as below." doi: "10.5281/zenodo.4341376" date-released: "2018-10-12" authors: - family-names: "Pivarski" given-names: "Jim" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0002-6649-343X" email: "jpivarski@gmail.com" - family-names: "Osborne" given-names: "Ianna" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0002-6955-1033" email: "iosborne@princeton.edu" - family-names: "Ifrim" given-names: "Ioana" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0002-6932-1385" email: "ii3193@princeton.edu" - family-names: "Schreiner" given-names: "Henry" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0002-7833-783X" email: "henryfs@princeton.edu" - family-names: "Hollands" given-names: "Angus" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0003-0788-3814" email: "goosey15@gmail.com" - family-names: "Biswas" given-names: "Anish" affiliation: "Manipal Institute Of Technology" orcid: "https://orcid.org/0000-0001-6149-9739" email: "anishbiswas271@gmail.com" - family-names: "Das" given-names: "Pratyush" affiliation: "Purdue University" orcid: "https://orcid.org/0000-0001-8140-0097" email: "reikdas@gmail.com" - family-names: "Roy Choudhury" given-names: "Santam" affiliation: "National Institute of Technology, Durgapur" orcid: "https://orcid.org/0000-0003-0153-9748" email: "santamdev404@gmail.com" - family-names: "Smith" given-names: "Nicholas" affiliation: "Fermilab" orcid: "https://orcid.org/0000-0002-0324-3054" email: "nick.smith@cern.ch" - family-names: "Goyal" given-names: "Manasvi" affiliation: "Harvard University" orcid: "https://orcid.org/0000-0001-6321-7491" email: "mg.manasvi@gmail.com" - family-names: "Fackeldey" given-names: "Peter" affiliation: "Princeton University" orcid: "https://orcid.org/0000-0003-4932-7162" email: "pf4572@princeton.edu" - family-names: "Krommydas" given-names: "Iason" affiliation: "Rice University" orcid: "https://orcid.org/0000-0001-7849-8863" email: "ik23@rice.edu"
Committers
Last synced: 8 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Jim Pivarski | j****i | 962 |
| Angus Hollands | g****5@g****m | 605 |
| Ianna Osborne | i****e@c****h | 166 |
| pre-commit-ci[bot] | 6****] | 117 |
| dependabot[bot] | 4****] | 79 |
| Henry Schreiner | h****r@c****h | 74 |
| ioanaif | i****m@g****m | 68 |
| allcontributors[bot] | 4****] | 59 |
| Pratyush Das | r****s@g****m | 50 |
| Peter Fackeldey | f****r@g****m | 41 |
| Manasvi Goyal | 5****l | 33 |
| Iason Krommydas | i****m@g****m | 21 |
| Anish Biswas | a****1@g****m | 20 |
| Nicholas Smith | n****h@c****h | 17 |
| Topher Cawlfield | 4****d | 12 |
| Anish Biswas | 8****f | 11 |
| Nikolai Hartmann | n****e@p****e | 9 |
| Dmitry Kalinkin | d****n@g****m | 8 |
| Matthew Feickert | m****t@c****h | 8 |
| Saransh Chopra | s****1@g****m | 8 |
| Andrea Zonca | c****e@a****m | 7 |
| Lindsey Gray | l****y@g****m | 7 |
| Martin Durant | m****t | 6 |
| maxymnaumchyk | 7****k | 5 |
| Doug Davis | d****s@d****o | 4 |
| Chris Burr | c****r | 4 |
| Anish Biswas | 8****n | 4 |
| Zoë Bilodeau | 7****a | 3 |
| Andres Rios Tascon | a****s@g****m | 3 |
| Anish Biswas | a****a@g****m | 2 |
| and 26 more... | ||
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 4 months ago
All Time
- Total issues: 398
- Total pull requests: 1,107
- Average time to close issues: 4 months
- Average time to close pull requests: 10 days
- Total issue authors: 80
- Total pull request authors: 35
- Average comments per issue: 2.06
- Average comments per pull request: 2.22
- Merged pull requests: 863
- Bot issues: 1
- Bot pull requests: 263
Past Year
- Issues: 94
- Pull requests: 459
- Average time to close issues: 16 days
- Average time to close pull requests: 5 days
- Issue authors: 31
- Pull request authors: 24
- Average comments per issue: 1.34
- Average comments per pull request: 1.86
- Merged pull requests: 344
- Bot issues: 0
- Bot pull requests: 111
Top Authors
Issue Authors
- agoose77 (120)
- jpivarski (74)
- ianna (31)
- lgray (29)
- ikrommyd (22)
- alexander-held (7)
- kmohrman (6)
- pfackeldey (6)
- nsmith- (5)
- martindurant (5)
- ariostas (4)
- gipert (3)
- yimuchen (3)
- jrueb (3)
- HealthyPear (3)
Pull Request Authors
- agoose77 (198)
- ianna (137)
- dependabot[bot] (112)
- pre-commit-ci[bot] (110)
- jpivarski (106)
- pfackeldey (102)
- ikrommyd (65)
- ManasviGoyal (54)
- allcontributors[bot] (41)
- henryiii (39)
- tcawlfield (22)
- Saransh-cpp (16)
- matthewfeickert (15)
- maxymnaumchyk (11)
- ariostas (10)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
- Total downloads: unknown
-
Total dependent packages: 18
(may contain duplicates) -
Total dependent repositories: 21
(may contain duplicates) - Total versions: 106
conda-forge.org: awkward
- Homepage: https://github.com/scikit-hep/awkward
- License: BSD-3-Clause
-
Latest release: 1.10.2
published about 3 years ago
Rankings
conda-forge.org: awkward-numba
- Homepage: https://github.com/scikit-hep/awkward
- License: BSD-3-Clause
-
Latest release: 0.14.0
published about 5 years ago
Rankings
Dependencies
- pytest >=6
- pytest-cov *
- actions/cache v3 composite
- actions/checkout v4 composite
- actions/setup-python v4 composite
- codecov/codecov-action v3 composite
- actions/checkout v4 composite
- actions/download-artifact v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact v3 composite
- docker/setup-qemu-action v3.0.0 composite
- pypa/cibuildwheel v2.15.0 composite
- pypa/gh-action-pypi-publish v1.8.10 composite
- actions/checkout v4 composite
- actions/download-artifact v3 composite
- actions/upload-artifact v3 composite
- pypa/gh-action-pypi-publish v1.8.10 composite
- softprops/action-gh-release v1 composite
- actions/checkout v4 composite
- aws-actions/configure-aws-credentials v4 composite
- actions/cache v3 composite
- actions/checkout v4 composite
- actions/download-artifact v3 composite
- actions/setup-python v4 composite
- actions/upload-artifact v3 composite
- aws-actions/configure-aws-credentials v4 composite
- mamba-org/setup-micromamba v1 composite
- mymindstorm/setup-emsdk v12 composite
- actions/checkout v4 composite
- actions/checkout v4 composite
- actions/checkout v4 composite
- actions/checkout v4 composite
- actions/upload-artifact v3 composite
- pypa/cibuildwheel v2.15.0 composite
- amannn/action-semantic-pull-request v5.2.0 composite
- actions/cache v3 composite
- actions/checkout v4 composite
- actions/setup-python v4 composite
- codecov/codecov-action v3 composite
- jwlawson/actions-setup-cmake v1.14 composite
- mamba-org/setup-micromamba v1 composite
- importlib_resources python_version < "3.9"
- numpy >=1.18.0
- pydantic <2
- pyodide-build ==0.23.3
- PyYAML *
- awkward *
- black *
- fsspec *
- h5py *
- ipyleaflet *
- jax >=0.2.7
- jaxlib >=0.1.57
- jupyterlite ==0.1.0
- jupyterlite-pyodide-kernel ==0.0.8
- jupyterlite-sphinx ==0.8.0
- lark-parser *
- matplotlib *
- myst-nb *
- numba >=0.50.0
- numexpr *
- numpy *
- pandas >=0.24.0
- pyarrow >=7.0.0
- pycparser *
- pydata-sphinx-theme *
- s3fs *
- sphinx >=4.5.0,<5.0.0
- sphinx-copybutton *
- sphinx-design *
- sphinx-external-toc *
- sphinx-sitemap *
- uproot *
- uproot3 *
- awkward >=2.0.0
- awkward >=2.0
- numpy *
- awkward_cpp ==23
- importlib_metadata >=4.13.0;python_version < "3.12"
- numpy >=1.18.0
- packaging *
- typing_extensions >=4.1.0; python_version < "3.11"