Data

Tools

Indexes

Applications

Experiments

Open Source Science

dsgrid

Python package for working with demand-side grid projects, datasets and queries

https://github.com/dsgrid/dsgrid

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.4%) to scientific vocabulary

Keywords

apache-spark electricity-load energy-data energy-demand energy-demand-forecasting python
Last synced: 6 months ago · JSON representation

Repository

Python package for working with demand-side grid projects, datasets and queries

Basic Info
Statistics
  • Stars: 28
  • Watchers: 5
  • Forks: 5
  • Open Issues: 59
  • Releases: 1
Topics
apache-spark electricity-load energy-data energy-demand energy-demand-forecasting python
Created about 5 years ago · Last pushed 6 months ago
Metadata Files
Readme License

README.md

dsgrid

Documentation codecov

Python API for contributing to and accessing demand-side grid model (dsgrid) projects and datasets.

⚠️ dsgrid is under active development and does not yet have a formal package release. Details listed here are subject to change. Please reach out to the dsgrid coordination team with any questions or other feedback. ⚠️

Install | Usage | Uninstall

Install

Virtual environment | Dependencies | from PIPY/pip | from pip+git | from cloned repository

Virtual environment

Create a virtual environment in which to install dsgrid. Anaconda or miniconda is recommended.

conda create -n dsgrid python=3.10 conda activate dsgrid

Dependencies

dsgrid uses Apache Spark to manage big data. There are no separate installation steps for Apache Spark beyond installing the dsgrid package, because the pyspark Python dependency handles it. However, you should be aware that Apache Spark's Microsoft Windows support is poor and essentially limited to local mode. That is, if you use dsgrid on a Windows machine you should not attempt to install a full version of Spark nor expect to run on a Spark cluster. As such, we recommend limiting dsgrid use on Windows to browsing the registry, registering and submitting small- to medium-sized datasets, or development work with small test projects. Full dsgrid functionality with large projects requires additional computational resources, e.g., high performance or cloud computing, typically on a Linux operating system.

Additional Notes

  • If pyspark complains about not finding Python, you may need to locate your python executable file (python.exe on Windows), copy it, and rename the copy to python3 (python3.exe on Windows)

Spark requires Java 8 or later with the JAVA_HOME environment variable set to the Java installation directory.

On Linux you can install OpenJDK with conda: conda install openjdk

Windows install instructions are below.

Windows

To install Apache Spark on Windows, follow these instructions.

From PIPY/pip

Not yet available

From pip+git

With ssh keys: ``` pip install git+ssh://git@github.com/dsgrid/dsgrid.git@main

or

pip install git+ssh://git@github.com/dsgrid/dsgrid.git@develop ```

From http: ``` pip install git+https://github.com/dsgrid/dsgrid.git@main

or

pip install git+https://github.com/dsgrid/dsgrid.git@develop ```

From Cloned Repository

First, clone the repository and change into the dsgrid directory. For example:

cd ~ # or other directory where you put repositories git clone git@github.com:dsgrid/dsgrid.git # or the http address cd dsgrid

Then install the pacakge using the pip -e flag to directly use the files in the cloned repository.

Users: pip install -e .

Developers: pip install -e '.[dev]'

Usage

dsgrid is primarily a command-line interface (CLI) tool. To see the available commands: dsgrid --help

Uninstall

pip uninstall dsgrid

If you are using a conda environment conda deactivate

Software Record

dsgrid is developed under NREL Software Record SWR-21-52, "demand-side grid model".

Owner

  • Name: NREL-dsgrid
  • Login: dsgrid
  • Kind: organization

GitHub Events

Total
  • Create event: 29
  • Issues event: 26
  • Release event: 1
  • Watch event: 4
  • Delete event: 29
  • Member event: 3
  • Issue comment event: 60
  • Push event: 214
  • Pull request event: 41
  • Pull request review comment event: 169
  • Pull request review event: 126
  • Fork event: 2
Last Year
  • Create event: 29
  • Issues event: 26
  • Release event: 1
  • Watch event: 4
  • Delete event: 29
  • Member event: 3
  • Issue comment event: 60
  • Push event: 214
  • Pull request event: 41
  • Pull request review comment event: 169
  • Pull request review event: 126
  • Fork event: 2

Committers

Last synced: 6 months ago

All Time
  • Total Commits: 1,539
  • Total Committers: 6
  • Avg Commits per committer: 256.5
  • Development Distribution Score (DDS): 0.349
Past Year
  • Commits: 131
  • Committers: 4
  • Avg Commits per committer: 32.75
  • Development Distribution Score (DDS): 0.206
Top Committers
Name Email Commits
Daniel Thom d****m@n****v 1,002
lixiliu 3****u 269
mooneyme m****y@n****v 177
Elaine Hale e****e@n****v 77
zack jensen z****n@n****v 7
roliveir r****a@n****v 7
Committer Domains (Top 20 + Academic)
nrel.gov: 5

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 107
  • Total pull requests: 164
  • Average time to close issues: 9 months
  • Average time to close pull requests: 21 days
  • Total issue authors: 5
  • Total pull request authors: 5
  • Average comments per issue: 0.82
  • Average comments per pull request: 1.29
  • Merged pull requests: 143
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 15
  • Pull requests: 39
  • Average time to close issues: N/A
  • Average time to close pull requests: 8 days
  • Issue authors: 2
  • Pull request authors: 4
  • Average comments per issue: 0.07
  • Average comments per pull request: 1.36
  • Merged pull requests: 30
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • daniel-thom (68)
  • elainethale (28)
  • lixiliu (8)
  • ashreeta (2)
  • jfhawkin (1)
Pull Request Authors
  • daniel-thom (141)
  • lixiliu (13)
  • elainethale (7)
  • JensZack (2)
  • mooneyme (1)
Top Labels
Issue Labels
enhancement (14) bug (9) limitation (6) low priority (4) future (3) AWS (2) documentation (2) missing_feature (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 1
proxy.golang.org: github.com/dsgrid/dsgrid
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.4%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 6 months ago

Dependencies

.github/workflows/codecov.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v3 composite
.github/workflows/gh-pages.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • peaceiris/actions-gh-pages v3.6.1 composite
.github/workflows/lint.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • psf/black stable composite
  • py-actions/flake8 v2 composite
.github/workflows/pull_request_tests.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v3 composite
docker/spark/Dockerfile docker
  • python 3.10-slim build
pyproject.toml pypi
emr/environment.yml pypi