Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/evandlang/g5nr_testing

https://github.com/evandlang/g5nr_testing

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.8%) to scientific vocabulary
Last synced: 5 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: EvanDLang
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Size: 46.9 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 1 year ago · Last pushed about 1 year ago
Metadata Files
Readme Changelog License

README.md

g5nr_testing

Time to generate sidecar files for 1 day: 69m49.437s

Setup

This uses the pixi package manager. To get a shell with all dependencies automatically installed and loaded, run pixi shell.

Alternatively, you can run individual scripts with pixi run python <script.py>.

If you want to install dependencies manually, refer to the pyproject.toml file.

To run create.py see the slurm script (slurm.sh)

Before submitting the script: - Update the path to your own bashrc file - Run pixi shell in you local terminal

Generating Sidecar Files

The sidecar generation script can be ran from the commandline. The user must provide the year, month and day. This will generate a sidecar file for one day with all the variables found in /nfs3m/css/curated01/g5nr/data/Ganymed/7km/c1440NR/DATA/0.0625deg/inst

python python generate_sidecar_files.py -y 2006 -m 1 -d 10

Changing File Paths to HTTPS

The filepaths in the sidecar files can be update to https paths with the change_paths function. The function will create a copy of the target parquet directory and will use a pattern to update filepaths.

The default arguments look for /nfs3m/css/curated01/g5nr/data/Ganymed/7km/c1440_NR and replace it with https://g5nr.nccs.nasa.gov/data.

The function can also ping each updated https path to make sure that it is valid by setting test_paths to True, This cannot be done on a Discover Node though.

```python from changepaths import changepaths

changepaths( <pathtoparquetdirectory>, pattern=r"(/nfs3m/css/curated01/g5nr/data/Ganymed/7km/c1440NR)", updatedroot="https://g5nr.nccs.nasa.gov/data", test_updates(True | False) ) ```

Owner

  • Login: EvanDLang
  • Kind: user

GitHub Events

Total
  • Member event: 1
  • Public event: 1
  • Push event: 3
  • Create event: 1
Last Year
  • Member event: 1
  • Public event: 1
  • Push event: 3
  • Create event: 1