Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/broadinstitute/terra-tools

scripts for use with Terra

https://github.com/broadinstitute/terra-tools

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 3 committers (33.3%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.1%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

scripts for use with Terra

Basic Info
  • Host: GitHub
  • Owner: broadinstitute
  • License: bsd-3-clause
  • Language: Jupyter Notebook
  • Default Branch: master
  • Size: 60.5 KB
Statistics
  • Stars: 7
  • Watchers: 26
  • Forks: 4
  • Open Issues: 2
  • Releases: 0
Created about 6 years ago · Last pushed about 2 years ago
Metadata Files
Readme License

README.md

Tools for use with Terra

To run a script locally:

  1. Clone the repository and cd into the terra-tools directory
  2. Create a virtual environment by running python3 -m venv venv
  3. Activate the virtual environment by running source venv/bin/activate
  4. Install the required packages by running pip install -r requirements.txt
  5. Authenticate with Google Cloud by running gcloud auth application-default login
  6. Run the script by running python3 scripts/path_to_script/<script name.py> <arguments>

To run a script using Docker:

docker run --rm -it -v "$HOME"/.config:/.config -v "$HOME"/Documents/:/data broadinstitute/terra-tools:latest bash -c "cd data; python3 /scripts/path_to_script/<script name.py> <arguments>"

  1. -v "$HOME"/.config:/.config
    • allows for authentication within the Docker of your Google credentials in your local $HOME directory where they are stored by default
  2. -v "$HOME"/Documents/:/data
    • mounts the directory named "Documents" to a folder named "data" within the Docker
    • files generated by scripts will be written to your "Documents" directory
    • change "Documents" to the path of any location on your local machine
  3. bash -c
    • directs Docker to run the following bash commands (within quotes and separated by semi-colon)

For example, to run the import_large_tsv.py script: 1. move input file (<tsv_name>) to Documents/ directory locally 2. run docker run --rm -v "$HOME"/.config:/.config -v "$HOME"/Documents/:/data broadinstitute/terra-tools:latest bash -c "cd data; python3 scripts/import_large_tsv/import_large_tsv.py --tsv data/<tsv_name> --project <terra_project> --workspace <terra_workspace>"

Prerequisites

  • Install the Google Cloud SDK from https://cloud.google.com/sdk/downloads
  • Set the Application Default Credentials (run gcloud auth application-default login)
  • Python 3.7

When running without the docker, check the packages in requirements.txt.

Owner

  • Name: Broad Institute
  • Login: broadinstitute
  • Kind: organization
  • Location: Cambridge, MA

Broad Institute of MIT and Harvard

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 29
  • Total Committers: 3
  • Avg Commits per committer: 9.667
  • Development Distribution Score (DDS): 0.414
Past Year
  • Commits: 4
  • Committers: 1
  • Avg Commits per committer: 4.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Morgan Taylor m****g@b****g 17
Sushma Chaluvadi 4****a@u****m 8
Willy Nojopranoto n****o@g****m 4
Committer Domains (Top 20 + Academic)
broadinstitute.org: 1

Issues and Pull Requests

Last synced: over 2 years ago

All Time
  • Total issues: 0
  • Total pull requests: 16
  • Average time to close issues: N/A
  • Average time to close pull requests: about 16 hours
  • Total issue authors: 0
  • Total pull request authors: 4
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 14
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: about 11 hours
  • Issue authors: 0
  • Pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • schaluva (8)
  • mmorgantaylor (4)
  • wnojopra (3)
  • samanthahv (1)
  • dependabot[bot] (1)
  • kilonzi (1)
Top Labels
Issue Labels
Pull Request Labels
dependencies (1)

Dependencies

requirements.txt pypi
  • firecloud ==0.16.26
  • gcs-oauth2-boto-plugin ==2.5
  • google-api-python-client ==1.8.0
  • google-auth-httplib2 ==0.0.3
  • google-cloud-bigquery ==2.7.0
  • google-cloud-storage ==1.38.0
  • pandas ==1.3.0
  • tqdm ==4.35.0
Dockerfile docker
  • google/cloud-sdk 295.0.0 build