Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/broadinstitute/carrot

https://github.com/broadinstitute/carrot - 1.0.0

List of features:

  • Query runs by software versions
    • REST API mappings and carrotcli commands for finding runs now have a softwareversions parameters which allows filtering by software and tags/commits used to build the runs. (#261)
  • Generate reports from run queries
    • Added REST API mappings and carrot_cli commands to submit run query parameters for generating a report from the runs matching those parameters. (#261)
  • Use any valid git refs for software versions
    • Fixed an issue where submitting a software build version that is a valid git ref but not a commit or tag would result in issues. (#269)
  • Updated user guide
    • Added clarity and updated outdated information in the user guide. (#264)

- Rust
Published by KevinCLydon over 3 years ago

https://github.com/broadinstitute/carrot - 0.6.3

List of features:

  • Create from copy for templates
    • Functionality for creating a template from a copy of another template has been added. It works the same as creating a test from a copy (#256)
  • Tracking of software tags
    • CARROT now keeps local metadata-only clones of git repos for software so that it can keep track of tags that correspond to commits and vice-versa for software versions. This is in preparation for the planned functionality for querying runs by software versions built for those runs. (#251)
  • Loosened restrictions on template and test editing
    • Runs will now keep track of the WDLs used the generate them. This is to allow removing the restriction on editing templates and tests after a successful run has been generated. (#257)
  • Fixed a rare bug that could potentially cause GitHub config credential exposure
    • There was an issue where, if GitHub authentication failed for some reason when attempting to run a test from GitHub, the credentials CARROT uses to connect to GitHub could be posted under the comment for running the test. This is very rare, because it would require that authentication failed for accessing the repo, but succeeded for posting the error to it. However, we have modified the way CARROT handles git credentials to prevent this in the future. (#263)

- Rust
Published by KevinCLydon over 3 years ago

https://github.com/broadinstitute/carrot - 0.6.2

List of features:

  • Support for R notebooks in reports
    • It is now possible to define Jupyter notebooks for reports that use the R programming language, using a docker image generated from the most recent version of the report Dockerfile (#254)
  • Create test from copy
    • Added functionality to the test create REST mapping (using the copy query param) and CLI command (using the --copy option) to allow creating new tests by copying from existing ones and specifying differences (#248)

- Rust
Published by KevinCLydon over 3 years ago

https://github.com/broadinstitute/carrot - 0.6.1

List of features:

  • No more consecutive failures limit for status manager
    • The consecutive failures limit for status updating was resulting in issues where the status manager thread would terminate due to minor transient issues with Cromwell, so it has been removed (#252)

- Rust
Published by KevinCLydon over 3 years ago

https://github.com/broadinstitute/carrot - 0.6.0

List of features:

  • GitHub PR Comparison runs
    • Added a new type of github run request for triggering two runs: one of the head commit, and one of the base commit (#242)
    • Added support for generating reports from these PR Comparison runs (#247)
  • Other
    • Fixed some dependency issues and bumped the required Rust up to 1.59.0

- Rust
Published by KevinCLydon over 3 years ago

https://github.com/broadinstitute/carrot - 0.5.1

List of features:

  • Specify machine-type for Google Cloud Build at the software level
    • Added a machine_type attribute to software, which will be used to specify what machine-type Google Cloud Build should use when building images for that software (#229)
  • Other
    • Updated some out-of-date dependencies (#222)

- Rust
Published by KevinCLydon about 4 years ago

https://github.com/broadinstitute/carrot - 0.5.0

List of features:

  • Download run data as CSVs
    • It is now possible to download run data as a ZIP of CSV files using the csv query param with the existing REST mappings for retrieving run data. This is enabled through the CLI with the --zip_csv option. (#217)
  • New format for reports
    • The processing for report generation has changed. (#215)
      • Previously, report notebooks would have to be made to allow the insertion of several default cells which fill in the data for the run. Now, the only default cell that will be inserted is the header.
      • Instead of supplying run information in a python dict in a cell, it will now be provided in the form of the CSVs that would be returned by a call to one of the find run REST mappings, as specified above.
      • Now, users will be able to prototype their report notebooks by downloading data for runs as CSVs and build the notebooks to open those files and load the data from there
  • GitHub comments look better now
    • GitHub comments for runs now have clickable links to files stored in GCS and also they're formatted to be a bit more readable now. (#214)

- Rust
Published by KevinCLydon about 4 years ago

https://github.com/broadinstitute/carrot - 0.4.1

List of features:

  • Bug fix
    • Fixed a bug preventing the creation of new reports (#209)

- Rust
Published by KevinCLydon about 4 years ago

https://github.com/broadinstitute/carrot - 0.4.0

List of features:

  • Merged with CARROT_CLI
    • Going forward, the code for CARROT_CLI now lives in this repo, to consolidate issue tracking to this repo and keep version numbers consistent. (#183)
  • Support for WDL Dependencies
    • WDL dependency zips can now be provided alongside WDLs when creating a Template. WDLs and dependency zips can be provided either via URIs as before, or uploaded from local files using multipart data. (#182)
  • Run error tracking
    • Errors associated with CARROT failures for runs will now be stored in the database and returned when retrieving run information from the REST API. (#180)

- Rust
Published by KevinCLydon over 4 years ago

https://github.com/broadinstitute/carrot - 0.3.3-gamma

This is partially a bug fix release and partially a preparatory release for 0.4.0.

List of features:

  • Configuration format change
    • Switched configuration to use a yaml file instead of environment variables (#159)
  • New failure mode
    • Runs will now be marked as failed when carrot fails to retrieve expected results (#149)
  • WDL Storage
    • Copies of WDLs for templates will now be stored either locally or in GCS for data provenance purposes (#142 and #166)
  • Configurable logging
    • Added the ability to specify different logging levels for different modules in the config file (#167)
  • Workflow options
    • Added fields to Test and Run entities for specifying the Cromwell workflow options file (#171)
  • Bug fixes
    • Modified the dockerfile for the docker image to use for report generation to include gsutil so result downloading works (#148)
    • Modified the configuration for the client that handles reading GitHub messages from Google PubSub in hopes of fixing the issue where the requests hang indefinitely (#173)
    • Fixed an issue where numeric run results were not being parsed properly (#175)

- Rust
Published by KevinCLydon over 4 years ago

https://github.com/broadinstitute/carrot - 0.3.2-gamma

List of features:

  • Full Error Messages from API
    • Requests to the REST API that previously resulted in a 500 status code and a generic error message will now include the full message for the error.

- Rust
Published by KevinCLydon about 5 years ago

https://github.com/broadinstitute/carrot - 0.3.1-gamma

List of features:

  • Git Tag Support
    • Modified software building functionality to allow for the use of git tags in place of commit hashes when specifying a software image build as an input using the image_build:[software_name]|[commit] format

- Rust
Published by KevinCLydon about 5 years ago

https://github.com/broadinstitute/carrot - 0.3.0-gamma

List of features:

  • Reporting
    • Added support for generating reports from completed test runs.
    • Reports are defined in the form of Jupyter Notebooks via a new entity called Report.
      • Once a report is defined, it can be mapped to a template, and then filled reports will be generated automatically for every completed test run based on that template.
      • They can also be generated manually via the REST API.
      • Reports are filled by first filling in the data for the run at the top of the notebook and then running the notebook in a WDL.
  • Updating and Deleting
    • With this change, two new things are possible:
      • Templates and tests that have no non-failed runs tied to them can receive updates that change, respectively, the WDLs and default inputs that define them.
      • Any pipeline, template, test, or result can be deleted as long as it doesn't violate their foreign key constraints (i.e. there are no rows in other tables that reference them). Template-result mappings can be deleted as long as there are no non-failed runs tied to their templates. Runs can be deleted as long as they have a failure state.
  • WDL Validation
    • When creating or updating a template, submitted WDLs will be retrieved and validated using WOMtool's validate functionality. If a WDL fails validation, the template will not be created/updated.

- Rust
Published by KevinCLydon about 5 years ago

https://github.com/broadinstitute/carrot - 0.2.1-beta

List of features:

  • Private GitHub repo support
    • GitHub integration now supports private repositories as well
      • Private repositories can be built using the custom docker build functionality
      • The carrot-publish-github-action can now be used with private repositories
      • Requires that a GitHub account used for building and commenting be given "read" access to the repository

- Rust
Published by KevinCLydon over 5 years ago

https://github.com/broadinstitute/carrot - 0.2.0-beta

List of features:

  • Subscriptions and notifications:
    • Subscribe to receive notifications for the following entities:
      • Pipelines
      • Templates
      • Tests
    • Notifications are sent as emails for the following events
      • Test run completion (either successful or unsuccessful)
      • Test run start from GitHub comment
  • Custom docker image builds
    • Docker images can now be generated automatically to supply as inputs to tests
    • Create a Software record that references a GitHub repo, specify that software with a commit as an input, and the image will be built
      • GitHub repo must be buildable by submitting to Google Cloud Build
  • GitHub integration
    • Runs can be started by commenting on a GitHub PR with a specific format
      • Repo must be configured, included adding carrot-publish-github-action to the repo's GitHub Actions workflow and creating a Software record for it
    • After a run is triggered from GitHub, comments will be posted to the PR with run information indicating that the run has started and then that it has finished
    • Currently only supports public repos
  • Just use your WDLs
    • CARROT now submits test and evaluation WDLs separately when processing a run, which means:
      • WDL 1.0 is now support
      • Using in_ and out_ prefixes when naming inputs and outputs in WDLs is no longer necessary
      • Giving corresponding names for test outputs that should map to eval inputs is no longer necessary
        • Those mappings should now be specified in the input JSONs for the eval WDL

- Rust
Published by KevinCLydon over 5 years ago

https://github.com/broadinstitute/carrot - 0.1.0-alpha

List of features:

  • REST API with the following functionality:
    • Define rerunnable tests
      • Define your tool as a pipeline
        • Group your tests based on their purpose
      • Specify a pair of WDLs for running and evaluating the results in a template
        • The WDL you want to test will run first, and its outputs will be fed into the WDL you design for evaluating them
      • Set default inputs in a test
        • Define the inputs that will be consistent between runs of the test
    • Define results to be extracted from tests
      • Specify a result for each important output you want from a test
    • Run those tests
      • Set the rest of your inputs and submit the test to run on a Cromwell server
      • Check the status of your run
    • Query results
      • View results for each run
      • Query for runs by pipeline, template, test, creator, status, date
  • Database migrations
    • Added migrations for applying and reverting PostgreSQL DB schema changes to work with CARROT
  • Docker configurations for testing and running
    • Created Dockerfile and Docker Compose config for running CARROT
      • Docker Compose file includes config for starting PostgreSQL and Cromwell servers to store data and run tests respectively
    • Created Dockerfile and Docker Compose for running unit tests without the need to set up a local environment

- Rust
Published by KevinCLydon about 6 years ago