carbonifer

Control carbon emission of your cloud infrastructure - Carbon-aware cloud infra

https://github.com/carboniferio/carbonifer

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
  • DOI references
  • Academic publication links
  • Committers with academic emails
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (13.0%) to scientific vocabulary

Keywords

aws azure carbon-aware carbon-emissions carbon-footprint carbonifer cloud data-centers gcp greenops sustainability terraform
Last synced: 6 months ago · JSON representation

Repository

Control carbon emission of your cloud infrastructure - Carbon-aware cloud infra

Basic Info
  • Host: GitHub
  • Owner: carboniferio
  • License: apache-2.0
  • Language: Go
  • Default Branch: main
  • Homepage: https://carbonifer.io/
  • Size: 374 KB
Statistics
  • Stars: 99
  • Watchers: 6
  • Forks: 11
  • Open Issues: 15
  • Releases: 9
Topics
aws azure carbon-aware carbon-emissions carbon-footprint carbonifer cloud data-centers gcp greenops sustainability terraform
Created about 3 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License Code of conduct

README.md

Go Go Reference

Carbonifer Logo

https://carbonifer.io/

Command Line Tool to control carbon emission of your cloud infrastructure. Reading Terraform files, carbonifer plan will estimate future Carbon Emissions of infrastructure and help make the right choices to reduce Carbon footprint.

Scope

This tool currently estimates usage emissions, not embodied emissions (manufacturing, transport, recycling...). It is not a full LCA (Life Cycle Assessment) tool.

This tool can analyze Infrastructure as Code definitions such as:

It can estimate Carbon Emissions of:

  • Google Cloud Platform
    • [x] Compute Engine
    • [x] Compute Instances (generic and custom machine types, and from template)
    • [x] Disks (boot, persistent and region-persistent, HDD or SSD)
    • [X] Machines with GPUs
    • [x] Cloud SQL
    • [x] Instance Group (including regional and Autoscaler)
    • [x] Google Kubernetes Engine (GKE) cluster
  • Amazon Web Services
    • [x] EC2 (including inline root, elastic, and ephemeral block storages)
    • [x] EBS Volumes
    • [x] RDS
    • [x] AutoScaling Group

The following will also be supported soon:

  • Amazon Web Services
    • [ ] Elastic Kubernetes Service (EKS)
    • [ ] Elastic Container Service (ECS)
  • Azure
    • [ ] Virtual Machines
    • [ ] Virtual Machine Scale Set
    • [ ] SQL

NB: This list of resources will be extended in the future A list of supported resource types is available in the Scope document.

Install Carbonifer CLI

Magic one-liner bash script

Using bash and curl. By default, it will install the latest version to /usr/local/bin:

bash curl -sfL https://raw.githubusercontent.com/carboniferio/carbonifer/main/install.sh | sudo bash

if you need more flexibility

bash export DEST_DIR=/path/to/dest/dir export VERSION=v1.2.3 curl -sSL https://github.com/username/repo/install.sh | bash

Go install

If you have go installed, you can use:

bash go install github.com/carboniferio/carbonifer@latest

Go will automatically install it in your $GOPATH/bin directory which should be in your $PATH.

Docker alternative

Alternatively, you can use the Docker image:

bash git clone https://github.com/carboniferio/carbonifer.git cd carbonifer docker build -t carbonifer . docker run -it --rm -v <your_tf_folder_with_config>:/tmp/ carbonifer ./carbonifer --config=/tmp/config.yaml plan /tmp/

Manual install

Download the latest release from releases page

Plan

carbonifer plan will read your Terraform folder and estimates Carbon Emissions.

```bash $ carbonifer plan

Average estimation of CO2 emissions per instance:


resource count replicas emissions per instance


googlecomputedisk.first 1 1 0.0422 gCO2eq/h
googlecomputeinstance.first 1 1 33.5977 gCO2eq/h
googlecomputeinstance.second 1 1 0.4248 gCO2eq/h
googlecomputeregiondisk.regional-first 1 2 0.0844 gCO2eq/h
google
sqldatabaseinstance.instance 1 2 2.0550 gCO2eq/h
googlecomputesubnetwork.first unsupported
googlecomputenetwork.vpc_network unsupported


Total 7 38.3433 gCO2eq/h


```

In case instances are in a managed group (GCP managed instance group, AWS autoscaling group...), the instances appear in the group name, with a count > 1 and emissions are shown for 1 instance. Of course, Total will sum all instances of the group:

```bash


resource type name count emissions per instance


googlecomputeinstancegroupmanager my-group-manager 3 0.5568 gCO2eq/h
googlecomputenetwork vpcnetwork unsupported
google
compute_subnetwork first unsupported


                                      Total              3        1.6704 gCO2eq/h        

```

The report is customizable (text or JSON, per hour, month...), cf Configuration

Example of a JSON report

```json { "Info": { "UnitTime": "h", "UnitWattTime": "Wh", "UnitCarbonEmissionsTime": "gCO2eq/h", "DateTime": "2023-02-18T14:52:08.757999+01:00", "AverageCPUUsage": 0.5, "AverageGPUUsage": 0.5 }, "Resources": [ { "Resource": { "Identification": { "Name": "first", "ResourceType": "google_compute_disk", "Provider": 2, "Region": "europe-west9", "Count": 1 }, "Specs": { "GpuTypes": null, "HddStorage": "1024", "SsdStorage": "0", "MemoryMb": 0, "VCPUs": 0, "ReplicationFactor": 1 } }, "PowerPerInstance": "0.76096", "CarbonEmissionsPerInstance": "0.04489664", "AverageCPUUsage": "0.5", "Count": "1" }, { "Resource": { "Identification": { "Name": "first", "ResourceType": "google_compute_instance", "Provider": 2, "Region": "europe-west9", "Count": 1 }, "Specs": { "GpuTypes": [ "nvidia-tesla-a100", "nvidia-tesla-k80", "nvidia-tesla-k80" ], "HddStorage": "0", "SsdStorage": "1317", "MemoryMb": 87040, "VCPUs": 12, "CPUType": "", "ReplicationFactor": 1 } }, "PowerPerInstance": "733.5648917187", "CarbonEmissionsPerInstance": "43.2803286114", "AverageCPUUsage": "0.5", "Count": "1" }, { "Resource": { "Identification": { "Name": "second", "ResourceType": "google_compute_instance", "Provider": 2, "Region": "europe-west9", "Count": 1 }, "Specs": { "GpuTypes": null, "HddStorage": "10", "SsdStorage": "0", "MemoryMb": 4098, "VCPUs": 2, "CPUType": "", "ReplicationFactor": 1 } }, "PowerPerInstance": "7.6091047343", "CarbonEmissionsPerInstance": "0.4489371793", "AverageCPUUsage": "0.5", "Count": "1" }, { "Resource": { "Identification": { "Name": "regional-first", "ResourceType": "google_compute_region_disk", "Provider": 2, "Region": "europe-west9", "Count": 1 }, "Specs": { "GpuTypes": null, "HddStorage": "1024", "SsdStorage": "0", "MemoryMb": 0, "VCPUs": 0, "CPUType": "", "ReplicationFactor": 2 } }, "PowerPerInstance": "1.52192", "CarbonEmissionsPerInstance": "0.08979328", "AverageCPUUsage": "0.5", "Count": "1" }, { "Resource": { "Identification": { "Name": "instance", "ResourceType": "google_sql_database_instance", "Provider": 2, "Region": "europe-west9", "Count": 1 }, "Specs": { "GpuTypes": null, "HddStorage": "0", "SsdStorage": "10", "MemoryMb": 15360, "VCPUs": 4, "CPUType": "", "ReplicationFactor": 2 } }, "PowerPerInstance": "36.807506875", "CarbonEmissionsPerInstance": "2.1716429056", "AverageCPUUsage": "0.5", "Count": "1" } ], "UnsupportedResources": [ { "Identification": { "Name": "vpc_network", "ResourceType": "google_compute_network", "Provider": 2, "Region": "", "Count": 1 } }, { "Identification": { "Name": "first", "ResourceType": "google_compute_subnetwork", "Provider": 2, "Region": "europe-west9", "Count": 1 } } ], "Total": { "Power": "780.264383328", "CarbonEmissions": "46.0355986163", "ResourcesCount": "5" } } ```

Existing terraform plan file

In case you want to read an existing terraform file, you need to pass it as argument. It can either be a raw tfplan or a json plan. This is useful when some variables or credentials are required to run terraform plan. In that case carbonifer plan won't try to run terraform plan for you, and won't expect to have any credentials or variable set (via env var...)

bash carbonifer plan /path/to/my/project.tfplan

Methodology

This tool will:

  1. Read resources from Terraform folder
  2. Calculate an estimation of power used by those resources in Watt per Hour
  3. Translate this electrical power into an estimation of Carbon Emissions depending on

We can estimate the Carbon Emissions of a resource by taking the electric use of a resource (in Watt-hour) and multiplying it by a carbon emission factor. This carbon emission factor depends on:

  • Cloud Provider
  • Region of the Data Center
  • The energy mix of this region/country
    • Average
    • (future) real-time

Those calculations and estimations are detailed in the Methodology document.

Limitations

We are currently supporting only

  • resources with a significative power usage (basically anything that has CPU, GPU, memory or disk)
  • resources that can be estimated beforehand (we discard for now data transfer)

Because this is just an estimation, the actual power usage and carbon emission should probably differ depending on the actual usage of the resource (CPU %), and actual grid energy mix (could be weather dependent), ... But that should be enough to make decisions about the choice of provider/region, instance type...

See the Scope document for more details.

Usage

carbonifer plan [target]

  • target can be
    • a terraform project folder
    • a terraform plan file (json or raw)
    • default: the current folder

Prerequisites

  • Terraform :
    • Carbonifer uses Terraform:
    • terrafom executable available in $PATH
    • if not existing, it installs it in a temp folder (.carbonifer)
    • versions supported
  • Cloud provider credentials (optional):
    • if not provided, if terraform does not need it, it won't fail
    • if terraform needs it (to read disk image info...), it will get credentials the same way terraform gets its credentials
    • terraform Google provider
    • terraform AWS provider
    • terraform Azure provider

Configuration

| Yaml key | CLI flag | Default | Description |---|---|---|---| | unit.time | | h | Time unit: h (hour), m (month), y (year) | unit.power | | w | Power unit: W (watt) or kW | unit.carbon | | g | Carbon emission in g (gram) or kg | out.format | -f <format> --format=<format> | text | text or json | out.file | -o <filename> --output=<filename>| | file to write report to. Default is standard output. | data.path | <arg> | | path of carbonifer data files (coefficents...). Default uses embedded files in binary | avg_cpu_use | | 0.5 | planned average percentage of CPU used | log | | warn | level of logs info, debug, warn, error

Extending Carbonifer

In order to add support for a new terraform resource type, there is a mapping mechanism where we can declare JQ filters to query the Terraform file and extract the necessary information.

Owner

  • Name: Carbonifer
  • Login: carboniferio
  • Kind: organization
  • Email: contact@carbonifer.io
  • Location: France

Take Control of the Carbon Emissions of your Cloud

GitHub Events

Total
  • Watch event: 13
  • Fork event: 2
Last Year
  • Watch event: 13
  • Fork event: 2

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 86
  • Total Committers: 3
  • Avg Commits per committer: 28.667
  • Development Distribution Score (DDS): 0.047
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Olivier Bierlaire o****r@c****o 82
Mathis Joffre 5****f 3
Olivier Bierlaire o****r@O****n 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 37
  • Total pull requests: 67
  • Average time to close issues: about 1 month
  • Average time to close pull requests: about 17 hours
  • Total issue authors: 5
  • Total pull request authors: 4
  • Average comments per issue: 0.59
  • Average comments per pull request: 0.12
  • Merged pull requests: 63
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • obierlaire (31)
  • Joffref (2)
  • BBBmau (1)
  • punndcoder28 (1)
  • Wolfsrudel (1)
Pull Request Authors
  • obierlaire (56)
  • Joffref (3)
  • baizon (2)
  • peterzahemszky (1)
Top Labels
Issue Labels
enhancement (20) aws (11) azure (4) bug (3) meta (2) gcp (1) good first issue (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 12
proxy.golang.org: github.com/carboniferio/carbonifer

Copyright © 2023 contact@carbonifer.io Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

  • Versions: 12
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 7.0%
Average: 8.2%
Dependent repos count: 9.3%
Last synced: 6 months ago

Dependencies

.github/workflows/go.yml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • golangci/golangci-lint-action v3 composite
  • google-github-actions/auth v1 composite
  • hashicorp/setup-terraform v2 composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-go v3 composite
  • crazy-max/ghaction-import-gpg v5 composite
  • goreleaser/goreleaser-action v4 composite
go.mod go
  • cloud.google.com/go/compute v1.14.0
  • cloud.google.com/go/compute/metadata v0.2.3
  • github.com/davecgh/go-spew v1.1.1
  • github.com/fsnotify/fsnotify v1.6.0
  • github.com/gogap/env_json v0.0.0-20150503135429-86150085ddbe
  • github.com/gogap/env_strings v0.0.1
  • github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
  • github.com/golang/protobuf v1.5.2
  • github.com/google/uuid v1.3.0
  • github.com/googleapis/enterprise-certificate-proxy v0.2.1
  • github.com/googleapis/gax-go/v2 v2.7.0
  • github.com/hashicorp/go-cleanhttp v0.5.2
  • github.com/hashicorp/go-version v1.6.0
  • github.com/hashicorp/hc-install v0.4.0
  • github.com/hashicorp/hcl v1.0.0
  • github.com/hashicorp/terraform-exec v0.17.2
  • github.com/hashicorp/terraform-json v0.14.0
  • github.com/heirko/go-contrib v0.0.0-20200825160048-11fc5e2235fa
  • github.com/heralight/logrus_mate v1.0.1
  • github.com/hoisie/redis v0.0.0-20160730154456-b5c6e81454e0
  • github.com/inconshreveable/mousetrap v1.0.1
  • github.com/magiconair/properties v1.8.7
  • github.com/mattn/go-runewidth v0.0.9
  • github.com/mitchellh/mapstructure v1.5.0
  • github.com/olekukonko/tablewriter v0.0.5
  • github.com/pelletier/go-toml/v2 v2.0.6
  • github.com/pmezard/go-difflib v1.0.0
  • github.com/shopspring/decimal v1.3.1
  • github.com/sirupsen/logrus v1.9.0
  • github.com/spf13/afero v1.9.3
  • github.com/spf13/cast v1.5.0
  • github.com/spf13/cobra v1.6.1
  • github.com/spf13/jwalterweatherman v1.1.0
  • github.com/spf13/pflag v1.0.5
  • github.com/spf13/viper v1.15.0
  • github.com/stretchr/testify v1.8.1
  • github.com/subosito/gotenv v1.4.2
  • github.com/yunabe/easycsv v0.0.2
  • github.com/zclconf/go-cty v1.10.0
  • go.opencensus.io v0.24.0
  • golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
  • golang.org/x/net v0.5.0
  • golang.org/x/oauth2 v0.4.0
  • golang.org/x/sys v0.4.0
  • golang.org/x/text v0.6.0
  • google.golang.org/api v0.108.0
  • google.golang.org/appengine v1.6.7
  • google.golang.org/genproto v0.0.0-20221227171554-f9683d7f8bef
  • google.golang.org/grpc v1.52.0
  • google.golang.org/protobuf v1.28.1
  • gopkg.in/ini.v1 v1.67.0
  • gopkg.in/yaml.v3 v3.0.1
go.sum go
  • 1357 dependencies
Dockerfile docker
  • golang 1.20 build