Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/awslabs/aws-crt-python

Python bindings for the AWS Common Runtime

https://github.com/awslabs/aws-crt-python

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.3%) to scientific vocabulary

Keywords from Contributors

aws-sdk
Last synced: 10 months ago · JSON representation

Repository

Python bindings for the AWS Common Runtime

Basic Info
  • Host: GitHub
  • Owner: awslabs
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 4.67 MB
Statistics
  • Stars: 90
  • Watchers: 23
  • Forks: 48
  • Open Issues: 19
  • Releases: 232
Created over 7 years ago · Last pushed 10 months ago
Metadata Files
Readme Contributing License Code of conduct Notice

README.md

AWS CRT Python

Version

Python 3 bindings for the AWS Common Runtime.

License

This library is licensed under the Apache 2.0 License.

Minimum Requirements:

  • Python 3.8+

Installation

To install from pip:

bash python3 -m pip install awscrt

To install from Github:

bash git clone https://github.com/awslabs/aws-crt-python.git cd aws-crt-python git submodule update --init python3 -m pip install .

See Advanced Build Options for more info about building from source.

Fork and Multiprocessing

aws-crt-python uses background threads. This makes os.fork() unsafe. In a forked child process, all background threads vanish. The child will hang or crash when it tries to communicate with any of these (vanished) threads.

Unfortunately, Python's multiprocessing module defaults to using fork when it creates child processes (on POSIX systems except macOS, in Python versions 3.13 and earlier). multiprocessing is used under the hood by many tools that do work in parallel, including concurrent.futures.ProcessPoolExecutor, and pytorch.multiprocessing.

If you need to use multiprocessing with aws-crt-python, set it to use "spawn" or "forkserver" instead of "fork" (see docs). The Python community agrees, and multiprocessing will changes its default from "fork" to "spawn" in 3.14. It already uses "spawn" by default on macOS (because system libraries may start threads) and on Windows (because fork does not exist).

If you must use fork with aws-crt-python, you may be able to avoid hangs and crashes if you manage your threads very carefully:

  1. Release all CRT resources with background threads (e.g. clean up any io.EventLoopGroup instances).
  2. Join all CRT threads before forking (use common.join_all_native_threads() ).

For an example, see test.test_s3.py.S3RequestTest.test_fork_workaround .

Mac-Only TLS Behavior

Please note that on Mac, once a private key is used with a certificate, that certificate-key pair is imported into the Mac Keychain. All subsequent uses of that certificate will use the stored private key and ignore anything passed in programmatically. Beginning in v0.6.2, when a stored private key from the Keychain is used, the following will be logged at the "info" log level:

static: certificate has an existing certificate-key pair that was previously imported into the Keychain. Using key from Keychain instead of the one provided.

Crash Handler

You can enable the crash handler by setting the environment variable AWS_CRT_CRASH_HANDLER=1 . This will print the callstack to stderr in the event of a fatal error.

Advanced Build Options

OpenSSL and LibCrypto

aws-crt-python does not use OpenSSL for TLS. On Apple and Windows devices, the OS's default TLS library is used. On Unix devices, s2n-tls is used. But s2n-tls uses libcrypto, the cryptography math library bundled with OpenSSL.

To simplify installation, aws-crt-python has its own copy of libcrypto. This lets you install a wheel from PyPI without having OpenSSL installed. Unix wheels on PyPI come with libcrypto statically compiled in. Code to build libcrypto comes from AWS-LC. AWS-LC's code is included in the PyPI source package, and the git repository includes it as a submodule.

If you need aws-crt-python to use the libcrypto included on your system, set environment variable AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO=1 while building from source:

sh AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO=1 python3 -m pip install --no-binary :all: --verbose awscrt

( --no-binary :all: ensures you do not use the precompiled wheel from PyPI)

aws-crt-python also exposes a number of cryptographic primitives. On Unix, those depend on libcrypto as described above. On Apple and Windows OS level crypto libraries are used whenever possible. One exception to above statement is that for ED25519 keygen on Windows and Apple, libcrypto is used as no viable OS level alternative exists. In that case Unix level notes about libcrypto apply to Apple and Windows as well. Libcrypto usage for ED25519 support is enabled on Windows and Apple by default and can be disabled by setting environment variable AWS_CRT_BUILD_DISABLE_LIBCRYPTO_USE_FOR_ED25519_EVERYWHERE as follows: (Note: ED25519 keygen functions will start returning not supported error in this case)

sh AWS_CRT_BUILD_DISABLE_LIBCRYPTO_USE_FOR_ED25519_EVERYWHERE=1 python3 -m pip install --no-binary :all: --verbose awscrt

( --no-binary :all: ensures you do not use the precompiled wheel from PyPI)

AWSCRTBUILDUSESYSTEM_LIBS

aws-crt-python depends on several C libraries that make up the AWS Common Runtime (libaws-c-common, libaws-c-s3, etc). By default, these libraries are built along with aws-crt-python and statically compiled in (their source code is under crt/).

To skip building these dependencies, because they're already available on your system, set environment variable AWS_CRT_BUILD_USE_SYSTEM_LIBS=1 while building from source:

sh AWS_CRT_BUILD_USE_SYSTEM_LIBS=1 python3 -m pip install .

If these dependencies are available as both static and shared libs, you can force the static ones to be used by setting: AWS_CRT_BUILD_FORCE_STATIC_LIBS=1

Windows SDK Version

aws-crt-python builds against windows sdk version 10.0.17763.0. This is the minimal version required for TLS 1.3 support on Windows. If you need a different Windows SDK version, you can set environment variable AWS_CRT_WINDOWS_SDK_VERSION=<version> while building from source:

Owner

  • Name: Amazon Web Services - Labs
  • Login: awslabs
  • Kind: organization
  • Location: Seattle, WA

AWS Labs

Committers

Last synced: about 3 years ago

All Time
  • Total Commits: 369
  • Total Committers: 27
  • Avg Commits per committer: 13.667
  • Development Distribution Score (DDS): 0.631
Past Year
  • Commits: 71
  • Committers: 13
  • Avg Commits per committer: 5.462
  • Development Distribution Score (DDS): 0.592
Top Committers
Name Email Commits
Michael Graeb g****m@a****m 136
Jonathan M. Henson h****o@a****m 43
Justin Boswell b****j@a****m 36
Bret Ambrose b****e@g****m 26
Dengke Tang 8****5@q****m 23
Colden Cullen c****n@c****m 16
Jonathan M. Henson j****n@g****m 12
Dengke Tang d****t@a****m 10
Justin Boswell j****l@g****m 9
TwistedTwigleg n****d@a****m 9
Steve Kim 8****K@u****m 8
Ryan Carper 5****r@u****m 8
James Siri j****i@a****m 7
David Oguns 3****S@u****m 4
ilevyor 8****r@u****m 4
Waqar Ahmed Khan w****7@g****m 3
xiazhvera z****a@a****m 3
Richard Elberger r****h@r****m 2
Nikola Forró n****o@r****m 2
Joel Knight k****l@g****m 1
Joel Knight k****l@a****m 1
Colden Cullen c****n@a****m 1
Dmitriy Musatkin 6****n@u****m 1
Andrew Tang t****1@1****m 1
Alex Weibel a****1@g****m 1
Joseph Klix j****x@g****m 1
TwistedTwigleg t****g@g****m 1
Committer Domains (Top 20 + Academic)
amazon.com: 9 126.com: 1 redhat.com: 1 richelberger.com: 1 coldencullen.com: 1 qq.com: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 62
  • Total pull requests: 348
  • Average time to close issues: 11 months
  • Average time to close pull requests: about 1 month
  • Total issue authors: 59
  • Total pull request authors: 29
  • Average comments per issue: 3.24
  • Average comments per pull request: 0.2
  • Merged pull requests: 274
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 10
  • Pull requests: 130
  • Average time to close issues: 3 days
  • Average time to close pull requests: 6 days
  • Issue authors: 10
  • Pull request authors: 15
  • Average comments per issue: 1.6
  • Average comments per pull request: 0.2
  • Merged pull requests: 108
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • major (2)
  • thomas-roos (2)
  • massi-ang (2)
  • nforro (1)
  • mayosuke (1)
  • DeboBurro (1)
  • darko-mesaros (1)
  • benkehoe (1)
  • liugongqx (1)
  • tipu (1)
  • Lewenhaupt (1)
  • sunpoet (1)
  • tollsimy (1)
  • papoteur-mga (1)
  • imgguti (1)
Pull Request Authors
  • graebm (63)
  • TingDaoK (53)
  • waahm7 (51)
  • sbSteveK (33)
  • bretambrose (32)
  • xiazhvera (22)
  • DmitriyMusatkin (19)
  • sfod (17)
  • TwistedTwigleg (10)
  • nforro (7)
  • alfred2g (6)
  • JonathanHenson (5)
  • jmklix (4)
  • alexw91 (3)
  • jonathan343 (2)
Top Labels
Issue Labels
feature-request (17) response-requested (15) bug (13) needs-triage (13) closing-soon (12) CRT/IoT (9) p3 (8) p2 (8) CRT/SDK (4) closed-for-staleness (3) guidance (1) pending-release (1) documentation (1) blocked (1) needs-reproduction (1)
Pull Request Labels
needs-review (10) response-requested (2) pr/needs-review (2)

Packages

  • Total packages: 15
  • Total downloads:
    • pypi 6,399,911 last-month
  • Total docker downloads: 205,299,862
  • Total dependent packages: 30
    (may contain duplicates)
  • Total dependent repositories: 77
    (may contain duplicates)
  • Total versions: 332
  • Total maintainers: 4
pypi.org: awscrt

A common runtime for AWS Python projects

  • Versions: 181
  • Dependent Packages: 22
  • Dependent Repositories: 77
  • Downloads: 6,399,911 Last month
  • Docker Downloads: 205,299,862
Rankings
Docker downloads count: 0.4%
Downloads: 0.4%
Dependent packages count: 0.6%
Dependent repos count: 1.7%
Average: 3.0%
Forks count: 6.6%
Stargazers count: 8.1%
Maintainers (1)
awscommonruntime
Last synced: 10 months ago
alpine-v3.18: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 12
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 9.9%
Forks count: 18.3%
Stargazers count: 21.1%
Maintainers (1)
alice@ayaya.dev
Last synced: 10 months ago
alpine-v3.18: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 12
  • Dependent Packages: 8
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 9.9%
Forks count: 18.3%
Stargazers count: 21.2%
Maintainers (1)
alice@ayaya.dev
Last synced: 10 months ago
alpine-edge: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 46
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Average: 14.3%
Dependent packages count: 14.6%
Forks count: 19.8%
Stargazers count: 22.8%
Maintainers (1)
achill@achill.org
Last synced: 11 months ago
alpine-edge: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 40
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 14.1%
Average: 14.5%
Forks count: 20.2%
Stargazers count: 23.7%
Maintainers (1)
achill@achill.org
Last synced: 10 months ago
conda-forge.org: awscrt
  • Versions: 29
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Forks count: 30.8%
Dependent repos count: 34.0%
Stargazers count: 36.2%
Average: 38.1%
Dependent packages count: 51.2%
Last synced: 10 months ago
anaconda.org: awscrt

Python 3 bindings for the AWS Common Runtime.

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 46.1%
Average: 48.3%
Dependent repos count: 50.6%
Last synced: 10 months ago
alpine-v3.20: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Last synced: 10 months ago
alpine-v3.22: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
fossdd@pwned.life
Last synced: 10 months ago
alpine-v3.22: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
fossdd@pwned.life
Last synced: 10 months ago
alpine-v3.21: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
fossdd@pwned.life
Last synced: 10 months ago
alpine-v3.20: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Last synced: 10 months ago
alpine-v3.19: py3-awscrt

Python bindings for the AWS Common Runtime

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Last synced: 10 months ago
alpine-v3.21: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Maintainers (1)
fossdd@pwned.life
Last synced: 10 months ago
alpine-v3.19: py3-awscrt-pyc

Precompiled Python bytecode for py3-awscrt

  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent repos count: 0.0%
Dependent packages count: 0.0%
Average: 100%
Last synced: 10 months ago

Dependencies

.github/workflows/ci.yml actions
  • actions/checkout v3 composite
  • awslabs/aws-crt-builder/.github/actions/check-submodules main composite
  • cross-platform-actions/action v0.10.0 composite
  • vmactions/freebsd-vm v0 composite
.github/workflows/closed-issue-message.yml actions
  • aws-actions/closed-issue-message v1 composite
.github/workflows/docs.yml actions
  • actions/checkout v3 composite
  • ad-m/github-push-action v0.6.0 composite
.github/workflows/handle-stale-discussions.yml actions
  • aws-github-ops/handle-stale-discussions v1 composite
.github/workflows/lint.yml actions
  • DoozyX/clang-format-lint-action v0.3.1 composite
  • actions/checkout v2 composite
.github/workflows/stale_issue.yml actions
  • aws-actions/stale-issue-cleanup v3 composite
requirements-dev.txt pypi
  • autopep8 * development
  • setuptools * development
  • sphinx * development
  • websockets * development
  • wheel * development
setup.py pypi