https://github.com/awslabs/amazon-kinesis-client-python - Release 3.1.0 of the Amazon Kinesis Client for Python

Release 3.1.0 (July 10, 2025)

• KCL 3.1.1 Changelog Upgrade KCL and KCL-Multilang dependencies from 3.0.3 to 3.1.1
• #301 Add multi-lang support for leaseAssignmentIntervalMillis

- Python
Published by ethkatnic 12 months ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 3.0.5 of the Amazon Kinesis Client for Python

Release 3.0.5 (June 6, 2025)

• #274 Add ability to set mvn repo search URL with a unique ENV Variable
• #279 Create sample KCL build tests
• #280 Add dependabot auto-merge and workflow dispatch
• #283 Update AWS credentials used for running sample workflow
• #287 Upgrade org.checkerframework:checker-qual from 2.5.2 to 3.49.4
• #285 Upgrade netty.version from 4.1.118.Final to 4.2.1.Final
• #284 Upgrade commons-beanutils:commons-beanutils from 1.9.4 to 1.11.0

- Python
Published by vincentvilo-aws about 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 3.0.3 of the Amazon Kinesis Client for Python

Release 3.0.3 (March 25, 2025)

• Downgrade logback from 1.5.16 to 1.3.15 to maintain JDK 8 compatibility

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.2.1 of the Amazon Kinesis Client for Python

Release 2.2.1 (March 22, 2025)

• Downgrade logback from 1.5.16 to 1.3.15 to maintain JDK 8 compatibility

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 3.0.2 of the Amazon Kinesis Client for Python

Release 3.0.2 (March 24, 2025)

:warning: [BREAKING CHANGES] - Release 3.0.2 contains a dependency version that is not compatible with JDK 8. Please upgrade to a later version if your KCL application requires JDK 8.

• KCL 3.0.2 Changelog Upgrade KCL and KCL-Multilang dependencies from 3.0.0 to 3.0.2
• #266 Upgrade netty.version from 4.1.108.Final to 4.1.118.Final
• #265 Upgrade logback.version from 1.3.14 to 1.5.16

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.2.0 of the Amazon Kinesis Client for Python

🚨Important: Migration to Python KCL 2.2.0 or later with MultiLangDaemon - Credential Provider Changes Required

Java KCL version 2.7.0 and later uses AWS SDK for Java 2.x instead of AWS SDK for Java 1.x. For the KCL Python 2.x versions, v2.2.0 is the first Python release to use Java KCL 2.7.0. All MultiLangDaemon users upgrading from earlier versions must update their credential provider configuration in the .properties file to use credentials provider name for AWS SDK for Java 2.x. Failure to do this will cause your multilang KCL application to fail during startup with credential provider construction errors. Please check the following link for the credentials provider mapping and MultiLangDaemon credentials provider configuration guide

Release 2.2.0 (2025-03-12)

• #1444 Fully remove dependency on the AWS SDK for Java 1.x which will reach end-of-support by December 31st, 2025.
  • The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
  • Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.
• #270 Upgrade logback.version from 1.3.14 to 1.5.16
• #270 Upgrade netty.version from 4.1.108.Final to 4.1.118.Final

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.6 of the Amazon Kinesis Client for Python

Release 2.1.6 (January 22, 2025)

• Updates KCL version to use KCL 2.6.1. Brings in changes from the KCL 2.6.1 release.

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 3.0.1 of the Amazon Kinesis Client for Python

Release 3.0.1 (November 6, 2024)

• New lease assignment / load balancing algorithm
  • KCL 3.x introduces a new lease assignment and load balancing algorithm. It assigns leases among workers based on worker utilization metrics and throughput on each lease, replacing the previous lease count-based lease assignment algorithm.
  • When KCL detects higher variance in CPU utilization among workers, it proactively reassigns leases from over-utilized workers to under-utilized workers for even load balancing. This ensures even CPU utilization across workers and removes the need to over-provision the stream processing compute hosts.
• Optimized DynamoDB RCU usage
  • KCL 3.x optimizes DynamoDB read capacity unit (RCU) usage on the lease table by implementing a global secondary index with leaseOwner as the partition key. This index mirrors the leaseKey attribute from the base lease table, allowing workers to efficiently discover their assigned leases by querying the index instead of scanning the entire table.
  • This approach significantly reduces read operations compared to earlier KCL versions, where workers performed full table scans, resulting in higher RCU consumption.
• Graceful lease handoff
  • KCL 3.x introduces a feature called "graceful lease handoff" to minimize data reprocessing during lease reassignments. Graceful lease handoff allows the current worker to complete checkpointing of processed records before transferring the lease to another worker. For graceful lease handoff, you should implement checkpointing logic within the existing shutdownRequested() method.
  • This feature is enabled by default in KCL 3.x, but you can turn off this feature by adjusting the configuration property isGracefulLeaseHandoffEnabled.
  • While this approach significantly reduces the probability of data reprocessing during lease transfers, it doesn't completely eliminate the possibility. To maintain data integrity and consistency, it's crucial to design your downstream consumer applications to be idempotent. This ensures that the application can handle potential duplicate record processing without adverse effects.
• New DynamoDB metadata management artifacts
  • KCL 3.x introduces two new DynamoDB tables for improved lease management:
    • Worker metrics table: Records CPU utilization metrics from each worker. KCL uses these metrics for optimal lease assignments, balancing resource utilization across workers. If CPU utilization metric is not available, KCL assigns leases to balance the total sum of shard throughput per worker instead.
    • Coordinator state table: Stores internal state information for workers. Used to coordinate in-place migration from KCL 2.x to KCL 3.x and leader election among workers.
  • Follow this documentation to add required IAM permissions for your KCL application.
• Other improvements and changes
  • Dependency on the AWS SDK for Java 1.x has been fully removed.
    • The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
    • Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.
  • idleTimeBetweenReadsInMillis (PollingConfig) now has a minimum default value of 200.
    • This polling configuration property determines the publishers wait time between GetRecords calls in both success and failure cases. Previously, setting this value below 200 caused unnecessary throttling. This is because Amazon Kinesis Data Streams supports up to five read transactions per second per shard for shared-throughput consumers.
  • Shard lifecycle management is improved to deal with edge cases around shard splits and merges to ensure records continue being processed as expected.
• Migration
  • The programming interfaces of KCL 3.x remain identical with KCL 2.x for an easier migration. For detailed migration instructions, please refer to the Migrate consumers from KCL 2.x to KCL 3.x page in the Amazon Kinesis Data Streams developer guide.
• Configuration properties
  • New configuration properties introduced in KCL 3.x are listed in this doc.
  • Deprecated configuration properties in KCL 3.x are listed in this doc. You need to keep the deprecated configuration properties during the migration from any previous KCL version to KCL 3.x.
• Metrics
  • New CloudWatch metrics introduced in KCL 3.x are explained in the Monitor the Kinesis Client Library with Amazon CloudWatch in the Amazon Kinesis Data Streams developer guide. The following operations are newly added in KCL 3.x:
    • LeaseAssignmentManager
    • WorkerMetricStatsReporter
    • LeaseDiscovery ### Release 3.0.0 (November 6, 2024)

We found an issue with the release 3.0.0 regarding the build failure. Please use the release 3.0.1 to use KCL 3.0.

- Python
Published by ethkatnic over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.5 of the Amazon Kinesis Client for Python

Release 2.1.5 (May 29, 2024)

• Fixed CI due to different macOS architecture PR #246
• Added necessary Java SDKs to run sample PR #248
• Upgraded boto dependency to boto3 PR #245
• Upgraded AWS SDK from 2.19.2 to 2.25.11 PR #248
• Upgraded aws-java-sdk from 1.12.370 to 1.12.668 PR #248

- Python
Published by vincentvilo-aws about 2 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.4 of the Amazon Kinesis Client for Python

Release 2.1.4 (April 23, 2024)

• Upgraded KCL and KCL-Multilang dependencies from 2.5.2 to 2.5.8 PR #239
• Upgraded ion-java from 1.5.1 to 1.11.4 PR #243
• Upgraded logback version from 1.3.0 to 1.3.12 PR #242
• Upgraded io.netty dependency from 4.1.86.Final to 4.1.94.Final PR #234
• Upgraded Google Guava dependency from 32.0.0-jre to 32.1.1-jre PR #234
• Upgraded jackson-databind from 2.13.4 to 2.13.5 PR #234
• Upgraded protobuf-java from 3.21.5 to 3.21.7 PR #234

- Python
Published by vincentvilo-aws about 2 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.3 of the Amazon Kinesis Client for Python

• Added the ability to specify STS endpoint and region PR #221
• Upgraded KCL and KCL-Multilang Dependencies from 2.5.1 to 2.5.2 PR #221

- Python
Published by brendan-p-lynch almost 3 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.2 of the Amazon Kinesis Client for Python (June 29th, 2023)

• Added the ability to pass in streamArn to multilang Daemon PR #221
• Upgraded KCL and KCL-Multilang Dependencies from 2.4.4 to 2.5.1 PR #221
• Upgraded Google Guava dependency from 31.0.1-jre to 32.0.0-jre PR #223
• Added aws-java-sdk-sts dependency PR #212

- Python
Published by pelaezryan about 3 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.1 of the Amazon Kinesis Client for Python

What's Changed

• Include the pom file in MANIFEST by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client-python/pull/206

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-python/compare/v2.1.0...v2.1.1

- Python
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.1.0 of the Amazon Kinesis Client for Python

What's Changed

• Introduce dependabot with mock pom by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client-python/pull/154
• Upgrade Java dependencies to match KCL release v2.4.4 by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client-python/pull/201

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-python/compare/v2.0.6...v2.1.0

- Python
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.6 of the Amazon Kinesis Client for Python

Release 2.0.6 (November 23, 2021)

• Upgraded multiple dependencies PR #152
  • Amazon Kinesis Client Library 2.3.9
  • ch.qos.logback 1.2.7

- Python
Published by avahuang0429 over 4 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.5 of the Amazon Kinesis Client for Python

Release 2.0.5 (November 11, 2021)

• Upgraded multiple dependencies PR #148
  • Amazon Kinesis Client Library 2.3.8
  • AWS SDK 2.17.52
• Added dependencies
  • AWS SDK json-utils 2.17.52
  • third-party-jackson-core 2.17.52
  • third-party-jackson-dataformat-cbor 2.17.52
• Updated samples/sample.properties reflecting support for InitialPositionInStreamExtended
  • Related: #804 Allowing user to specify an initial timestamp in which daemon will process records.
  • Feature released with previous release 2.0.4

- Python
Published by avahuang0429 over 4 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.4: Fix breaking release 2.0.3

Release 2.0.4 (October 26, 2021)

• Revert/downgrade multiple dependencies as KCL 2.3.7 contains breaking change PR #145
  • Amazon Kinesis Client Library 2.3.6
  • AWS SDK 2.16.98
• Upgraded dependencies
  • jackson-dataformat-cbor 2.12.4
  • AWS SDK 1.12.3

- Python
Published by avahuang0429 over 4 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.2 of the Amazon Kinesis Client for Python

Release 2.0.2 (June 4, 2021)

• Upgraded multiple dependencies in PR #137
  • Amazon Kinesis Client Library 2.3.4
  • AWS SDK 2.16.75
  • AWS Java SDK 1.11.1031
  • Amazon ion java 1.5.1
  • Jackson 2.12.3
  • io.netty 4.1.65.Final
  • typeface netty 2.0.5
  • reactivestreams 1.0.3
  • guava 30.1.1-jre
  • Error prone annotations 2.7.1
  • j2objc annotations 2.7.1
  • Animal sniffer annotations 1.20
  • slf4j 1.7.30
  • protobuf 3.17.1
  • Joda time 2.10.10
  • Apache httpclient 4.5.13
  • Apache httpcore 4.4.14
  • commons lang3 3.12.0
  • commons logging 1.2
  • commons beanutils 1.9.4
  • commons codec 1.15
  • commons collections4 4.4
  • commons io 2.9.0
  • jcommander 1.81
  • rxjava 2.2.21
• Added Amazon Glue schema registry 1.0.2

- Python
Published by AvinashChowdary about 5 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.1 of the Amazon Kinesis Client for Python

Release 2.0.1 (February 27, 2019)

• Updated to version 2.1.2 of the Amazon Kinesis Client Library for Java.
  This update also includes version 2.4.0 of the AWS Java SDK.
  • PR #92

- Python
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 2.0.0 of the Amazon Kinesis Client for Python

Release 2.0.0 (January 15, 2019)

• Introducing support for Enhanced Fan-Out
• Updated to version 2.1.0 of the Amazon Kinesis Client for Java
  • Version 2.1.0 now defaults to using RegisterStreamConsumer Kinesis API, which provides dedicated throughput compared to GetRecords.
  • Version 2.1.0 now defaults to using SubscribeToShard Kinesis API, which provides lower latencies than GetRecords.
  • WARNING: RegisterStreamConsumer and SubscribeToShard are new APIs, and may require updating any explicit IAM policies
  • For more information about Enhaced Fan-Out and Polling with the KCL check out the announcement and developer documentation.
• Introducing version 3 of the RecordProcessorBase which supports the new ShardRecordProcessor interface
  • The shutdown method from version 2 has been removed and replaced by leaseLost and shardEnded methods.
  • Introducing leaseLost method, which takes LeaseLostInput object and is invoked when a lease is lost.
  • Introducing shardEnded method, which takes ShardEndedInput object and is invoked when all records from a split/merge have been processed.
• Updated AWS SDK version to 2.2.0
• MultiLangDaemon now uses logging using logback
  • MultiLangDaemon supports custom logback.xml file via the --log-configuration option.
  • amazon_kclpy_helper script supports --log-configuration option for command generation.

- Python
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.5.1 of the Amazon Kinesis Client for Python

Release 1.5.1 (January 2, 2019)

• Updated to version 1.9.3 of the Amazon Kinesis Client Library for Java.
  • PR #87
• Changed to now download jars from Maven using https.
  • PR #87
• Changed to raise exception when downloading from Maven fails.
  • PR #80

- Python
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.5.0 of the Amazon Kinesis Client for Python

Release 1.5.0 (February 7, 2018)

• Updated to version 1.9.0 of the Amazon Kinesis Client Library for Java
  • Version 1.9.0 now uses the ListShards Kinesis API, which provides a higher call rate than DescribeStream.
  • WARNING: ListShards is a new API, and may require updating any explicit IAM policies
  • PR #71

- Python
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.5 of the Amazon Kinesis Client for Python

Release 1.4.5 (June 28, 2017)

• Record processors can now be notified, and given a final opportunity to checkpoint, when the KCL is being shutdown.
  • PR #53
  • PR #56
  • PR #57

To use this feature the record processor must implement the shutdown_requested operation from the respective processor module. See v2/processor.py or kcl.py for the required API.

- Python
Published by pfifer about 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.4 of the Amazon Kinesis Client Library for Python

Release 1.4.4 (April 7, 2017)

• PR #47: Update to release 1.7.5 of the Amazon Kinesis Client.
  • Additionally updated to version 1.11.115 of the AWS Java SDK.
  • Fixes Issue #43.
  • Fixes Issue #27.

- Python
Published by pfifer about 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.3 of the Amazon Kinesis Client Library for Python

Release 1.4.3 (January 3, 2017)

• PR #39: Make record objects subscriptable for backwards compatibility.

- Python
Published by pfifer over 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.2 of the Amazon Kinesis Client for Python

Release 1.4.2 (November 21, 2016)

• PR #35: Downloading JAR files now runs correctly.

- Python
Published by pfifer over 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.1 of the Amazon Kinesis Client for Python

Release 1.4.1 (November 18, 2016)

• Installation of the library into a virtual environment on macOS, and Windows now correctly downloads the jar files.
  • Fixes Issue #33

- Python
Published by pfifer over 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Release 1.4.0 of the Amazon Kinesis Client for Python

Release 1.4.0 (November 9, 2016)

• Added a new v2 record processor class that allows access to updated features.
  • Record processor initialization
  • The initialize method receives an InitializeInput object that provides shard id, and the starting sequence and sub sequence numbers.
  • Process records calls
  • The process_records calls now receives a ProcessRecordsInput object that, in addition to the records, now includes the millisBehindLatest for the batch of records
  • Records are now represented as a Record object that adds new data, and includes some convenience methods
    • Adds a binary_data method that handles the base 64 decode of the data.
    • Includes the sub sequence number of the record.
    • Includes the approximate arrival time stamp of the record.
  • Record processor shutdown
  • The method shutdown now receives a ShutdownInput object.
• Checkpoint methods now accept a sub sequence number in addition to the sequence number.

amazon_kclpy 1.4.0 is available on PyPI

- Python
Published by pfifer over 9 years ago

https://github.com/awslabs/amazon-kinesis-client-python - Version 1.3.1 of the Amazon Kinesis Client Library For Python

Release Notes

Release 1.3.1

• Version number increase to stay inline with PyPI.

Release 1.3.0

• Updated dependency to Amazon KCL version 1.6.4

- Python
Published by pfifer almost 10 years ago