Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/awslabs/amazon-kinesis-client-nodejs

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release v3.0.1 of the Amazon Kinesis Client for Node.js

Release (v3.0.1 - May 28, 2025)

  • #414 Bump commons-beanutils:commons-beanutils from 1.9.4 to 1.11.0
  • #413 Bump mocha from 10.4.0 to 11.5.0
  • #410 Bump commander from 12.0.0 to 14.0.0
  • #403 Bump io.netty:netty-handler from 4.1.115.Final to 4.1.118.Final
  • #384 Bump ch.qos.logback:logback-core from 1.3.14 to 1.3.15
  • #368 Bump io.netty:netty-common from 4.1.108.Final to 4.1.115.Final
  • #367 Bump braces from 3.0.2 to 3.0.3

- JavaScript
Published by pelaezryan about 1 year ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.3.1 of the Amazon Kinesis Client for Node.js

Release 2.3.1 (March 25, 2025)

  • Downgrade logback from 1.5.16 to 1.3.15 to maintain JDK 8 compatability

- JavaScript
Published by pelaezryan over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.3.0

🚨Important: Migration to Node KCL 2.3.0 (or later) with MultiLangDaemon - Credential Provider Changes Required

Java KCL version 2.7.0 and later uses AWS SDK for Java 2.x instead of AWS SDK for Java 1.x. For the KCL Node 2.x versions, v2.3.0 is the first node release to use Java KCL 2.7.0. All MultiLangDaemon users upgrading from earlier versions must update their credential provider configuration in the .properties file to use credentials provider name for AWS SDK for Java 2.x. Failure to do this will cause your multilang KCL application to fail during startup with credential provider construction errors. Please check the following link for the credentials provider mapping and MultiLangDaemon credentials provider configuration guide.

:warning: [BREAKING CHANGES] - Release 2.3.0 contains a dependency version that is not compatible with JDK 8. Please upgrade to a later version if your KCL application requires JDK 8.

Release 2.3.0 (March 12, 2025)

- JavaScript
Published by pelaezryan over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.7

- JavaScript
Published by pelaezryan over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 3.0.0 of the Amazon Kinesis Client for Node.js

Release Notes

Release 3.0.0 (November 6, 2024)

  • New lease assignment / load balancing algorithm
    • KCL 3.x introduces a new lease assignment and load balancing algorithm. It assigns leases among workers based on worker utilization metrics and throughput on each lease, replacing the previous lease count-based lease assignment algorithm.
    • When KCL detects higher variance in CPU utilization among workers, it proactively reassigns leases from over-utilized workers to under-utilized workers for even load balancing. This ensures even CPU utilization across workers and removes the need to over-provision the stream processing compute hosts.
  • Optimized DynamoDB RCU usage
    • KCL 3.x optimizes DynamoDB read capacity unit (RCU) usage on the lease table by implementing a global secondary index with leaseOwner as the partition key. This index mirrors the leaseKey attribute from the base lease table, allowing workers to efficiently discover their assigned leases by querying the index instead of scanning the entire table.
    • This approach significantly reduces read operations compared to earlier KCL versions, where workers performed full table scans, resulting in higher RCU consumption.
  • Graceful lease handoff
    • KCL 3.x introduces a feature called "graceful lease handoff" to minimize data reprocessing during lease reassignments. Graceful lease handoff allows the current worker to complete checkpointing of processed records before transferring the lease to another worker. For graceful lease handoff, you should implement checkpointing logic within the existing shutdownRequested() method.
    • This feature is enabled by default in KCL 3.x, but you can turn off this feature by adjusting the configuration property isGracefulLeaseHandoffEnabled.
    • While this approach significantly reduces the probability of data reprocessing during lease transfers, it doesn't completely eliminate the possibility. To maintain data integrity and consistency, it's crucial to design your downstream consumer applications to be idempotent. This ensures that the application can handle potential duplicate record processing without adverse effects.
  • New DynamoDB metadata management artifacts
    • KCL 3.x introduces two new DynamoDB tables for improved lease management:
      • Worker metrics table: Records CPU utilization metrics from each worker. KCL uses these metrics for optimal lease assignments, balancing resource utilization across workers. If CPU utilization metric is not available, KCL assigns leases to balance the total sum of shard throughput per worker instead.
      • Coordinator state table: Stores internal state information for workers. Used to coordinate in-place migration from KCL 2.x to KCL 3.x and leader election among workers.
    • Follow this documentation to add required IAM permissions for your KCL application.
  • Other improvements and changes
    • Dependency on the AWS SDK for Java 1.x has been fully removed.
      • The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
      • Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.
    • idleTimeBetweenReadsInMillis (PollingConfig) now has a minimum default value of 200.
      • This polling configuration property determines the publishers wait time between GetRecords calls in both success and failure cases. Previously, setting this value below 200 caused unnecessary throttling. This is because Amazon Kinesis Data Streams supports up to five read transactions per second per shard for shared-throughput consumers.
    • Shard lifecycle management is improved to deal with edge cases around shard splits and merges to ensure records continue being processed as expected.
  • Migration
    • The programming interfaces of KCL 3.x remain identical with KCL 2.x for an easier migration. For detailed migration instructions, please refer to the Migrate consumers from KCL 2.x to KCL 3.x page in the Amazon Kinesis Data Streams developer guide.
  • Configuration properties
    • New configuration properties introduced in KCL 3.x are listed in this doc.
    • Deprecated configuration properties in KCL 3.x are listed in this doc. You need to keep the deprecated configuration properties during the migration from any previous KCL version to KCL 3.x.
  • Metrics
    • New CloudWatch metrics introduced in KCL 3.x are explained in the Monitor the Kinesis Client Library with Amazon CloudWatch in the Amazon Kinesis Data Streams developer guide. The following operations are newly added in KCL 3.x:
      • LeaseAssignmentManager
      • WorkerMetricStatsReporter
      • LeaseDiscovery

- JavaScript
Published by dominic-aws over 1 year ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.6 of the Amazon Kinesis Client for Node.js

Release 2.2.6 (April 25, 2024)

  • PR #327 Upgraded amazon-kinesis-client from 2.5.5 to 2.5.8
  • PR #329 Upgraded aws-sdk from 2.1562.0 to 2.1603.0
  • PR #95 Upgraded jcommander from 1.72 to 1.82
  • PR #271 Upgraded org.codehaus.mojo:animal-sniffer-annotations from 1.20 to 1.23
  • PR #266 Upgraded com.amazonaws:aws-java-sdk-core from 1.12.370 to 1.12.512
  • PR #313 Upgraded logback.version from 1.3.12 to 1.5.3
  • PR #305 Upgraded org.slf4j:slf4j-api from 2.0.5 to 2.0.12
  • PR #325 Upgraded mocha from 9.2.2 to 10.4.0
  • PR #307 Upgraded com.google.protobuf:protobuf-java from 3.21.7 to 3.25.3
  • PR #262 Upgraded checker-qual from 2.5.2 to 3.36.0
  • PR #303 Upgraded commander from 8.3.0 to 12.0.0
  • PR #287 Upgraded sinon from 14.0.2 to 17.0.1
  • PR #318 Upgraded awssdk.version from 2.20.43 to 2.25.11
  • PR #319 Upgraded org.reactivestreams:reactive-streams from 1.0.3 to 1.0.4
  • PR #320 Upgraded netty-reactive.version from 2.0.6 to 2.0.12
  • PR #330 Upgraded io.netty:netty-codec-http from 4.1.100.Final to 4.1.108.Final
  • PR #331 Upgraded ion-java from 1.5.1 to 1.11.4
  • PR #211 Upgraded fasterxml-jackson.version from 2.13.4 to 2.14.1

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-nodejs/compare/v2.2.5...v2.2.6

- JavaScript
Published by lucienlu-aws about 2 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.5 of the Amazon Kinesis Client for Node.js

Release 2.2.5 (February 29, 2024)

  • PR #309 Updated amazon-kinesis-client and amazon-kinesis-client multilang from 2.5.4 to 2.5.5 and updated awssdk.version to match amazon-kinesis-client from 2.19.2 to 2.20.43

- JavaScript
Published by brendan-p-lynch over 2 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.4 of the Amazon Kinesis Client Library for Node.js

Release 2.2.4 (January 16, 2024)

  • PR #298 Added dependency on aws-sdk arns
  • PR #293 Updated logback-classic to 1.3.12

- JavaScript
Published by brendan-p-lynch over 2 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.3 of the Amazon Kinesis Client Library for Node.js

What's Changed

  • PR #291 Updated KCL and KCL multilang to the latest version 2.5.4
  • PR #284 Updated netty to 4.1.100.Final, fasterxml-jackson to 2.13.5, and guava to 32.1.1-jre
  • PR #277 Updated com.google.protobuf:protobuf-java from 3.21.5 to 3.21.7

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-nodejs/compare/v2.2.2...v2.2.3

- JavaScript
Published by brendan-p-lynch over 2 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.2 of the Amazon Kinesis Client Library for Node.js

What's Changed

  • https://github.com/awslabs/amazon-kinesis-client-nodejs/pull/207 Add endpoints-spi dependency

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-nodejs/compare/v2.2.1...v2.2.2

- JavaScript
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.1 of the Amazon Kinesis Client Library for Node.js

What's Changed

  • Keep Java dependencies in sync with the KCL V2.4.4 by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client-nodejs/pull/202
  • Prepare for release 2.2.1 by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client-nodejs/pull/206

Full Changelog: https://github.com/awslabs/amazon-kinesis-client-nodejs/compare/v2.2.0...v2.2.1

- JavaScript
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.2.0 of the Amazon Kinesis Client Library for Node.js

Release 2.2.0 (September 15, 2022)

  • PR #165 Update Java dependencies
    • KCL and KCL-multilang are updated to the latest version 2.4.13

- JavaScript
Published by zengyu714 almost 4 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.1.0 of the Amazon Kinesis Client Library for Node.js

Release 2.1.0 (January 31, 2020)

Milestone #4 * Fixing bootstrap to use HTTPS * PR #75 * Adding support for Win32 platform * PR #67 * Relicensing to Apache-2.0 * PR #69

- JavaScript
Published by joshua-kim over 6 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 2.0.0 of the Amazon Kinesis Client Library for Node.js

Release 2.0.0 (March 6, 2019)

  • Added support for Enhanced Fan-Out.
    Enhanced Fan-Out provides dedicated throughput per stream consumer, and uses an HTTP/2 push API (SubscribeToShard) to deliver records with lower latency.
  • Updated the Amazon Kinesis Client Library for Java to version 2.1.2.
  • Added support for the newer methods to the KCLManager.
    While the original shutdown method will continue to work it's recommended to upgrade to the newer interface.
    • The shutdown has been replaced by leaseLost and shardEnded.
    • Added the leaseLost method which is invoked when a lease is lost.
      leaseLost replaces shutdown where shutdownInput.reason was ZOMBIE.
    • Added the shardEnded method which is invoked when all records from a split or merge have been processed.
      shardEnded replaces shutdown where shutdownInput.reason was TERMINATE.
  • Updated the AWS Java SDK version to 2.4.0
  • MultiLangDaemon now provides logging using Logback.
    • MultiLangDaemon supports custom configurations for logging via a Logback XML configuration file.
    • The kcl-bootstrap program was been updated to accept either -l or --log-configuration to provide a Logback XML configuration file.

- JavaScript
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 0.8.0 of the Amazon Kinesis Client Library for Node.js

Release 0.8.0 (February 12, 2019)

  • Updated the dependency on [Amazon Kinesis Client for Java][amazon-kcl-github] to 1.9.3
    • This adds support for ListShards API. This API is used in place of DescribeStream API to provide more throughput during ShardSyncTask. Please consult the AWS Documentation for ListShards for more information.
    • ListShards supports higher call rate, which should reduce instances of throttling when attempting to synchronize the shard list.
    • WARNING: ListShards is a new API, and may require updating any explicit IAM policies
    • PR #59
  • Changed to now download jars from Maven using https.

- JavaScript
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 0.7.0 of the Amazon Kinesis Client Library for Node.js

Release 0.7.0 (August 2, 2017)

  • Updated the dependency on [Amazon Kinesis Client for Java][amazon-kcl-github] to 1.8.1.
    This adds support for setting a timeout when dispatching records to the node.js record processor. If the record processor doesn't respond in the given time the Java processor is terminated. The timeout for the this can be set by adding timeoutInSeconds = <timeout value>. The default for this is no timeout.
    Setting this can cause the KCL to exit suddenly, before using this ensure that you have an automated restart for your application
    Updating minimum requirement for the JDK version to 8
  • Added support to handle graceful shutdown requests.

- JavaScript
Published by pfifer almost 9 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - Release 0.6.0 of the Amazon Kinesis Client for Node.js

Release 0.6.0 (December 12, 2016)

  • Updated the dependency on [Amazon Kinesis Client for Java][amazon-kcl-github] to 1.7.2.
    • PR #23
    • PR #24

The release is available on npm as aws-kcl

- JavaScript
Published by pfifer over 9 years ago

https://github.com/awslabs/amazon-kinesis-client-nodejs - v0.5.0

Version 0.5.0 of the Amazon Kinesis Client Library for Node.js.

- JavaScript
Published by rutvikhora over 11 years ago