Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of https://github.com/awslabs/amazon-kinesis-client

https://github.com/awslabs/amazon-kinesis-client - Release 2.7.1 of the Amazon Kinesis Client Library for Java

Release 2.7.1 (2025-07-11)

  • #1508 Upgrade schema-registry-common and schema-registry-serde from 1.1.19 to 1.1.24

- Java
Published by lucienlu-aws 11 months ago

https://github.com/awslabs/amazon-kinesis-client - Release 3.1.1 of the Amazon Kinesis Client Library for Java

Release (3.1.1 - July 8, 2025)

  • #1495 Add millisBehindLatestThresholdForReducedTps property to PollingConfig
  • #1498 Add multilang support for leaseAssignmentIntervalMillis config
  • #1499 Add Lease Table ARN Hash as Consumer ID in UserAgent for Kinesis requests
  • #1493 Bump awssdk.version from 2.25.64 to 2.31.62
  • #1502 Bump io.reactivex.rxjava3:rxjava from 3.1.8 to 3.1.11
  • #1457 Bump org.projectlombok:lombok from 1.18.24 to 1.18.38
  • #1505 Bump schema-registry-common and schema-registry-serde from 1.1.19 to 1.1.24

- Java
Published by lucienlu-aws 11 months ago

https://github.com/awslabs/amazon-kinesis-client - Release 3.1.0 of the Amazon Kinesis Client Library for Java

Release (3.1.0 - June 09, 2025)

  • #1481 Convert awssdk object to lease early to release memory
  • #1480 Added the ability to control sleep time in recordsfetcher
  • #1479 Overriding the DataFetcher to return a custom GetRecordsResponseAdapter so that customers can have custom logic to send data to KinesisClientRecord
  • #1478 Bump commons-beanutils:commons-beanutils from 1.9.4 to 1.11.0 in /amazon-kinesis-client-multilang

- Java
Published by pelaezryan 12 months ago

https://github.com/awslabs/amazon-kinesis-client - Release v1.15.3 of the Amazon Kinesis Client Library for Java

Release (1.15.3 - May 27, 2025)

  • #1475 Upgraded protobuf dependency from 4.27.1 to 4.31.0

- Java
Published by pelaezryan about 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 3.0.3 of the Amazon Kinesis Client Library for Java

Release 3.0.3 (May 7, 2025)

  • #1464 Add config for LeaseAssignmentManager frequency and improve assignment time of newly created leases
  • #1463 Extend ShardConsumer constructor to have ConsumerTaskFactory as a parameter to support DynamoDB Streams Kinesis Adapter compatibility
  • #1472 Remove unused synchronized keyword

- Java
Published by lucienlu-aws about 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 3.0.2 of the Amazon Kinesis Client Library for Java

Release 3.0.2 (March 12, 2025)

  • #1443 Reduce DynamoDB IOPS for smaller stream and worker count applications
  • The below two PRs are intended to support DynamoDB Streams Kinesis Adapter compatibility
    • #1441 Make consumerTaskFactory overridable by customers
    • #1440 Make ShutdownTask, ProcessTask, InitializeTask, BlockOnParentTask, and ShutdownNotificationTask overridable by customers
  • #1437 Suppress LeaseAssignmentManager excessive WARN logs when there is no issue
  • #1439 Upgrade io.netty:netty-handler from 4.1.108.Final to 4.1.118.Final
  • #1400 Upgrade com.fasterxml.jackson.core:jackson-databind from 2.10.1 to 2.12.7.1

- Java
Published by lucienlu-aws about 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.7.0 of the Amazon Kinesis Client Library for Java

Release 2.7.0 (2025-03-11)

  • #1444 Fully remove dependency on the AWS SDK for Java 1.x which will reach end-of-support by December 31st, 2025.
    • The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
    • Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.

- Java
Published by vincentvilo-aws about 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.6.1 of the Amazon Kinesis Client Library for Java

Release 2.6.1 of the Amazon Kinesis Client Library for Java * #1365 Add config to enable point in time recovery for the DynamoDB lease table created by KCL * #1325 Add logic to sync from lease table on first app bootstrap * #1372 Fix debug and trace level logging to log all worker loops * #1319 Fix a race condition between ShardConsumer shutdown and initialization * #1354 Fix bug where the in-memory lease is updated even when DDB fails to update * #1363 Fix some of the warnings emitted by maven during build * #1358 Remove ShutdownNotificationAware and update javadocs * #1329 Remove unnecessary lambda * #1340 Generate wrappers from proto files instead of shipping them directly * #1332 Enforce consistent code style with palantir-java-format * #1294 Upgrade org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.0 to 3.3.1 * #1296 Upgrade org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.5 * #1312 Upgrade commons-io:commons-io from 2.15.1 to 2.16.1 * #1331 Upgrade maven-compiler-plugin from 3.11.0 to 3.13.0 * #1335 Upgrade org.slf4j:slf4j-api from 2.0.7 to 2.0.13 * #1336 Upgrade io.reactivex.rxjava3:rxjava from 3.1.6 to 3.1.8 * #1337 Upgrade com.google.protobuf:protobuf-java from 3.21.12 to 4.27.0 * #1341 Upgrade awssdk.version from 2.25.11 to 2.25.64 * #1342 Upgrade org.apache.maven.plugins:maven-javadoc-plugin from 3.6.3 to 3.7.0 * #1343 Upgrade org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.5 * #1383 Upgrade com.google.protobuf:protobuf-java from 4.27.0 to 4.27.5

- Java
Published by vincentvilo-aws over 1 year ago

https://github.com/awslabs/amazon-kinesis-client - v3.0.1

What's Changed

  • Update backward compatibility check script by @akidambisrinivasan in https://github.com/awslabs/amazon-kinesis-client/pull/1393
  • Update version to 3.0.1-SNAPSHOT by @akidambisrinivasan in https://github.com/awslabs/amazon-kinesis-client/pull/1394
  • Address KCLv3 issues from github by @akidambisrinivasan in https://github.com/awslabs/amazon-kinesis-client/pull/1398
  • Fix checkpointOwner copy issue for multistream lease by @chenylee-aws in https://github.com/awslabs/amazon-kinesis-client/pull/1401
  • Preparation for v3.0.1 by @chenylee-aws in https://github.com/awslabs/amazon-kinesis-client/pull/1402

Full Changelog: https://github.com/awslabs/amazon-kinesis-client/compare/v3.0.0...v3.0.1

- Java
Published by chenylee-aws over 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 3.0.0 of the Amazon Kinesis Client Library for Java

Changelog

For 1.x release notes, please see v1.x/CHANGELOG.md

For 2.x release notes, please see v2.x/CHANGELOG.md

Release 3.0.0 (November 06, 2024)

  • New lease assignment / load balancing algorithm
    • KCL 3.x introduces a new lease assignment and load balancing algorithm. It assigns leases among workers based on worker utilization metrics and throughput on each lease, replacing the previous lease count-based lease assignment algorithm.
    • When KCL detects higher variance in CPU utilization among workers, it proactively reassigns leases from over-utilized workers to under-utilized workers for even load balancing. This ensures even CPU utilization across workers and removes the need to over-provision the stream processing compute hosts.
  • Optimized DynamoDB RCU usage
    • KCL 3.x optimizes DynamoDB read capacity unit (RCU) usage on the lease table by implementing a global secondary index with leaseOwner as the partition key. This index mirrors the leaseKey attribute from the base lease table, allowing workers to efficiently discover their assigned leases by querying the index instead of scanning the entire table.
    • This approach significantly reduces read operations compared to earlier KCL versions, where workers performed full table scans, resulting in higher RCU consumption.
  • Graceful lease handoff
    • KCL 3.x introduces a feature called "graceful lease handoff" to minimize data reprocessing during lease reassignments. Graceful lease handoff allows the current worker to complete checkpointing of processed records before transferring the lease to another worker. For graceful lease handoff, you should implement checkpointing logic within the existing shutdownRequested() method.
    • This feature is enabled by default in KCL 3.x, but you can turn off this feature by adjusting the configuration property isGracefulLeaseHandoffEnabled.
    • While this approach significantly reduces the probability of data reprocessing during lease transfers, it doesn't completely eliminate the possibility. To maintain data integrity and consistency, it's crucial to design your downstream consumer applications to be idempotent. This ensures that the application can handle potential duplicate record processing without adverse effects.
  • New DynamoDB metadata management artifacts
    • KCL 3.x introduces two new DynamoDB tables for improved lease management:
      • Worker metrics table: Records CPU utilization metrics from each worker. KCL uses these metrics for optimal lease assignments, balancing resource utilization across workers. If CPU utilization metric is not available, KCL assigns leases to balance the total sum of shard throughput per worker instead.
      • Coordinator state table: Stores internal state information for workers. Used to coordinate in-place migration from KCL 2.x to KCL 3.x and leader election among workers.
    • Follow this documentation to add required IAM permissions for your KCL application.
  • Other improvements and changes
    • Dependency on the AWS SDK for Java 1.x has been fully removed.
      • The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
      • Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.
    • idleTimeBetweenReadsInMillis (PollingConfig) now has a minimum default value of 200.
      • This polling configuration property determines the publishers wait time between GetRecords calls in both success and failure cases. Previously, setting this value below 200 caused unnecessary throttling. This is because Amazon Kinesis Data Streams supports up to five read transactions per second per shard for shared-throughput consumers.
    • Shard lifecycle management is improved to deal with edge cases around shard splits and merges to ensure records continue being processed as expected.
  • Migration
    • The programming interfaces of KCL 3.x remain identical with KCL 2.x for an easier migration, with the exception of those applications that do not use the recommended approach of using the Config Builder. These applications will have to refer to the troubleshooting guide. For detailed migration instructions, please refer to the Migrate consumers from KCL 2.x to KCL 3.x page in the Amazon Kinesis Data Streams developer guide.
  • Configuration properties
    • New configuration properties introduced in KCL 3.x are listed in this doc.
    • Deprecated configuration properties in KCL 3.x are listed in this doc. You need to keep the deprecated configuration properties during the migration from any previous KCL version to KCL 3.x.
  • Metrics
    • New CloudWatch metrics introduced in KCL 3.x are explained in the Monitor the Kinesis Client Library with Amazon CloudWatch in the Amazon Kinesis Data Streams developer guide. The following operations are newly added in KCL 3.x:
      • LeaseAssignmentManager
      • WorkerMetricStatsReporter
      • LeaseDiscovery

- Java
Published by lucienlu-aws over 1 year ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.15.2 of the Amazon Kinesis Client Library for Java

Release (1.15.2 - Aug 14, 2024)

  • #1371 Fix a bug in debug and trace logging levels for worker
  • #1224 Modify RecordProcessorCheckpointer#advancePosition Metrics usage to ensure proper closure
  • #1345 Generate wrappers from proto files instead of shipping them directly
  • #1346 Upgrade com.google.protobuf:protobuf-java from 3.23.4 to 4.27.1
  • #1338 Upgrade org.apache.logging.log4j:log4j-api from 2.20.0 to 2.23.1
  • #1327 Upgrade com.google.guava:guava from 33.0.0-jre to 33.2.0-jre
  • #1283 Upgrade com.fasterxml.jackson.core:jackson-core from 2.15.2 to 2.17.0
  • #1284 Upgrade aws-java-sdk.version from 1.12.647 to 1.12.681
  • #1288 Upgrade commons-logging:commons-logging from 1.2 to 1.3.1
  • #1289 Upgrade org.projectlombok:lombok from 1.18.22 to 1.18.32
  • #1248 Upgrade org.apache.maven.plugins:maven-surefire-plugin from 2.22.2 to 3.2.5
  • #1234 Upgrade org.apache.maven.plugins:maven-javadoc-plugin from 3.4.1 to 3.6.3
  • #1137 Upgrade maven-failsafe-plugin from 2.22.2 to 3.1.2
  • #1134 Upgrade jackson-core from 2.15.0 to 2.15.2
  • #1119 Upgrade maven-source-plugin from 3.2.1 to 3.3.0
  • #1165 Upgrade protobuf-java from 3.19.6 to 3.23.4

- Java
Published by lucienlu-aws almost 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.6.0 of Amazon Kinesis Client Library for Java

Release 2.6.0 (2024-05-01)

  • #1317 Add enablePriorityLeaseAssignment config
  • #1320 Update lease taker to get unassigned leases
  • #1318 Internally construct and use stream ARNs for all streams in multi-stream mode
  • #1291 Update RetrievalFactory implementations to utilize the StreamIdentifier field of StreamConfig
  • #1308 Move shutdownComplete call to ShardConsumer
  • #1313 Add additional integration tests for multi-stream and cross account access
  • #1273 Optimize currentStreamConfigMap by cleaning up lingering streams
  • #1302 Fix gracefulShutdown behavior in Scheduler

- Java
Published by lucienlu-aws about 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.8 of Amazon Kinesis Client Library for Java

Release 2.5.8 (2024-03-27)

  • #1278 Upgrade awssdk.version from 2.25.3 to 2.25.11
  • #1279 Upgrade org.apache.maven.plugins:maven-gpg-plugin from 3.1.0 to 3.2.1
  • #1280 Upgrade org.apache.commons:commons-lang3 from 3.12.0 to 3.14.0
  • #1282 Upgrade org.apache.maven.plugins:maven-javadoc-plugin from 3.5.0 to 3.6.3
  • #1277 Reuse 'ShardSyncTaskManager' instance for existing stream to avoid duplicate enqueue of 'ShardSyncTask'

- Java
Published by brendan-p-lynch about 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.7 of Amazon Kinesis Client Library for Java

Changelog

For 1.x release notes, please see v1.x/CHANGELOG.md

Release 2.5.7 (2024-03-19)

  • #1275 Update PollingConfig maxRecords method to return PollingConfig

  • #1236 Upgrade commons-io:commons-io from 2.11.0 to 2.15.1

  • #1189 Upgrade org.apache.maven.plugins:maven-resources-plugin from 3.3.0 to 3.3.1

  • #1139 Upgrade maven-surefire-plugin from 2.22.2 to 3.1.2

  • #1138 Upgrade maven-failsafe-plugin from 2.22.2 to 3.1.2

  • #1125 Upgrade maven-gpg-plugin from 3.0.1 to 3.1.0

- Java
Published by brendan-p-lynch about 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release v2.5.6 of Amazon Kinesis Client Library for Java

Release 2.5.6 (2024-03-08)

  • #1271 Adding snapshot for 2.5.6-SNAPSHOT
  • #1268 Upgrade ch.qos.logback:logback-classic dependency from 1.3.12 to 1.3.14
  • #1268 Upgrade awssdk.version from 2.20.43 to 2.25.3
  • #1268 Upgrade aws-java-sdk.version from 1.12.405 to 1.12.668
  • #1268 Upgrade gsr.version from 1.1.17 to 1.1.19

- Java
Published by brendan-p-lynch about 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.5 of the Amazon Kinesis Client Library for Java

Release 2.5.5 (2024-02-26)

  • #1257 Prevent improper error logging during worker shutdown
  • #1260 Add Deletion protection config
  • #1258 Fix issue in configuring metricsEnabledDimensions
  • #1259 Add snapshot to version

- Java
Published by zachjhum over 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.15.1 of the Amazon Kinesis Client Library for Java

Release (1.15.1 - Feb 5, 2023)

  • #1214 Added backoff logic for ShardSyncTaskIntegrationTest
  • #1214 Upgrade Guava version from 31.0.1 to 32.1.1
  • #1252 Upgrade aws-java-sdk from 1.12.406 to 1.12.647

- Java
Published by vincentvilo-aws over 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.4 of the Amazon Kinesis Client Library for Java

Release 2.5.4 (December 12, 2023) https://github.com/awslabs/amazon-kinesis-client/pull/1232 Upgrade ch.qos.logback:logback-classic dependency from 1.3.0 to 1.3.12 in /amazon-kinesis-client https://github.com/awslabs/amazon-kinesis-client/pull/1233 Upgrade ch.qos.logback:logback-classic dependency from 1.3.0 to 1.3.12 in /amazon-kinesis-client-multilang https://github.com/awslabs/amazon-kinesis-client/pull/1230 Bug fix which now allows MultiLangDaemon to configure idleTimeBetweenReadsInMillis https://github.com/awslabs/amazon-kinesis-client/pull/1229 Added link to javadoc.io-hosted Javadoc in the README https://github.com/awslabs/amazon-kinesis-client/pull/1218 Added doc for leases and the lease lifecycle to help explain lease lifecycle logic. https://github.com/awslabs/amazon-kinesis-client/pull/1226 Upgraded KCL from 2.5.3 to 2.5.4-SNAPSHOT

- Java
Published by brendan-p-lynch over 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.3 of the Amazon Kinesis Client Library for Java

Release 2.5.3 (November 8, 2023)

  • #1219 Provided streamArn in getRecords request
  • #1216 Updated AWS Glue Schema Registry from version 1.1.14 to 1.1.17.
  • #1205 Updated the FAQ with impact of changing default checkpoint.
  • #1203 Added links from README.md to FAQ and doc folder.
  • #1202 Introduced a FAQ for Kinesis Client Library
  • #1200 Added test case for StreamIdentifier serialization.

- Java
Published by brendan-p-lynch over 2 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.2 of the Amazon Kinesis Client Library for Java

Release 2.5.2 (2023-08-07)

  • #1184 [#367] Enhanced multi-lang AWSCredentialsProvider=... decoder and c…
  • #1186 Provided documentation for multilang's new NestedPropertyKey enhancement.
  • #1181 CVE-2023-2976: Upgrade Google Guava dependency version from 32.0.0-jre to 32.1.1-jre
  • #1159 Bug fix in lease refresher integration test with occasional failures
  • #1157 Fix NPE on graceful shutdown before DDB LeaseCoordinator starts.
  • #1152 Adding resharding integration tests and changing ITs to not run by default
  • #1162 Only deleting resource created by ITs
  • #1158 Checkstyle: tightened LineLength restriction from 170 to 150.
  • #1151 Modified dependabot.yml to set the correct v[1|2].x label.
  • #1164 Upgraded KCL Version from 2.5.1 to 2.5.2-SNAPSHOT

- Java
Published by stair-aws almost 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.1 of the Amazon Kinesis Client Library for Java

  • #1143 Upgrade MultiLangDaemon to support StreamARN
  • #1145 Introduced GitHub actions to trigger Maven builds during merge/pull requests
  • #1136 Added testing architecture and KCL 2.x basic polling/streaming tests
  • #1153 Checkstyle: added UnusedImports check.
  • #1150 Enabled Checkstyle validation of test resources.
  • #1149 Bound Checkstyle to validate goal for automated enforcement.
  • #1148 Code cleanup to faciliate Checkstyle enforcement.
  • #1142 Upgrade Google Guava dependency version from 31.1-jre to 32.0.0-jre
  • #1115 Update KCL version from 2.5.0 to 2.5.1-SNAPSHOT

- Java
Published by pelaezryan almost 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.15.0 of the Amazon Kinesis Client Library for Java

Release (1.15.0 - Jun 8, 2023)

  • #1108 Add support for Stream ARNs
  • #1111 More consistent testing behavior with HashRangesAreAlwaysComplete
  • #1054 Upgrade log4j-core from 2.17.1 to 2.20.0
  • #1103 Upgrade jackson-core from 2.13.0 to 2.15.0
  • #943 Upgrade nexus-staging-maven-plugin from 1.6.8 to 1.6.13
  • #1044 Upgrade aws-java-sdk.version from 1.12.406 to 1.12.408
  • #1055 Upgrade maven-compiler-plugin from 3.10.0 to 3.11.0

- Java
Published by lucienlu-aws almost 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.5.0 of the Amazon Kinesis Client Library for Java

Release 2.5.0 (May 19, 2023)

  • #1109 Add support for stream ARNs
  • #1065 Allow tags to be added when lease table is created
  • #1094 Code cleanup to introduce better testing
  • #1088 Minimize race in PSSM to optimize shard sync calls
  • #1086 Add additional SingleStreamTracker constructor with stream position parameter
  • #1084 More consistent testing behavior with restartAfterRequestTimerExpires
  • #1066 More consistent testing behavior with HashRangesAreAlwaysComplete
  • #1072 Upgrade nexus-staging-maven-plugin from 1.6.8 to 1.6.13
  • #1073 Upgrade slf4j-api from 2.0.6 to 2.0.7
  • #1090 Upgrade awssdk.version from 2.20.8 to 2.20.43
  • #1071 Upgrade maven-compiler-plugin from 3.8.1 to 3.11.0

- Java
Published by lucienlu-aws about 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.8 of the Amazon Kinesis Client Library for Java

Release 2.4.8 (March 21, 2023) - https://github.com/awslabs/amazon-kinesis-client/pull/1080 Added metric in ShutdownTask for scenario when parent leases are missing. - https://github.com/awslabs/amazon-kinesis-client/pull/1077 Reverted changes to pom property - https://github.com/awslabs/amazon-kinesis-client/pull/1069 Fixed flaky InitializationWaitsWhenLeaseTableIsEmpty test

- Java
Published by noahbt about 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.7 of the Amazon Kinesis Client Library for Java

Release 2.4.7 (March 20, 2023)

  • https://github.com/awslabs/amazon-kinesis-client/pull/1063 Allow leader to learn new leases upon re-election to avoid unnecessary shardSyncs
  • https://github.com/awslabs/amazon-kinesis-client/pull/1060 Add new metric to be emitted on lease creation
  • https://github.com/awslabs/amazon-kinesis-client/pull/1057 Added more logging in Scheduler w.r.t. StreamConfigs.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1059 DRY: simplification of HierarchicalShardSyncerTest.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1062 Fixed retry storm in PrefetchRecordsPublisher.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1061 Fixed NPE in LeaseCleanupManager.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1069 Fixed flaky InitializationWaitsWhenLeaseTableIsEmpty test
  • https://github.com/awslabs/amazon-kinesis-client/pull/1056 Clean up in-memory state of deleted kinesis stream in MultiStreamMode
  • https://github.com/awslabs/amazon-kinesis-client/pull/1058 Documentation: added <pre> tags so fixed-format diagrams aren't garbled.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1053 Exposed convenience method of ExtendedSequenceNumber#isSentinelCheckpoint()
  • https://github.com/awslabs/amazon-kinesis-client/pull/1077 Reverted changes to pom property
  • https://github.com/awslabs/amazon-kinesis-client/pull/1043 Removed a .swp file, and updated .gitignore.
  • https://github.com/awslabs/amazon-kinesis-client/pull/1047 Upgrade awssdk.version from 2.19.31 to 2.20.8
  • https://github.com/awslabs/amazon-kinesis-client/pull/1046 Upgrade maven-javadoc-plugin from 3.3.1 to 3.5.0
  • https://github.com/awslabs/amazon-kinesis-client/pull/1038 Upgrade gsr.version from 1.1.13 to 1.1.14
  • https://github.com/awslabs/amazon-kinesis-client/pull/1037 Upgrade aws-java-sdk.version from 1.12.370 to 1.12.405

- Java
Published by noahbt about 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.10 of the Amazon Kinesis Client Library for Java

  • Updated aws-java-sdk from 1.12.130 to 1.12.406
  • Updated com.google.protobuf from 3.19.4 to 3.19.6

- Java
Published by ZeyuLi-AWS over 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.5 of the Amazon Kinesis Client Library for Java

#1014 Use AFTERSEQUENCENUMBER iterator type for expired iterator request

- Java
Published by chenylee-aws over 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.4 of the Amazon Kinesis Client Library for Java

What's Changed

  • Upgrade aws sdk to the latest version by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client/pull/1017
  • Correct the KCL version in the main pom by @zengyu714 in https://github.com/awslabs/amazon-kinesis-client/pull/1020

Full Changelog: https://github.com/awslabs/amazon-kinesis-client/compare/v2.4.3...v2.4.4

- Java
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.9 of the Amazon Kinesis Client Library for Java

  • #995 Every other change for DynamoDBStreamsKinesis Adapter Compatibility
  • #970 PeriodicShardSyncManager Changes Needed for DynamoDBStreamsKinesisAdapter

- Java
Published by kdsjasonwng over 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.3 of the Amazon Kinesis Client Library for Java

What's Changed

  • #980 logback-classic: 1.2.9 -> 1.4.0
  • #983
    • protobuf-java: 3.19.2 -> 3.21.5
    • slf4j.version: 1.7.32 -> 2.0.0
    • schema-registry-serde: 1.1.9 -> 1.1.13
  • #984 awssdk.version from 2.17.108 to 2.17.267
  • #987 guava: 31.0.1-jre -> 31.1-jre
  • #988 jcommander: 1.81 to 1.82
  • #990 Upgrade dependencies
    • aws-java-sdk.version: 1.12.130 -> 1.12.296
    • lombok: 1.18.22 -> 1.18.24
    • rxjava: 3.1.3 -> 3.1.5
    • maven-resources-plugin: 2.6 -> 3.3.0
    • logback-classic: 1.4.0 -> 1.3.0
    • awssdk.version: 2.17.267 -> 2.17.268

Full Changelog: https://github.com/awslabs/amazon-kinesis-client/compare/v2.4.2...v2.4.3

- Java
Published by zengyu714 over 3 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.4.2 of the Amazon Kinesis Client Library for Java

  • #972 Upgrade Lombok to version 1.18.24

- Java
Published by kdsjasonwng almost 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.10 of the Amazon Kinesis Client Library for Java

Milestone#66 * #868 Adding a new metric: Application-level MillisBehindLatest * #879 Keep dependencies up-to-date * #886 Get latest counter before attempting a take to ensure take succeeds * #888 Configure dependabot for v1.x branch

- Java
Published by zengyu714 over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.7 of the Amazon Kinesis Client Library for Java

Release (1.14.7 - Dec 22, 2021) * #881 Update log4j test dependency from 2.16.0 to 2.17.0 and some other dependencies

- Java
Published by zengyu714 over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.6 of the Amazon Kinesis Client Library for Java

Release (1.14.6 - Dec 15, 2021) * #876 Update log4j test dependency from 2.15.0 to 2.16.0

- Java
Published by rexcsn over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.5 of the Amazon Kinesis Client Library for Java

Release 1.14.5 (Dec 10, 2021) * #872 Update log4j test dependency from 1.2.17 to 2.15.0 * #873 Upgrading version of AWS Java SDK to 1.12.128

- Java
Published by rexcsn over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.9 of the Amazon Kinesis Client Library for Java

Milestone#65 * #866 Update logback dependency.

- Java
Published by shanmsac over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.8 of the Amazon Kinesis Client Library for Java

Release 2.3.8 (October 27, 2021)

Milestone#64 * #860 Upgrade Glue schema registry from 1.1.4 to 1.1.5. * #861 Revert PR#847 due to regression for leases without owners and added new tests.

- Java
Published by shanmsac over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.7 of the Amazon Kinesis Client Library for Java

Release 2.3.7 (October 11, 2021)

Known bugs due to #847. 2.3.6 recommended. Milestone#63 * #842 Fixing typo is debug logs. * #846 Fix DynamoDBLeaseTaker logging of available leases * #847 Make use of Java 8 to simplify computeLeaseCounts() * #853 Add configurable initial position for orphaned stream * #854 Create DynamoDB tables on On-Demand billing mode by default. * #855 Emit Glue Schema Registry usage metrics * #857 Fix to shutdown PrefetchRecordsPublisher in gracefull manner * #858 Upgrade AWS SDK version to 2.17.52.

- Java
Published by shanmsac over 4 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.6 of the Amazon Kinesis Client Library for Java

Release 2.3.6 (July 9, 2021)

Milestone#62 * #836 Upgraded AWS SDK version to 2.16.98 * #835 Upgraded Glue Schema Registry version to 1.1.1 * #828 Modified wildcard imports to individual imports. * #817 Updated the Worker shutdown logic to make sure that the LeaseCleanupManager also terminates all the threads that it has started. * #794 Silence warning when there are no stale streams to delete.

- Java
Published by AvinashChowdary almost 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.4 of the Amazon Kinesis Client Library for Java

Release (1.14.4 - June 14, 2021)

  • Milestone#61
  • #816 Updated the Worker shutdown logic to make sure that the LeaseCleanupManager also terminates all the threads that it has started.
  • #821 Upgrading version of AWS Java SDK to 1.12.3

- Java
Published by AvinashChowdary almost 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.5 of the Amazon Kinesis Client Library for Java

Release 2.3.5 (June 14, 2021)

Milestone#59 * #824 Upgraded dependencies * logback-classic version to 1.2.3 * AWS Java SDK version to 1.12.3 * AWS SDK version to 2.16.81 * #815 Converted Future to CompletableFuture which helps in proper conversion to Scala using Scala Future Converters. * #810 Bump commons-io from 2.6 to 2.7 * #804 Allowing user to specify an initial timestamp in which daemon will process records. * #802 Upgraded guava from 26.0-jre to 29.0-jre * #801 Fixing a bug that causes to block indefinitely when trying to unlock a lock that isn't locked. * #762 Added support for web identity token in multilang

- Java
Published by AvinashChowdary almost 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.3 of the Amazon Kinesis Client Library for Java

Latest Release (1.14.3 - May 3, 2021)

  • Milestone#60
  • #811 Fixing a bug in KinesisProxy that can lead to undetermined behavior during partial failures.
  • #811 Adding guardrails to handle duplicate shards from the service.

- Java
Published by joshua-kim about 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.2 of the Amazon Kinesis Client Library for Java

Latest Release (1.14.2 - February 24, 2021)

  • Milestone#57
  • #790 Fixing a bug that caused paginated ListShards calls with the ShardFilter parameter to fail when the lease table was being initialized.

- Java
Published by joshua-kim over 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.4 of the Amazon Kinesis Client Library for Java

Latest Release 2.3.4 (February 19, 2021)

Milestone#56 * #788 Fixing a bug that caused paginated ListShards calls with the ShardFilter parameter to fail when the lease table was being initialized.

- Java
Published by joshua-kim over 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.3 of the Amazon Kinesis Client Library for Java

Latest Release 2.3.3 (December 23, 2020)

Milestone#55 * Fixing bug in PrefetchRecordsPublisher which was causing retry storms if initial request fails. * Fixing bug where idleTimeBetweenReadsInMillis property was ignored in PollingConfig.

Release 2.3.2 (November 19, 2020)

Milestone#54 * Adding support for Glue Schema Registry. Deserialize and read schemas associated with the records. * Updating AWS SDK version to 2.15.31

- Java
Published by ashwing over 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.14.1 of the Amazon Kinesis Client Library for Java

Latest Release (1.14.1 - January 27, 2021)

  • Milestone#56

  • Fix for cross DDB table interference when multiple KCL applications are run in same JVM.

  • Fix and guards to avoid potential checkpoint rewind during shard end, which may block children shard processing.

  • Fix for thread cycle wastage on InitializeTask for deleted shard.

  • Improved logging in LeaseCleanupManager that would indicate why certain shards are not cleaned up from the lease table.

- Java
Published by ashwing over 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release v2.3.1 of the Amazon Kinesis Client Library for Java

Latest Release 2.3.1 (October 20, 2020)

Milestone#53 * Introducing support for processing multiple kinesis data streams with the same KCL 2.x for java consumer application * To build a consumer application that can process multiple streams at the same time, you must implement a new interface called MultistreamTracker (https://github.com/awslabs/amazon-kinesis-client/blob/0c5042dadf794fe988438436252a5a8fe70b6b0b/amazon-kinesis-client/src/main/java/software/amazon/kinesis/processor/MultiStreamTracker.java)

  • MultistreamTracker will also publish various metrics around the current active streams being processed, the number of streams which are deleted at this time period or are pending deletion.

- Java
Published by yatins47 over 5 years ago

https://github.com/awslabs/amazon-kinesis-client - Release v1.14.0 of the Amazon Kinesis Client Library for Java

Release 1.14.0 (August 17, 2020)

Milestone#50

  • Behavior of shard synchronization is moving from each worker independently learning about all existing shards to workers only discovering the children of shards that each worker owns. This optimizes memory usage, lease table IOPS usage, and number of calls made to kinesis for streams with high shard counts and/or frequent resharding.

  • When bootstrapping an empty lease table, KCL utilizes the ListShard API's filtering option (the ShardFilter optional request parameter) to retrieve and create leases only for a snapshot of shards open at the time specified by the ShardFilter parameter. The ShardFilter parameter enables you to filter out the response of the ListShards API, using the Type parameter. KCL uses the Type filter parameter and the following of its valid values to identify and return a snapshot of open shards that might require new leases.

    • Currently, the following shard filters are supported:
      • AT_TRIM_HORIZON - the response includes all the shards that were open at TRIM_HORIZON.
      • AT_LATEST - the response includes only the currently open shards of the data stream.
      • AT_TIMESTAMP - the response includes all shards whose start timestamp is less than or equal to the given timestamp and end timestamp is greater than or equal to the given timestamp or still open.
    • ShardFilter is used when creating leases for an empty lease table to initialize leases for a snapshot of shards specified at KinesisClientLibConfiguration#initialPositionInStreamExtended.
    • For more information about ShardFilter, see the official AWS documentation on ShardFilter.

  • Introducing support for the ChildShards response of the GetRecords API to perform lease/shard synchronization that happens at SHARD_END for closed shards, allowing a KCL worker to only create leases for the child shards of the shard it finished processing.

    • For KCL 1.x applications, this uses the ChildShards response of the GetRecords API.
    • For more information, see the official AWS Documentation on GetRecords and ChildShard.

  • KCL now also performs additional periodic shard/lease scans in order to identify any potential holes in the lease table to ensure the complete hash range of the stream is being processed and create leases for them if required. When KinesisClientLibConfiguration#shardSyncStrategyType is set to ShardSyncStrategyType.SHARD_END, PeriodicShardSyncManager#leasesRecoveryAuditorInconsistencyConfidenceThreshold will be used to determine the threshold for number of consecutive scans containing holes in the lease table after which to enforce a shard sync. When KinesisClientLibConfiguration#shardSyncStrategyType is set to ShardSyncStrategyType.PERIODIC, leasesRecoveryAuditorInconsistencyConfidenceThreshold is ignored.

    • New configuration options are available to configure PeriodicShardSyncManager in KinesisClientLibConfiguration

    | Name | Default | Description | | ----------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | leasesRecoveryAuditorInconsistencyConfidenceThreshold | 3 | Confidence threshold for the periodic auditor job to determine if leases for a stream in the lease table is inconsistent. If the auditor finds same set of inconsistencies consecutively for a stream for this many times, then it would trigger a shard sync. Only used for ShardSyncStrategyType.SHARD_END. | * New CloudWatch metrics are also now emitted to monitor the health of PeriodicShardSyncManager:

    | Name | Description | | --------------------------- | ------------------------------------------------------ | | NumStreamsWithPartialLeases | Number of streams that had holes in their hash ranges. | | NumStreamsToSync | Number of streams which underwent a full shard sync. |

  • Introducing deferred lease cleanup. Leases will be deleted asynchronously by LeaseCleanupManager upon reaching SHARD_END, when a shard has either expired past the stream’s retention period or been closed as the result of a resharding operation.

    • New configuration options are available to configure LeaseCleanupManager.

    | Name | Default | Description | | ----------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------- | | leaseCleanupIntervalMillis | 1 minute | Interval at which to run lease cleanup thread. | | completedLeaseCleanupIntervalMillis | 5 minutes | Interval at which to check if a lease is completed or not. | | garbageLeaseCleanupIntervalMillis | 30 minutes | Interval at which to check if a lease is garbage (i.e trimmed past the stream's retention period) or not. |

  • Including an optimization to KinesisShardSyncer to only create leases for one layer of shards.

  • Changing default shard prioritization strategy to be NoOpShardPrioritization to allow prioritization of completed shards. Customers who are upgrading to this version and are reading from TRIM_HORIZON should continue using ParentFirstPrioritization while upgrading.

  • Upgrading version of AWS SDK to 1.11.844.

  • #719 Upgrading version of Google Protobuf to 3.11.4.

  • #712 Allowing KCL to consider lease tables in UPDATING healthy.

- Java
Published by ychunxue almost 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.3.0 of the Amazon Kinesis Client Library for Java

Latest Release (2.3.0 - August 17, 2020)

  • Milestone#52

  • Behavior of shard synchronization is moving from each worker independently learning about all existing shards to workers only discovering the children of shards that each worker owns. This optimizes memory usage, lease table IOPS usage, and number of calls made to kinesis for streams with high shard counts and/or frequent resharding.

  • When bootstrapping an empty lease table, KCL utilizes the ListShard API's filtering option (the ShardFilter optional request parameter) to retrieve and create leases only for a snapshot of shards open at the time specified by the ShardFilter parameter. The ShardFilter parameter enables you to filter out the response of the ListShards API, using the Type parameter. KCL uses the Type filter parameter and the following of its valid values to identify and return a snapshot of open shards that might require new leases.

    • Currently, the following shard filters are supported:
      • AT_TRIM_HORIZON - the response includes all the shards that were open at TRIM_HORIZON.
      • AT_LATEST - the response includes only the currently open shards of the data stream.
      • AT_TIMESTAMP - the response includes all shards whose start timestamp is less than or equal to the given timestamp and end timestamp is greater than or equal to the given timestamp or still open.
    • ShardFilter is used when creating leases for an empty lease table to initialize leases for a snapshot of shards specified at RetrievalConfig#initialPositionInStreamExtended.
    • For more information about ShardFilter, see the official AWS documentation on ShardFilter.

  • Introducing support for the ChildShards response of the GetRecords and the SubscribeToShard APIs to perform lease/shard synchronization that happens at SHARD_END for closed shards, allowing a KCL worker to only create leases for the child shards of the shard it finished processing.

    • For shared throughout consumer applications, this uses the ChildShards response of the GetRecords API. For dedicated throughput (enhanced fan-out) consumer applications, this uses the ChildShards response of the SubscribeToShard API.
    • For more information, see the official AWS Documentation on GetRecords, SubscribeToShard, and ChildShard.

  • KCL now also performs additional periodic shard/lease scans in order to identify any potential holes in the lease table to ensure the complete hash range of the stream is being processed and create leases for them if required. PeriodicShardSyncManager is the new component that is responsible for running periodic lease/shard scans.

    • New configuration options are available to configure PeriodicShardSyncManager in LeaseManagementConfig

    | Name | Default | Description | | ----------------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | leasesRecoveryAuditorExecutionFrequencyMillis | 120000 (2 minutes) | Frequency (in millis) of the auditor job to scan for partial leases in the lease table. If the auditor detects any hole in the leases for a stream, then it would trigger shard sync based on leasesRecoveryAuditorInconsistencyConfidenceThreshold. | | leasesRecoveryAuditorInconsistencyConfidenceThreshold | 3 | Confidence threshold for the periodic auditor job to determine if leases for a stream in the lease table is inconsistent. If the auditor finds same set of inconsistencies consecutively for a stream for this many times, then it would trigger a shard sync | * New CloudWatch metrics are also now emitted to monitor the health of PeriodicShardSyncManager:

    | Name | Description | | --------------------------- | ------------------------------------------------------ | | NumStreamsWithPartialLeases | Number of streams that had holes in their hash ranges. | | NumStreamsToSync | Number of streams which underwent a full shard sync. |

  • Introducing deferred lease cleanup. Leases will be deleted asynchronously by LeaseCleanupManager upon reaching SHARD_END, when a shard has either expired past the stream’s retention period or been closed as the result of a resharding operation.

    • New configuration options are available to configure LeaseCleanupManager.

    | Name | Default | Description | | ----------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------- | | leaseCleanupIntervalMillis | 1 minute | Interval at which to run lease cleanup thread. | | completedLeaseCleanupIntervalMillis | 5 minutes | Interval at which to check if a lease is completed or not. | | garbageLeaseCleanupIntervalMillis | 30 minutes | Interval at which to check if a lease is garbage (i.e trimmed past the stream's retention period) or not. |

  • Introducing experimental support for multistreaming, allowing a single KCL application to multiplex processing multiple streams.

    • New configuration options are available to enable multistreaming in RetrievalConfig#appStreamTracker.

  • Fixing a bug in PrefetchRecordsPublisher restarting while it was already running.

  • Including an optimization to HierarchicalShardSyncer to only create leases for one layer of shards.

  • Adding support to prepare and commit lease checkpoints with arbitrary bytes.

    • This allows checkpointing of an arbitrary byte buffer up to the maximum permitted DynamoDB item size (currently 400 KB as of release), and can be used for recovery by passing a serialized byte buffer to RecordProcessorCheckpointer#prepareCheckpoint and RecordProcessorCheckpointer#checkpoint.

  • Upgrading version of AWS SDK to 2.14.0.

  • #725 Allowing KCL to consider lease tables in UPDATING healthy.

- Java
Published by joshua-kim almost 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.11 of the Amazon Kinesis Client Library for Java

Release 2.2.11 (May 28, 2020)

Milestone#51 * Adjusting HTTP2 initial window size to 512 KB * PR#706 * Updating protobuf-java to version 3.11.4 * PR#718 * Updating the AWS Java SDK to version 2.13.25 * PR#722

- Java
Published by micah-jaffe about 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.10 of the Amazon Kinesis Client Library for Java

Release 2.2.10 (March 26, 2020)

Milestone#48 * Fixing a bug in DynamoDB billing mode support for special regions. * PR#703 * Adding request id logging to ShardConsumerSubscriber. * PR#705

- Java
Published by joshua-kim about 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.13.3 of the Amazon Kinesis Client Library for Java

Release 1.13.3 (March 2, 2020)

[Milestone#49](https://github.com/awslabs/amazon-kinesis-client/milestone/49） * Refactoring shard closure verification performed by ShutdownTask. * PR #684 * Fixing the bug in ShardSyncTaskManager to resolve the issue of new shards not being processed after resharding. * PR #694

- Java
Published by ychunxue about 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.9 of the Amazon Kinesis Client Library for Java

Release 2.2.9 (Febuary 17, 2020)

Milestone#47 * Updating the AWS SDK version to 2.10.66. * PR#687 * Adding request id logging to the SubscribeToShard response. * PR#678

- Java
Published by joshua-kim over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.8 of the Amazon Kinesis Client Library for Java

Release 2.2.8 (January 28, 2020)

Milestone#46 * Updating the AWS SDK version to 2.10.56. * PR#679 * Making ShardConsumerTest resilient to race conditions. * PR#668 * Updating integration test naming. * PR#667

- Java
Published by joshua-kim over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.13.2 of the Amazon Kinesis Client Library for Java

Latest Release (1.13.2 Janurary 13, 2020)

  • Adding backward compatible constructors that use the default DDB Billing Mode (#673)

- Java
Published by Cory-Bradshaw over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.13.1 of the Amazon Kinesis Client Library for Java

Release 1.13.1 (December 31, 2019)

Milestone#44 * Adding BillingMode Support to KCL 1.x. This enables the customer to specify if they want provisioned capacity for DDB, or pay per request. * PR #656 * Ensure ShardSyncTask invocation from ShardSyncTaskManager for pending ShardEnd events. * PR #659 * Fix the LeaseManagementIntegrationTest failure. * PR #670

- Java
Published by ychunxue over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.7 of the Amazon Kinesis Client Library for Java

Release 2.2.7 (December 2, 2019)

Milestone#45 * Updating the AWS SDK version to 2.10.25 * PR#657 * Adding a configurable DynamoDB billing mode * PR#582

- Java
Published by micah-jaffe over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.6 of the Amazon Kinesis Client Library for Java

Release 2.2.6 (November 7, 2019)

Milestone#43 * Updating the SDK version to 2.9.25. * PR#638 * Clearing the local cache on a subscription termination, to avoid noisy logs on new subscriptions. * PR#642 * Updating the SDK version to 2.10.0 in order to fix the premature H2 stream close issue. * PR#649 * NOTE: SDK has a known connection teardown issue when multiple H2 streams are used within a connection. This might result in shard consumers sticking to a stale service host and not progressing. If your shard consumer gets stuck, use the following configuration as a workaround. This configuration might result in up to 5X increase in total connections. KinesisAsyncClient kinesisClient = KinesisAsyncClient.builder() .region(region) .httpClientBuilder(NettyNioAsyncHttpClient.builder().maxConcurrency(Integer.MAX_VALUE).maxHttp2Streams(1)) .build()

- Java
Published by ychunxue over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.13.0 of the Amazon Kinesis Client Library for Java

Release 1.13.0 (November 5, 2019)

Milestone#42 * Handling completed and blocked tasks better during graceful shutdown. * PR #640

- Java
Published by ychunxue over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.5 of the Amazon Kinesis Client Library for Java

Release 2.2.5 (October 23, 2019)

Milestone#40 * Updating Sonatype to dedicated AWS endpoint. * PR#619 * Introducing a validation step to verify if ShardEnd is reached, to prevent shard consumer stuck scenarios in the event of malformed response from service. * PR#624

- Java
Published by ychunxue over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.12.0 of the Amazon Kinesis Client Library for Java

Release 1.12.0 (October 17, 2019)

Milestone#41 * Adding logging around shard end codepaths. * PR #585 * Updating checkpointing failure message to refer to javadocs. * PR #590 * Updating Sonatype to dedicated AWS endpoint. * PR #618 * Introducing a validation step to verify if ShardEnd is reached, to prevent shard consumer stuck scenarios in the event of malformed response from service. * PR #623 * Updating AWS SDK to 1.11.655 * PR #626

- Java
Published by joshua-kim over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.4 of the Amazon Kinesis Client Library for Java

Release 2.2.4 (September 23, 2019)

Milestone#39 * Making FanoutRecordsPublisher test cases resilient to delayed thread operations * PR#612 * Drain delivery queue in the FanoutRecordsPublisher to make slow consumers consume events at their pace * PR#607 * Fix to prevent the onNext event going to stale subscription when restart happens in PrefetchRecordsPublisher * PR#606

- Java
Published by micah-jaffe over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.3 of the Amazon Kinesis Client Library for Java

Release 2.2.3 (September 4, 2019)

Milestone#38 * Fix to prevent data loss and stuck shards in the event of failed records delivery in Polling readers * PR#603

- Java
Published by micah-jaffe over 6 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.2 of the Amazon Kinesis Client Library for Java

Release 2.2.2 (August 19, 2019)

Milestone#36 * Fix to prevent invalid ShardConsumer state transitions due to rejected executor service executions. * PR#560 * Fixing a bug in which initial subscription failure caused a shard consumer to get stuck. * PR#562 * Making CW publish failures visible by executing the async publish calls in a blocking manner and logging on exception. * PR#584 * Update shard end checkpoint failure messaging. * PR#591 * A fix for resiliency and durability issues that occur in the reduced thread mode - Nonblocking approach. * PR#573 * Preventing duplicate delivery due to unacknowledged event, while completing the subscription. * PR#596

- Java
Published by micah-jaffe almost 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.11.2 of the Amazon Kinesis Client Library for Java

Release 1.11.2 (August 15, 2019)

Milestone#35 * Added support for metrics emission in PeriodicShardSyncer. * PR #592

- Java
Published by micah-jaffe almost 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.11.1 of the Amazon Kinesis Client Library for Java

Release 1.11.1 (August 9, 2019)

Milestone#34 * Updated the version of the AWS Java SDK to 1.11.603. * PR #587 * Added logging to KinesisDataFetcher when reaching the end of a shard due to a null next iterator. * PR #585

- Java
Published by micah-jaffe almost 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.11.0 of the Amazon Kinesis Client Library for Java

Release 1.11.0 (August 7, 2019)

Milestone#33 * Updated the version of the AWS Java SDK to 1.11.603 * Improved exception handling and logging in KinesisClientLibLeaseCoordinator to avoid NullPointerExceptions when no leases are found. * PR #558 * Introducing optional new periodic shard sync strategy to perform shard discovery and lease cleanup on a single worker. * PR #579

- Java
Published by micah-jaffe almost 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.1 of the Amazon Kinesis Client Library for Java

Release 2.2.1 (June 28, 2019)

Milestone#32 * Add periodic logging for the state of the thread pool executor service. This service executes the async tasks submitted to and by the ShardConsumer. * Add logging of failures from RxJava layer. * PR#559

- Java
Published by micah-jaffe almost 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.2.0 of the Amazon Kinesis Client Library for Java

Release 2.2.0 (April 8, 2019)

Milestone#31 * Updated License to Apache License 2.0 * PR#523 * Introducing configuration for suppressing logs from ReadTimeoutExceptions caused while calling SubscribeToShard.
Suppression can be configured by setting LifecycleConfig#readTimeoutsToIgnoreBeforeWarning(Count). * PR#528

- Java
Published by Cory-Bradshaw about 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.10.0 of the Amazon Kinesis Client Library for Java

Release 1.10.0 (April 8, 2019)

Milestone#31 * Updated License to Apache License 2.0 * PR#522

- Java
Published by Cory-Bradshaw about 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.1.3 of the Amazon Kinesis Client Library for Java

Release 2.1.3 (March 18, 2019)

Milestone#30 * Added a message to recommend using KinesisClientUtil when an acquire timeout occurs in the FanOutRecordsPublisher. * PR#514 * Added a sleep between retries while waiting for a newly created stream consumer to become active. * PR#506 * Added timeouts on all futures returned from the DynamoDB and Kinesis clients.
The timeouts can be configured by setting LeaseManagementConfig#requestTimeout(Duration) for DynamoDB, and PollingConfig#kinesisRequestTimeout(Duration) for Kinesis. * PR#518 * Upgraded to SDK version 2.5.10. * PR#518 * Artifacts for the Amazon Kinesis Client for Java are now signed by a new GPG key: pub 4096R/86368934 2019-02-14 [expires: 2020-02-14] uid Amazon Kinesis Tools <amazon-kinesis-tools@amazon.com>

- Java
Published by pfifer about 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.1.2 of the Amazon Kinesis Client for Java

Release 2.1.2 (February 18, 2019)

Milestone#29 * Fixed handling of the progress detection in the ShardConsumer to restart from the last accepted record, instead of the last queued record. * PR#492 * Fixed handling of exceptions when using polling so that it will no longer treat SdkExceptions as an unexpected exception. * PR#497 * PR#502 * Fixed a case where lease loss would block the Scheduler while waiting for a record processor's processRecords method to complete. * PR#501

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.1.1 of the Amazon Kinesis Client for Java

Release 2.1.1 (February 6, 2019)

Milestone#28 * Introducing SHUT_DOWN_STARTED state for the WorkerStateChangeListener. * PR#457 * Fixed a bug with AWSSessionCredentials using AWSSecretID instead of AWSAccessID and vice versa. * PR#486 * Upgrading SDK version to 2.4.0, which includes a fix for a possible deadlock when using Enhanced Fan-Out. * PR#493

- Java
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.1.0 of the Amazon Kinesis Client for Java

Release 2.1.0 (January 14, 2019)

Milestone #27 * Introducing MultiLangDaemon support for Enhanced Fan-Out.
* MultiLangDaemon now supports the following command line options. * --properties-file: Properties file that the KCL should use to set up the Scheduler. * --log-configuration: logback.xml that the KCL should use for logging. * Updated AWS SDK dependency to 2.2.0. * MultiLangDaemon now uses logback for logging.

- Java
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.5 of the Amazon Kinesis Client for Java

Release 2.0.5 (November 12, 2018)

Milestone #26 * Fixed a deadlock condition that could occur when using the polling model.
When using the PollingConfig and a slower record processor it was possible to hit a deadlock in the retrieval of records. * PR #462 * Issue #448 * Adjusted RetrievalConfig, and FanOutConfig to use accessors instead of direct member access.
* PR #453

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.9.3 of the Amazon Kinesis Client

Release 1.9.3 (October 30, 2018)

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.4 of the Amazon Kinesis Client for Java

Release 2.0.4 (October 18, 2018)

Milestone #25 * Added method to retrieve leases from the LeaseCoordinator and LeaseTaker. * PR #428 * Fixed a race condition shutting down the Scheduler before it has completed initialization. * PR #439 * Issue #427 * Added HierarchicalShardSyncer which replaces the static ShardSyncer.
HierarchicalShardSyncer removes the contention between multiple instances of the Scheduler when running under a single JVM. * PR #395 * Issue #415 * Added TaskExecutionListener which allows monitoring of tasks being executed by the ShardConsumer.
The listener is invoked before and after a task is executed by the ShardConsumer. * PR #417

- Java
Published by sahilpalvia over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.3 of the Amazon Kinesis Client for Java

Release 2.0.3 (October 8, 2018)

Milestone #23 * Fixed an issue where the KinesisAsyncClient could be misconfigured to use HTTP 1.1.
Using HTTP 1.1 with SubscribeToShard is unsupported, and could cause misdelivery of records to the record processor.
* Issue #391 * PR #434 * PR #433 * Lower the severity of ReadTimeout exceptions.
ReadTimeout exceptions can occur if the client is unable to request data from Kinesis for more than client timeout, which defaults to 30 seconds. This can occur if the record processor blocks for more than the timeout period. ReadTimeout could also occur as part of Issue #391.
* Issue #399 * PR #403 * Added a callback that allows applications to take actions after DynamoDB table creation.
Applications can now install a callback that is called after creating the DynamoDB table by implementing TableCreatorCallback.
* PR #413 * Updated the guava dependency to 26.0-jre.
* PR #420 * Issue #416 * Added some additional debug logging around the initialization of the FanOutRecordsPublisher.
* PR #398 * Upgraded AWS SDK version to 2.0.6
* PR #434

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

Release 7

  • Don't attempt to validate continuation sequence number when it's null

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

Experimental Release 2.0.3-experimental-6

  • Upgraded to AWS SDK 2.0.4

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

Release 2.0.3-experimental-5

  • Validate the continuation sequence number in SubscribeToShardEvent's as well as the records.

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

  • Log the SubscribeToShardRequest before making the request, and after connecting.

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

  • Log requestId of successful SubscribeToShard requests
  • Validate sequence numbers in advancePosition() of the checkpointer to catch direct uses of checkpoint()

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

  • Added sequence number validation in the ShardConsumer
  • Added sequence number validation in the ShardRecordProcessorCheckpointer

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Experimental release of the Amazon Kinesis Client

This is an experimental release of the Amazon Kinesis Client for Java released to assist with tracking down possible record issues

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.2 of the Amazon Kinesis Client

Release 2.0.2 (September 4, 2018)

Milestone #22 * Fixed an issue where the a warning would be logged every second if logWarningForTaskAfterMillis was set.
The logging for last time of data arrival now respects the value of logWarningForTaskAfterMillis.
* PR #383 * Issue #381 * Moved creation of WorkerStateChangedListener and GracefulShutdownCoordinator to the CoordinatorConfig. Originally the WorkerStateChangedListener and GracefulShutdownCoordinator were created by methods on the SchedulerCoordinatorFactory, but they should have been configuration options.
The original methods have been deprecated, and may be removed at a later date.
* PR #385 * PR #388 * Removed dependency on Apache Commons Lang 2.6.
The dependency on Apache Commons Lang 2.6 has removed, and all usages updated to use Apache Commons Lang 3.7.
* PR #386 * Issue #370 * Fixed a typo in the MutliLang Daemon shutdown hook.
* PR #387 * Added method onAllInitializationAttemptsFailed(Throwable) to WorkerStateChangedListener to report when all initialization attempts have failed.
This method is a default method, and it isn't require to implement the method. This method is only called after all attempts to initialize the Scheduler have failed. * PR #369

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.9.2 of the Amazon Kinesis Client

Release 1.9.2 (September 4, 2018)

  • Allow use of Immutable Clients
  • Allow the use of AT_TIMESTAMP for MultiLang Daemon Clients.
    • [Issue #341)(https://github.com/awslabs/amazon-kinesis-client/issues/341)
    • PR #342
  • Update the cache for KinesisProxy#getShard on cache misses.
  • Changed release process to use a standard process.
  • Removed tests that expected a null region response for unknown regions.
  • Updated the version of the AWS Java SDK to 1.11.400

- Java
Published by pfifer over 7 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.1 of the Amazon Kinesis Client Library for Java

Release 2.0.1 (August 21, 2018)

  • Mark certain internal components with @KinesisClientInternalApi attribute.
    Components marked as internal may be deprecated at a faster rate than public components.
  • Fixed an issue where ResourceNotFoundException on subscription to a shard was not triggering end of shard handling.
    If a lease table contains a shard that is no longer present in the stream attempt to subscribe to that shard will trigger a ResourceNotFoundException. These exception are treated the same as reaching the end of a shard.
  • Fixed an issue where the KCL would not Use the configured DynamoDB IOPs when creating the lease table.
  • Make the maximum number of Scheduler initialization attempts configurable.
    The maximum number of Scheduler initialization attempts can be configured via CoordinatorConfig#maxInitializationAttempts.
  • Fixed an issue where it was possible to get a duplicate record when resubscribing to a shard.
    Subscribe to shard requires periodic resubscribing, and uses a new concept of a continuation sequence number. If the continuation sequence number was equal to the last record that record would be processed a second time. Resubscribing now uses AFTER_SEQUENCE_NUMBER to ensure that only later records are returned.
  • Upgraded to AWS SDK 2.0.1
  • Fixed an issue where time based restart of the subscription wasn't resetting the lastRequestTime.
    If a subscription hasn't delivered any data for more than 30 seconds it will be canceled and restarted. This detection is based of the lastRequestTime which wasn't getting reset after the restart was triggered.
  • Fixed an issue where requesting on the subscription from the FanOutRecordsPublisher could trigger an unexpected failure.
    Due to a race condition the underlying flow in the subscription could be set to something else. The method is now synchronized, and verifies that the subscriber it was created with is still the subscriber in affect.
    This issue generally would only appear when multiple errors were occurring while connecting to Kinesis.
  • Fixed an issue where the number of requested items could exceed the capacity of the RxJava queue.
    There was an off by one issue when determining whether to make a request to the SDK subscription. This changes the calculation to represent the capacity as a queue.

- Java
Published by pfifer almost 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 2.0.0 of the Amazon Kinesis Client Library for Java

Release 2.0.0

  • The Maven groupId, along with the version, for the Amazon Kinesis Client has changed from com.amazonaws to software.amazon.kinesis.
    To add a dependency on the new version of the Amazon Kinesis Client:
    xml <dependency> <groupId>software.amazon.kinesis</groupId> <artifactId>amazon-kinesis-client</artifactId> <version>2.0.0</version> </dependency>
  • Added support for Enhanced Fan Out.
    Enhanced Fan Out provides for lower end to end latency, and increased number of consumers per stream.
    • Records are now delivered via streaming, reducing end-to-end latency.
    • The Amazon Kinesis Client will automatically register a new consumer if required.
      When registering a new consumer, the Kinesis Client will default to the application name unless configured otherwise.
    • SubscribeToShard maintains long lived connections with Kinesis, which in the AWS Java SDK 2.0 is limited by default.
      The KinesisClientUtil has been added to assist configuring the maxConcurrency of the KinesisAsyncClient.
      WARNING: The Amazon Kinesis Client may see significantly increased latency, unless the KinesisAsyncClient is configured to have a maxConcurrency high enough to allow all leases plus additional usages of the KinesisAsyncClient.
    • The Amazon Kinesis Client now uses 3 additional Kinesis API's:
      WARNING: If using a restrictive Kinesis IAM policy you may need to add the following API methods to the policy.
    • SubscribeToShard
    • DescribeStreamSummary
    • DescribeStreamConsumer
    • RegisterStreamConsumer
    • New configuration options are available to configure Enhanced Fan Out.

| Name | Default | Description | |-----------------|---------|---------------------------------------------------------------------------------------------------------------------| | consumerArn | Unset | The ARN for an already created consumer. If this is set, the Kinesis Client will not attempt to create a consumer. | | streamName | Unset | The name of the stream that a consumer should be create for if necessary | | consumerName | Unset | The name of the consumer to create. If this is not set the applicationName will be used instead. | | applicationName | Unset | The name of the application. This is used as the name of the consumer unless consumerName is set. |

  • Modular Configuration of the Kinesis Client The Kinesis Client has migrated to a modular configuration system, and the KinesisClientLibConfiguration class has been removed.
    Configuration has been split into 7 classes. Default versions of the configuration can be created from the ConfigsBuilder.
    Please [see the migration guide for more information][migration-guide].

    • CheckpointConfig
    • CoordinatorConfig
    • LeaseManagementConfig
    • LifecycleConfig
    • MetricsConfig
    • ProcessorConfig
    • RetrievalConfig

  • Upgraded to AWS Java SDK 2.0
    The Kinesis Client now uses the AWS Java SDK 2.0. The dependency on AWS Java SDK 1.11 has been removed. All configurations will only accept 2.0 clients.

    • When configuring the KinesisAsyncClient the KinesisClientUtil#createKinesisAsyncClient can be used to configure the Kinesis Client
    • If you need support for AWS Java SDK 1.11 you will need to add a direct dependency.
      When adding a dependency you must ensure that the 1.11 versions of Jackson dependencies are excluded
      [Please see the migration guide for more information][migration-guide]

  • MultiLangDaemon is now a separate module
    The MultiLangDaemon has been separated to its own Maven module and is no longer available in amazon-kinesis-client. To include the MultiLangDaemon, add a dependency on amazon-kinesis-client-multilang.

- Java
Published by pfifer almost 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.9.1 of the Amazon Kinesis Client for Java

Release 1.9.1 (April 30, 2018)

  • Added the ability to create a prepared checkpoint when at SHARD_END.
  • Added the ability to subscribe to worker state change events.
  • Added support for custom lease managers.
    A custom LeaseManager can be provided to Worker.Builder that will be used to provide lease services. This makes it possible to implement custom lease management systems in addition to the default DynamoDB system.
  • Updated the version of the AWS Java SDK to 1.11.219

- Java
Published by pfifer about 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.9.0 of the Amazon Kinesis Client for Java

Latest Release (1.9.0)

  • Introducing support for ListShards API. This API is used in place of DescribeStream API to provide more throughput during ShardSyncTask. Please consult the AWS Documentation for ListShards for more information.

    • ListShards supoorts higher call rate, which should reduce instances of throttling when attempting to synchronize the shard list.
    • Added configuration parameters for ListShards usage

    | Name | Default | Description | | ---- | ------- | ----------- | | listShardsBackoffTimeInMillis | 1500 ms | This is the default backoff time between 2 ListShards calls when throttled. | | listShardsRetryAttempts | 50 | This is the maximum number of times the KinesisProxy will retry to make ListShards calls on being throttled. |

  • Updating the version of AWS Java SDK to 1.11.272.

    • Version 1.11.272 is now the minimum support versiojn of the SDK.

  • Deprecating the following methods, and classes. These methods, and classes will be removed in a future release.

PR #293

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.10 of the Amazon Kinesis Client for Java

Release 1.8.10

  • Allow providing a custom IKinesisProxy implementation.
  • Checkpointing on a different thread should no longer emit a warning about NullMetricsScope.
  • Upgraded the AWS Java SDK to version 1.11.271

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.9 of the Amazon Kinesis Client for Java

Release 1.8.9

  • Allow disabling check for the case where a child shard has an open parent shard.
    There is a race condition where it's possible for the a parent shard to appear open, while having child shards. This check can now be disabled by setting ignoreUnexpectedChildShards to true.
  • Upgraded the AWS SDK for Java to 1.11.261

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.8 of the Amazon Kinesis Client for Java

Release 1.8.8

  • Fixed issues with leases losses due to ExpiredIteratorException in PrefetchGetRecordsCache and AsynchronousFetchingStrategy.
    PrefetchGetRecordsCache will request for a new iterator and start fetching data again.
  • Added warning message for long running tasks.
    Logging long running tasks can be enabled by setting the following configuration property:

| Name | Default | Description | | ---- | ------- | ----------- | | logWarningForTaskAfterMillis | Not set | Milliseconds after which the logger will log a warning message for the long running task |

  • PR#259
    • Handling spurious lease renewal failures gracefully.
      Added better handling of DynamoDB failures when updating leases. These failures would occur when a request to DynamoDB appeared to fail, but was actually successful.
  • PR#247
    • ShutdownTask gets retried if the previous attempt on the ShutdownTask fails.
  • PR#267
    • Fix for using maxRecords from KinesisClientLibConfiguration in GetRecordsCache for fetching records.
  • PR#264

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.7 of the Amazon Kinesis Client Library for Java

Release 1.8.7

  • Don't add a delay for synchronous requests to Kinesis
    Removes a delay that had been added for synchronous GetRecords calls to Kinesis.

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.6 of the Amazon Kinesis Client Library for Java

Release 1.8.6

  • Add prefetching of records from Kinesis
    Prefetching will retrieve and queue additional records from Kinesis while the application is processing existing records.
    Prefetching can be enabled by setting dataFetchingStrategy to PREFETCH_CACHED. Once enabled an additional fetching thread will be started to retrieve records from Kinesis. Retrieved records will be held in a queue until the application is ready to process them.
    Pre-fetching supports the following configuration values:

| Name | Default | Description | | ---- | ------- | ----------- | | dataFetchingStrategy | DEFAULT | Which data fetching strategy to use | | maxPendingProcessRecordsInput | 3 | The maximum number of process records input that can be queued | | maxCacheByteSize | 8 MiB | The maximum number of bytes that can be queued | | maxRecordsCount | 30,000 | The maximum number of records that can be queued | | idleMillisBetweenCalls | 1,500 ms | The amount of time to wait between calls to Kinesis |

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.5 of the Amazon Kinesis Client for Java

Release 1.8.5 (September 26, 2017)

  • Only advance the shard iterator for the accepted response.
    This fixes a race condition in the KinesisDataFetcher when it's being used to make asynchronous requests. The shard iterator is now only advanced when the retriever calls DataFetcherResult#accept().

- Java
Published by pfifer over 8 years ago

https://github.com/awslabs/amazon-kinesis-client - Release 1.8.4 of the Amazon Kinesis Client for Java

Release 1.8.4 (September 22, 2017)

  • Create a new completion service for each request.
    This ensures that canceled tasks are discarded. This will prevent a cancellation exception causing issues processing records.

- Java
Published by pfifer over 8 years ago