https://github.com/astronomy-commons/adc-streaming - v2.6.0

This release primarily adds better support for publishing relatively large messages quickly, without overflowing the internal publishing queue or losing messages due to timeout, as well as fixing an authentication issue for long-running clients when using OAUTHBEARER tokens.

What's Changed

• Remove oauth_cb workaround for KIP-768 support by @lpsinger in https://github.com/astronomy-commons/adc-streaming/pull/72
• Improve handling of high message volumes by @cnweaver in https://github.com/astronomy-commons/adc-streaming/pull/73

Full Changelog: https://github.com/astronomy-commons/adc-streaming/compare/v2.5.0...v2.6.0

- Python
Published by cnweaver 11 months ago

https://github.com/astronomy-commons/adc-streaming - v2.5.0

This minor release adds support for specifying message keys when publishing.

What's Changed

• Add support for sending messages with keys by @cnweaver in https://github.com/astronomy-commons/adc-streaming/pull/70

Full Changelog: https://github.com/astronomy-commons/adc-streaming/compare/v2.4.0...v2.5.0

- Python
Published by cnweaver over 1 year ago

https://github.com/astronomy-commons/adc-streaming - v2.4.0

This minor release exposes two new capabilities: to read from a specific time or logical offset and to publish to multiple topics using a single Producer object.

What's Changed

• Allow specifying logical offset or datetime start position on Consumer.stream() by @alchzh in https://github.com/astronomy-commons/adc-streaming/pull/65
• Multi-topic publishing by @cnweaver in https://github.com/astronomy-commons/adc-streaming/pull/69 . Users can continue to specify a single topic when constructing a Producer and have calls to write() automatically send messages to that topic, or can now specify the target topic when calling write().

Full Changelog: https://github.com/astronomy-commons/adc-streaming/compare/v2.3.2...v2.4.0

- Python
Published by cnweaver about 2 years ago

https://github.com/astronomy-commons/adc-streaming - v2.3.2

This is a bug-fix release which updates the list of compatible dependency versions to avoid recent versions of librdkafka/confluent-kafka with known communications problems.

What's Changed

• Update confluent-kafka version to skip recent buggy releases by @lpsinger in https://github.com/astronomy-commons/adc-streaming/pull/67
• Update doc strings and comments for confluent-kafka workarounds by @lpsinger in https://github.com/astronomy-commons/adc-streaming/pull/68

Full Changelog: https://github.com/astronomy-commons/adc-streaming/compare/v2.3.1...v2.3.2

- Python
Published by cnweaver about 3 years ago

https://github.com/astronomy-commons/adc-streaming - v2.3.1

This is a bug-fix release which reverts to using no compression by default when sending messages. Advanced users who wish to use compression can continue to do so by configuring the compression_type ProducerConfig option.

- Python
Published by cnweaver over 3 years ago

https://github.com/astronomy-commons/adc-streaming - v2.3.0

This release adds new minor features (thanks to @lpsinger), but is otherwise fully interface backwards-compatible: - #61: Enable Zstd compression of messages by default, with the option to choose any compression method supported by Apache Kafka. - #62: Add the option to adjust the producer message.max.bytes. This can be important to be able to send messages larger than the default maximum, when the configured maximum on the broker is also larger than the default.

- Python
Published by cnweaver over 3 years ago

https://github.com/astronomy-commons/adc-streaming - v2.2.0

This release includes bug fixes and improvements since v2.1.0: - #60: The build system has been updated with a pyproject.toml file. - #58: adc.Consumer now supports a stop method which can be used to end the stream loop conveniently from another thread. - #55: adc.Consumer._stream_forever handles errors raised by librdkafka better.

- Python
Published by cnweaver almost 4 years ago

https://github.com/astronomy-commons/adc-streaming - v2.1.0: Add support for OIDC/OAUTHBEARER Authentication

This release includes https://github.com/astronomy-commons/adc-streaming/pull/53 by @lpsinger which enables the new(ly enhanced) OIDC authentication mechanism supported by the latest (version 3.1.0) Apache Kafka release. When not using this new feature this release remains compatible with older Kafka versions.

Also included is a bug-fix to avoid crashing when attempting to read a topic with multiple partitions 'to its end' while new messages are being actively published.

- Python
Published by cnweaver over 4 years ago

https://github.com/astronomy-commons/adc-streaming - v1.2.0: Improve KafkaException to show more low-level detail

This release includes #43, which adds more detail to the adc.errors.KafkaException class by exposing the low-level confluent_kafka.KafkaError directly on the exception so that callers can make judgements about the retriability of errors.

- Python
Published by spenczar about 5 years ago

https://github.com/astronomy-commons/adc-streaming - v1.1.0: Obey stored offsets

This release includes the changes from #42, so that offsets which are stored are used. This changes the external API in some backwards-incompatible ways, so perhaps this ought to be a 2.0.0 release if we were sticklers for semver, but we have few enough users to be pretty sure nobody was using the explicit offsets API.

- Python
Published by spenczar almost 6 years ago

https://github.com/astronomy-commons/adc-streaming - v1.0.2: Fix conda build

This release fixes a misconfiguration of the conda build. It does not change anything user-facing.

- Python
Published by spenczar almost 6 years ago

https://github.com/astronomy-commons/adc-streaming - v1.0.1: Higher default producer timeout

The default timeout for producing a message is raised in this release to 10 seconds from 2 seconds. Connecting to a Kafka broker with TLS and SCRAM authentication can take at least 2s normally with a round-trip latency of 100ms between the producer and broker, which resulted in the first message timing out consistently. 10s is a more reasonable default for producing to a broker over the internet.

In addition, #38 fixes a minor formatting bug when warning about reading from a LATEST offset, and #39 removes fastavro as a dependency because it is no longer used since v1.0.0.

- Python
Published by spenczar almost 6 years ago

https://github.com/astronomy-commons/adc-streaming - v1.0.0: First stable release

This release provides a new API, described thoroughly in #36. Consumer and Producer behavior now has dedicated classes and configurations, annotated with types. The open API is preserved.

- Python
Published by spenczar almost 6 years ago

https://github.com/astronomy-commons/adc-streaming - v0.2.0: Use certifi trust store by default

This release adds the feature described in #31: it configures the Kafka client to use certifi's set of CA certificates by default. This should improve portability when talking to brokers with SSL enabled.

- Python
Published by spenczar about 6 years ago