Skip to content

Topic Admin API, list_topics(), Python 3.7, fixes #382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jun 7, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b7439a7
Add list_topics() API (#161, @tbsaunde, @stephan-hof)
tbsaunde May 15, 2018
f6cda94
Python 3.7 support
edenhill May 9, 2018
bb0f00c
Topic Admin API
edenhill May 9, 2018
bb39363
travis: use adminapi librdkafka branch
edenhill May 9, 2018
0a67983
Require futures on Python 2.x
edenhill May 9, 2018
400fde8
Refactored list_topics() to return specific types rather than dict
edenhill May 16, 2018
3d1e0a6
Added (optional) memory tracking to integration tests
edenhill May 16, 2018
ac3f603
Fixes and refactoring for integration tests
edenhill May 16, 2018
4bd840c
test_docs: skip imported modules and inherited objects
edenhill May 18, 2018
dad896d
Admin API: return multiple futures, bugfixes, docs
edenhill May 18, 2018
64d7fa9
Added Py2&Py3 compatible cfl_PyInt abstraction
edenhill May 18, 2018
4c6de0d
Fix memory leaks in generic setters
edenhill May 18, 2018
f669580
Fix call ordering to avoid callback crash on implicit close (#265)
edenhill May 21, 2018
f821652
Producer: Header values may now be unicode strings (auto convert to U…
edenhill May 21, 2018
136efd3
Fix KafkaError refcounting
edenhill May 21, 2018
3b4f3fb
Python 2/3 compat, minor test improvements
edenhill May 21, 2018
bd72bf6
Updated for librdkafka's internal -> background queue rename
edenhill May 25, 2018
a08db1c
Require librdkafka >=v0.11.5
edenhill May 29, 2018
c87cc7f
Change TypeError to ValueError for invalid input
edenhill May 31, 2018
4f0f69e
Add __len__ to ConfigResource and overwrite option to .set_config
edenhill May 31, 2018
868787d
librdkafka AdminOptions_new interface update
edenhill May 31, 2018
3aa92b3
Don't return the future from the C Admin methods, not needed
edenhill May 31, 2018
41bb5a7
adminapi example: added an async and delta/incremental alter_configs …
edenhill May 31, 2018
b01e50b
Python 2 safe unicode in source
edenhill May 31, 2018
d52858c
Add throttle_cb support (#237) (#377)
Jun 2, 2018
e4eeaa1
Move Admin API to `admin` sub module, use enums, fix doc strings
edenhill Jun 2, 2018
d613894
Include avro in docs, fix docstrings in confluent_kafka, Consumer, avro
edenhill Jun 2, 2018
1266b3c
Added ThrottleEvent to docs
edenhill Jun 2, 2018
2f32034
Rename AdminClientImpl -> _AdminClientImpl
edenhill Jun 5, 2018
6d928d2
travis: use librdkafka master branch instead of adminapi
edenhill Jun 5, 2018
8d1e8d3
travis: remove librdkafka v0.11.4
edenhill Jun 7, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,12 @@ matrix:
dist: trusty
python: "2.7"
env: LD_LIBRARY_PATH="$PWD/tmp-build/lib" LIBRDKAFKA_VERSION=master
# Source package verification with Python 2.7 and librdkafka v0.9.5
# Source package verification with Python 3.6 and librdkafka master
- os: linux
language: python
dist: trusty
python: "2.7"
env: LD_LIBRARY_PATH="$PWD/tmp-build/lib" LIBRDKAFKA_VERSION=v0.9.5
# Source package verification with Python 3.5 and librdkafka master
- os: linux
language: python
dist: trusty
python: "3.5"
env: LD_LIBRARY_PATH="$PWD/tmp-build/lib" LIBRDKAFKA_VERSION=v0.9.5
python: "3.6"
env: LD_LIBRARY_PATH="$PWD/tmp-build/lib" LIBRDKAFKA_VERSION=master
# cibuildwheel for osx
- os: osx
env: CIBW_BEFORE_BUILD="tools/bootstrap-librdkafka.sh --require-ssl master tmp" CFLAGS="-Itmp/include" LDFLAGS="-Ltmp/lib"
Expand Down
11 changes: 3 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ Prerequisites
=============

* Python >= 2.7 or Python 3.x
* [librdkafka](https://github.com/edenhill/librdkafka) >= 0.9.5 (latest release is embedded in wheels)
* [librdkafka](https://github.com/edenhill/librdkafka) >= 0.11.5 (latest release is embedded in wheels)

librdkafka is embedded in the macosx manylinux wheels, for other platforms, SASL Kerberos/GSSAPI support or
when a specific version of librdkafka is desired, following these guidelines:
Expand Down Expand Up @@ -277,14 +277,9 @@ In order to run full test suite, simply execute:
**NOTE**: Requires `tox` (please install with `pip install tox`), several supported versions of Python on your path, and `librdkafka` [installed](tools/bootstrap-librdkafka.sh) into `tmp-build`.


**Run integration tests:**

To run the integration tests, uncomment the following line from `tox.ini` and add the paths to your Kafka and Confluent Schema Registry instances. You can also run the integration tests outside of `tox` by running this command from the source root.

examples/integration_test.py <kafka-broker> [<test-topic>] [<schema-registry>]

**WARNING**: These tests require an active Kafka cluster and will create new topics.
**Integration tests:**

See [tests/README.md](tests/README.md) for instructions on how to run integration tests.



Expand Down
29 changes: 27 additions & 2 deletions confluent_kafka/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__all__ = ['cimpl', 'avro', 'kafkatest']
__all__ = ['cimpl', 'admin', 'avro', 'kafkatest']
from .cimpl import (Consumer, # noqa
KafkaError,
KafkaException,
Expand All @@ -15,5 +15,30 @@
OFFSET_STORED,
OFFSET_INVALID)


__version__ = version()[0]


class ThrottleEvent (object):
"""
ThrottleEvent contains details about a throttled request.
Set up a throttle callback by setting the ``throttle_cb`` configuration
property to a callable that takes a ThrottleEvent object as its only argument.
The callback will be triggered from poll(), consume() or flush() when a request
has been throttled by the broker.

This class is typically not user instantiated.

:ivar str broker_name: The hostname of the broker which throttled the request
:ivar int broker_id: The broker id
:ivar float throttle_time: The amount of time (in seconds) the broker throttled (delayed) the request
"""
def __init__(self, broker_name,
broker_id,
throttle_time):

self.broker_name = broker_name
self.broker_id = broker_id
self.throttle_time = throttle_time

def __str__(self):
return "{}/{} throttled for {} ms".format(self.broker_name, self.broker_id, int(self.throttle_time * 1000))
Loading