-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
initial implementation based on SSE client from Python SDK #1
Merged
Merged
Changes from 13 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
615539d
initial implementation based on SSE client from Python SDK
eli-darkly aaca796
makefile fixes
eli-darkly e2e87cd
rm debugging
eli-darkly e41d913
major refactoring & cleanup, better encapsulation of retry logic
eli-darkly 2b89408
misc fixes
eli-darkly c9b558f
move private methods to end
eli-darkly d06808b
remove restart method which can't work as currently implemented
eli-darkly 740389f
fix types
eli-darkly 718e334
update test deps
eli-darkly 638c8d2
comments
eli-darkly 4393d75
redefine jitter to be additive
eli-darkly 25b5c27
also ensure backoff is additive
eli-darkly 3c2d311
default jitter should be 100% since it is now additive
eli-darkly 675a1ad
rm unused
eli-darkly b9b7c02
only close the pool if we created it, not the opposite
eli-darkly 50cc1f9
Apply suggestions from code review
eli-darkly 5b73f76
fix shutdown of pool
eli-darkly ab43395
Merge branch 'eb/sc-133323/initial' of github.com:launchdarkly/python…
eli-darkly ad49637
add missing annotation
eli-darkly d24f44b
specify module names
eli-darkly 3858bd4
drop Python 3.6
eli-darkly 050f566
lint
eli-darkly b8c6a8b
lint
eli-darkly 9b3769c
lint
eli-darkly 4f6a133
lint
eli-darkly 9f4c6af
better lint command
eli-darkly File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
version: 2.1 | ||
|
||
orbs: | ||
win: circleci/windows@1.0.0 | ||
|
||
workflows: | ||
test: | ||
jobs: | ||
- test-linux: | ||
name: Python 3.6 | ||
docker-image: cimg/python:3.6 | ||
- test-linux: | ||
name: Python 3.7 | ||
docker-image: cimg/python:3.7 | ||
- test-linux: | ||
name: Python 3.8 | ||
docker-image: cimg/python:3.8 | ||
- test-linux: | ||
name: Python 3.9 | ||
docker-image: cimg/python:3.9 | ||
- test-linux: | ||
name: Python 3.10 | ||
docker-image: cimg/python:3.10 | ||
- test-windows: | ||
name: Windows Python 3 | ||
|
||
jobs: | ||
test-linux: | ||
parameters: | ||
docker-image: | ||
type: string | ||
docker: | ||
- image: <<parameters.docker-image>> | ||
steps: | ||
- checkout | ||
- run: | ||
name: install requirements | ||
command: | | ||
pip install -r test-requirements.txt | ||
python setup.py install | ||
pip freeze | ||
- run: | ||
name: run tests | ||
command: | | ||
mkdir test-reports | ||
pytest -s --junitxml=test-reports/junit.xml testing -W error::SyntaxWarning | ||
- run: | ||
name: lint | ||
command: make lint | ||
|
||
- run: | ||
name: build SSE contract test service | ||
command: make build-contract-test-service | ||
- run: | ||
name: start SSE contract test service | ||
command: make start-contract-test-service | ||
background: true | ||
- run: | ||
name: run SSE contract tests | ||
command: make run-contract-tests | ||
|
||
- store_test_results: | ||
path: test-reports | ||
- store_artifacts: | ||
path: test-reports | ||
|
||
test-windows: | ||
executor: | ||
name: win/vs2019 | ||
shell: powershell.exe | ||
steps: | ||
- checkout | ||
- run: | ||
name: install Python 3 | ||
command: choco install python --no-progress | ||
- run: | ||
name: install requirements | ||
command: | | ||
python --version | ||
pip install -r test-requirements.txt | ||
python setup.py install | ||
- run: | ||
name: run tests | ||
command: | | ||
mkdir test-reports | ||
python -m pytest -s --junitxml=test-reports/junit.xml testing; | ||
- store_test_results: | ||
path: test-reports | ||
- store_artifacts: | ||
path: test-reports |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -45,6 +45,7 @@ coverage.xml | |
*,cover | ||
.hypothesis/ | ||
.pytest_cache | ||
.mypy_cache | ||
|
||
# Translations | ||
*.mo | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
version: 2 | ||
|
||
publications: | ||
- url: https://pypi.org/project/launchdarkly-eventsource/ | ||
description: PyPI | ||
|
||
jobs: | ||
- docker: {} | ||
template: | ||
name: python |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Change log | ||
|
||
All notable changes to the LaunchDarkly SSE Client for Python will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# Contributing to this project | ||
|
||
## Submitting bug reports and feature requests | ||
|
||
The LaunchDarkly SDK team monitors the [issue tracker](https://github.com/launchdarkly/python-eventsource/issues) in this repository. Bug reports and feature requests specific to this project should be filed in this issue tracker. The SDK team will respond to all newly filed issues within two business days. | ||
|
||
Some of this code is used by the LaunchDarkly Python SDK. For issues or requests that are more generally related to the LaunchDarkly Python SDK, rather than specifically for the code in this repository, please use the [`go-server-sdk`](https://github.com/launchdarkly/python-server-sdk) repository. | ||
|
||
## Submitting pull requests | ||
|
||
We encourage pull requests and other contributions from the community. Before submitting pull requests, ensure that all temporary or unintended code is removed. Don't worry about adding reviewers to the pull request; the LaunchDarkly SDK team will add themselves. The SDK team will acknowledge all pull requests within two business days. | ||
|
||
## Build instructions | ||
|
||
### Prerequisites | ||
|
||
This project should be developed against its minimum compatible version, Python 3.5. | ||
|
||
To install the runtime and test requirements: | ||
|
||
``` | ||
pip install -r requirements.txt | ||
pip install -r test-requirements.txt | ||
``` | ||
|
||
### Testing | ||
|
||
To run all unit tests: | ||
|
||
``` | ||
pytest | ||
``` | ||
|
||
To run the standardized contract tests that are run against all LaunchDarkly SSE client implementations: | ||
``` | ||
make contract-tests | ||
``` | ||
|
||
### Linting | ||
|
||
To run the linter and check type hints: | ||
|
||
``` | ||
make lint | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
|
||
lint: | ||
mypy --config-file mypy.ini ld_eventsource testing | ||
|
||
.PHONY: lint | ||
|
||
TEMP_TEST_OUTPUT=/tmp/sse-contract-test-service.log | ||
|
||
build-contract-test-service: | ||
@cd contract-tests && pip install -r requirements.txt | ||
|
||
start-contract-test-service: | ||
@cd contract-tests && python service.py | ||
|
||
start-contract-test-service-bg: | ||
@echo "Test service output will be captured in $(TEMP_TEST_OUTPUT)" | ||
@make start-contract-test-service >$(TEMP_TEST_OUTPUT) 2>&1 & | ||
|
||
run-contract-tests: | ||
@curl -s https://raw.githubusercontent.com/launchdarkly/sse-contract-tests/v2.0.0/downloader/run.sh \ | ||
| VERSION=v2 PARAMS="-url http://localhost:8000 -debug -stop-service-at-end" sh | ||
|
||
contract-tests: build-contract-test-service start-contract-test-service-bg run-contract-tests | ||
|
||
.PHONY: build-contract-test-service start-contract-test-service run-contract-tests contract-tests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
# LaunchDarkly SSE Client for Python | ||
|
||
[![Circle CI](https://img.shields.io/circleci/project/launchdarkly/python-eventsource.png)](https://circleci.com/gh/launchdarkly/python-eventsource) | ||
[![PyPI](https://img.shields.io/pypi/v/launchdarkly-eventsource.svg?maxAge=2592000)](https://pypi.python.org/pypi/launchdarkly-eventsource) | ||
[![Documentation](https://img.shields.io/static/v1?label=GitHub+Pages&message=API+reference&color=00add8)](https://launchdarkly.github.io/python-eventsource) | ||
|
||
## Overview | ||
|
||
The `launchdarkly/python-eventsource` package allows Python developers to consume Server-Sent-Events (SSE) from a remote API. The SSE specification is defined here: [https://html.spec.whatwg.org/multipage/server-sent-events.html](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) | ||
|
||
This package's primary purpose is to support the [LaunchDarkly SDK for Python](https://github.com/launchdarkly/python-server-sdk), but it can be used independently. In its simplest configuration, it emulates the behavior of the EventSource API as defined in the SSE specification, with the addition of exponential backoff behavior for retries. However, it also includes optional features used by LaunchDarkly SDKs that are not part of the core specification, such as: | ||
|
||
* Customizing the backoff/jitter behavior. | ||
* Setting read timeouts, custom headers, and other HTTP request properties. | ||
* Specifying that connections should be retried under circumstances where the standard EventSource behavior would not retry them, such as if the server returns an HTTP error status. | ||
|
||
This is a synchronous implementation which blocks the caller's thread when reading events or reconnecting. | ||
|
||
## Supported Python versions | ||
|
||
This version of the package is compatible with Python 3.5 through 3.10. It is tested with the most recent patch releases of those versions. | ||
|
||
## Contributing | ||
|
||
We encourage pull requests and other contributions from the community. Check out our [contributing guidelines](CONTRIBUTING.md) for instructions on how to contribute to this SDK. | ||
|
||
## About LaunchDarkly | ||
|
||
* LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can: | ||
* Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases. | ||
* Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?). | ||
* Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file. | ||
* Grant access to certain features based on user attributes, like payment plan (eg: users on the ‘gold’ plan get access to more features than users in the ‘silver’ plan). Disable parts of your application to facilitate maintenance, without taking everything offline. | ||
* LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Read [our documentation](https://docs.launchdarkly.com/sdk) for a complete list. | ||
* Explore LaunchDarkly | ||
* [launchdarkly.com](https://www.launchdarkly.com/ "LaunchDarkly Main Website") for more information | ||
* [docs.launchdarkly.com](https://docs.launchdarkly.com/ "LaunchDarkly Documentation") for our documentation and SDK reference guides | ||
* [apidocs.launchdarkly.com](https://apidocs.launchdarkly.com/ "LaunchDarkly API Documentation") for our API documentation | ||
* [blog.launchdarkly.com](https://blog.launchdarkly.com/ "LaunchDarkly Blog Documentation") for the latest product updates |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# SSE client contract test service | ||
|
||
This directory contains an implementation of the cross-platform SSE testing protocol defined by https://github.com/launchdarkly/sse-contract-tests. See that project's `README` for details of this protocol, and the kinds of SSE client capabilities that are relevant to the contract tests. This code should not need to be updated unless the SSE client has added or removed such capabilities. | ||
|
||
To run these tests locally, run `make contract-tests`. This downloads the correct version of the test harness tool automatically. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
Flask==2.0.2 | ||
urllib3>=1.22.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
from stream_entity import StreamEntity | ||
|
||
import json | ||
import logging | ||
import os | ||
import sys | ||
import urllib3 | ||
from flask import Flask, request | ||
from flask.logging import default_handler | ||
from logging.config import dictConfig | ||
|
||
default_port = 8000 | ||
|
||
# logging configuration | ||
dictConfig({ | ||
'version': 1, | ||
'formatters': { | ||
'default': { | ||
'format': '[%(asctime)s] [%(name)s] %(levelname)s: %(message)s', | ||
} | ||
}, | ||
'handlers': { | ||
'console': { | ||
'class': 'logging.StreamHandler', | ||
'formatter': 'default' | ||
} | ||
}, | ||
'root': { | ||
'level': 'INFO', | ||
'handlers': ['console'] | ||
}, | ||
'loggers': { | ||
'werkzeug': { 'level': 'ERROR' } # disable irrelevant Flask app logging | ||
} | ||
}) | ||
|
||
app = Flask(__name__) | ||
app.logger.removeHandler(default_handler) | ||
|
||
stream_counter = 0 | ||
streams = {} | ||
global_log = logging.getLogger('testservice') | ||
|
||
http_client = urllib3.PoolManager() | ||
eli-darkly marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
@app.route('/', methods=['GET']) | ||
def status(): | ||
body = { | ||
'capabilities': [ | ||
'comments', | ||
'headers', | ||
'last-event-id', | ||
'read-timeout' | ||
] | ||
} | ||
return (json.dumps(body), 200, {'Content-type': 'application/json'}) | ||
|
||
@app.route('/', methods=['DELETE']) | ||
def delete_stop_service(): | ||
global_log.info("Test service has told us to exit") | ||
os._exit(0) | ||
|
||
@app.route('/', methods=['POST']) | ||
def post_create_stream(): | ||
global stream_counter, streams | ||
|
||
options = json.loads(request.data) | ||
|
||
stream_counter += 1 | ||
stream_id = str(stream_counter) | ||
resource_url = '/streams/%s' % stream_id | ||
|
||
stream = StreamEntity(options) | ||
streams[stream_id] = stream | ||
|
||
return ('', 201, {'Location': resource_url}) | ||
|
||
@app.route('/streams/<id>', methods=['POST']) | ||
def post_stream_command(id): | ||
global streams | ||
|
||
params = json.loads(request.data) | ||
|
||
stream = streams[id] | ||
if stream is None: | ||
return ('', 404) | ||
if not stream.do_command(params.get('command')): | ||
return ('', 400) | ||
return ('', 204) | ||
|
||
@app.route('/streams/<id>', methods=['DELETE']) | ||
def delete_stream(id): | ||
global streams | ||
|
||
stream = streams[id] | ||
if stream is None: | ||
return ('', 404) | ||
stream.close() | ||
return ('', 204) | ||
|
||
if __name__ == "__main__": | ||
port = default_port | ||
if sys.argv[len(sys.argv) - 1] != 'service.py': | ||
port = int(sys.argv[len(sys.argv) - 1]) | ||
global_log.info('Listening on port %d', port) | ||
app.run(host='0.0.0.0', port=port) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a real shame. Can't wait to bump that up in the SDK and then subsequently here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, 3.5 and 3.6 are now EOL. As I understand it, we still don't want to drop support for them till the next major version of the Python SDK... but, if we don't start using
python-eventsource
until the next major version of the SDK, then we can drop 3.5 and 3.6 from this project now.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are there things from 3.7+ that you think would be particularly helpful in this code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I wasn't suggesting it because of that. I was more thinking it's a shame to have to support to EOL releases with something completely new, but since it's going to be used in the SDK I totally get that.
I think it is fine to leave it like this for now so you can start using it right away and then we can drop support later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...In fact, we should have (or at least could have) already dropped 3.5 in our 7.0.0 release, since it had already been EOL for a month by then. I think that was probably just the result of having started that work a few months earlier.