Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial implementation based on SSE client from Python SDK #1

Merged
merged 26 commits into from
Dec 29, 2022
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
615539d
initial implementation based on SSE client from Python SDK
eli-darkly Dec 30, 2021
aaca796
makefile fixes
eli-darkly Dec 30, 2021
e2e87cd
rm debugging
eli-darkly Dec 30, 2021
e41d913
major refactoring & cleanup, better encapsulation of retry logic
eli-darkly Jan 14, 2022
2b89408
misc fixes
eli-darkly Jan 14, 2022
c9b558f
move private methods to end
eli-darkly Jan 14, 2022
d06808b
remove restart method which can't work as currently implemented
eli-darkly Jan 19, 2022
740389f
fix types
eli-darkly Jan 19, 2022
718e334
update test deps
eli-darkly Jan 19, 2022
638c8d2
comments
eli-darkly Jan 19, 2022
4393d75
redefine jitter to be additive
eli-darkly Jan 19, 2022
25b5c27
also ensure backoff is additive
eli-darkly Jan 19, 2022
3c2d311
default jitter should be 100% since it is now additive
eli-darkly Jan 19, 2022
675a1ad
rm unused
eli-darkly Jan 19, 2022
b9b7c02
only close the pool if we created it, not the opposite
eli-darkly Jan 19, 2022
50cc1f9
Apply suggestions from code review
eli-darkly Jan 19, 2022
5b73f76
fix shutdown of pool
eli-darkly Jan 19, 2022
ab43395
Merge branch 'eb/sc-133323/initial' of github.com:launchdarkly/python…
eli-darkly Jan 19, 2022
ad49637
add missing annotation
eli-darkly Jan 19, 2022
d24f44b
specify module names
eli-darkly Jan 19, 2022
3858bd4
drop Python 3.6
eli-darkly Dec 29, 2022
050f566
lint
eli-darkly Dec 29, 2022
b8c6a8b
lint
eli-darkly Dec 29, 2022
9b3769c
lint
eli-darkly Dec 29, 2022
4f6a133
lint
eli-darkly Dec 29, 2022
9f4c6af
better lint command
eli-darkly Dec 29, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
version: 2.1

orbs:
win: circleci/windows@1.0.0

workflows:
test:
jobs:
- test-linux:
name: Python 3.6
docker-image: cimg/python:3.6
- test-linux:
name: Python 3.7
docker-image: cimg/python:3.7
- test-linux:
name: Python 3.8
docker-image: cimg/python:3.8
- test-linux:
name: Python 3.9
docker-image: cimg/python:3.9
- test-linux:
name: Python 3.10
docker-image: cimg/python:3.10
- test-windows:
name: Windows Python 3

jobs:
test-linux:
parameters:
docker-image:
type: string
docker:
- image: <<parameters.docker-image>>
steps:
- checkout
- run:
name: install requirements
command: |
pip install -r test-requirements.txt
python setup.py install
pip freeze
- run:
name: run tests
command: |
mkdir test-reports
pytest -s --junitxml=test-reports/junit.xml testing -W error::SyntaxWarning
- run:
name: lint
command: make lint

- run:
name: build SSE contract test service
command: make build-contract-test-service
- run:
name: start SSE contract test service
command: make start-contract-test-service
background: true
- run:
name: run SSE contract tests
command: make run-contract-tests

- store_test_results:
path: test-reports
- store_artifacts:
path: test-reports

test-windows:
executor:
name: win/vs2019
shell: powershell.exe
steps:
- checkout
- run:
name: install Python 3
command: choco install python --no-progress
- run:
name: install requirements
command: |
python --version
pip install -r test-requirements.txt
python setup.py install
- run:
name: run tests
command: |
mkdir test-reports
python -m pytest -s --junitxml=test-reports/junit.xml testing;
- store_test_results:
path: test-reports
- store_artifacts:
path: test-reports
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ coverage.xml
*,cover
.hypothesis/
.pytest_cache
.mypy_cache

# Translations
*.mo
Expand Down
10 changes: 10 additions & 0 deletions .ldrelease/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2

publications:
- url: https://pypi.org/project/launchdarkly-eventsource/
description: PyPI

jobs:
- docker: {}
template:
name: python
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Change log

All notable changes to the LaunchDarkly SSE Client for Python will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org).
45 changes: 45 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Contributing to this project

## Submitting bug reports and feature requests

The LaunchDarkly SDK team monitors the [issue tracker](https://github.com/launchdarkly/python-eventsource/issues) in this repository. Bug reports and feature requests specific to this project should be filed in this issue tracker. The SDK team will respond to all newly filed issues within two business days.

Some of this code is used by the LaunchDarkly Python SDK. For issues or requests that are more generally related to the LaunchDarkly Python SDK, rather than specifically for the code in this repository, please use the [`go-server-sdk`](https://github.com/launchdarkly/python-server-sdk) repository.

## Submitting pull requests

We encourage pull requests and other contributions from the community. Before submitting pull requests, ensure that all temporary or unintended code is removed. Don't worry about adding reviewers to the pull request; the LaunchDarkly SDK team will add themselves. The SDK team will acknowledge all pull requests within two business days.

## Build instructions

### Prerequisites

This project should be developed against its minimum compatible version, Python 3.5.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a real shame. Can't wait to bump that up in the SDK and then subsequently here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, 3.5 and 3.6 are now EOL. As I understand it, we still don't want to drop support for them till the next major version of the Python SDK... but, if we don't start using python-eventsource until the next major version of the SDK, then we can drop 3.5 and 3.6 from this project now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there things from 3.7+ that you think would be particularly helpful in this code?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I wasn't suggesting it because of that. I was more thinking it's a shame to have to support to EOL releases with something completely new, but since it's going to be used in the SDK I totally get that.

I think it is fine to leave it like this for now so you can start using it right away and then we can drop support later.

Copy link
Contributor Author

@eli-darkly eli-darkly Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...In fact, we should have (or at least could have) already dropped 3.5 in our 7.0.0 release, since it had already been EOL for a month by then. I think that was probably just the result of having started that work a few months earlier.


To install the runtime and test requirements:

```
pip install -r requirements.txt
pip install -r test-requirements.txt
```

### Testing

To run all unit tests:

```
pytest
```

To run the standardized contract tests that are run against all LaunchDarkly SSE client implementations:
```
make contract-tests
```

### Linting

To run the linter and check type hints:

```
make lint
```
25 changes: 25 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@

lint:
mypy --config-file mypy.ini ld_eventsource testing

.PHONY: lint

TEMP_TEST_OUTPUT=/tmp/sse-contract-test-service.log

build-contract-test-service:
@cd contract-tests && pip install -r requirements.txt

start-contract-test-service:
@cd contract-tests && python service.py

start-contract-test-service-bg:
@echo "Test service output will be captured in $(TEMP_TEST_OUTPUT)"
@make start-contract-test-service >$(TEMP_TEST_OUTPUT) 2>&1 &

run-contract-tests:
@curl -s https://raw.githubusercontent.com/launchdarkly/sse-contract-tests/v2.0.0/downloader/run.sh \
| VERSION=v2 PARAMS="-url http://localhost:8000 -debug -stop-service-at-end" sh

contract-tests: build-contract-test-service start-contract-test-service-bg run-contract-tests

.PHONY: build-contract-test-service start-contract-test-service run-contract-tests contract-tests
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# LaunchDarkly SSE Client for Python

[![Circle CI](https://img.shields.io/circleci/project/launchdarkly/python-eventsource.png)](https://circleci.com/gh/launchdarkly/python-eventsource)
[![PyPI](https://img.shields.io/pypi/v/launchdarkly-eventsource.svg?maxAge=2592000)](https://pypi.python.org/pypi/launchdarkly-eventsource)
[![Documentation](https://img.shields.io/static/v1?label=GitHub+Pages&message=API+reference&color=00add8)](https://launchdarkly.github.io/python-eventsource)

## Overview

The `launchdarkly/python-eventsource` package allows Python developers to consume Server-Sent-Events (SSE) from a remote API. The SSE specification is defined here: [https://html.spec.whatwg.org/multipage/server-sent-events.html](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events)

This package's primary purpose is to support the [LaunchDarkly SDK for Python](https://github.com/launchdarkly/python-server-sdk), but it can be used independently. In its simplest configuration, it emulates the behavior of the EventSource API as defined in the SSE specification, with the addition of exponential backoff behavior for retries. However, it also includes optional features used by LaunchDarkly SDKs that are not part of the core specification, such as:

* Customizing the backoff/jitter behavior.
* Setting read timeouts, custom headers, and other HTTP request properties.
* Specifying that connections should be retried under circumstances where the standard EventSource behavior would not retry them, such as if the server returns an HTTP error status.

This is a synchronous implementation which blocks the caller's thread when reading events or reconnecting.

## Supported Python versions

This version of the package is compatible with Python 3.5 through 3.10. It is tested with the most recent patch releases of those versions.

## Contributing

We encourage pull requests and other contributions from the community. Check out our [contributing guidelines](CONTRIBUTING.md) for instructions on how to contribute to this SDK.

## About LaunchDarkly

* LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can:
* Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases.
* Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?).
* Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file.
* Grant access to certain features based on user attributes, like payment plan (eg: users on the ‘gold’ plan get access to more features than users in the ‘silver’ plan). Disable parts of your application to facilitate maintenance, without taking everything offline.
* LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Read [our documentation](https://docs.launchdarkly.com/sdk) for a complete list.
* Explore LaunchDarkly
* [launchdarkly.com](https://www.launchdarkly.com/ "LaunchDarkly Main Website") for more information
* [docs.launchdarkly.com](https://docs.launchdarkly.com/ "LaunchDarkly Documentation") for our documentation and SDK reference guides
* [apidocs.launchdarkly.com](https://apidocs.launchdarkly.com/ "LaunchDarkly API Documentation") for our API documentation
* [blog.launchdarkly.com](https://blog.launchdarkly.com/ "LaunchDarkly Blog Documentation") for the latest product updates
5 changes: 5 additions & 0 deletions contract-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SSE client contract test service

This directory contains an implementation of the cross-platform SSE testing protocol defined by https://github.com/launchdarkly/sse-contract-tests. See that project's `README` for details of this protocol, and the kinds of SSE client capabilities that are relevant to the contract tests. This code should not need to be updated unless the SSE client has added or removed such capabilities.

To run these tests locally, run `make contract-tests`. This downloads the correct version of the test harness tool automatically.
2 changes: 2 additions & 0 deletions contract-tests/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Flask==2.0.2
urllib3>=1.22.0
106 changes: 106 additions & 0 deletions contract-tests/service.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
from stream_entity import StreamEntity

import json
import logging
import os
import sys
import urllib3
from flask import Flask, request
from flask.logging import default_handler
from logging.config import dictConfig

default_port = 8000

# logging configuration
dictConfig({
'version': 1,
'formatters': {
'default': {
'format': '[%(asctime)s] [%(name)s] %(levelname)s: %(message)s',
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'default'
}
},
'root': {
'level': 'INFO',
'handlers': ['console']
},
'loggers': {
'werkzeug': { 'level': 'ERROR' } # disable irrelevant Flask app logging
}
})

app = Flask(__name__)
app.logger.removeHandler(default_handler)

stream_counter = 0
streams = {}
global_log = logging.getLogger('testservice')

http_client = urllib3.PoolManager()
eli-darkly marked this conversation as resolved.
Show resolved Hide resolved

@app.route('/', methods=['GET'])
def status():
body = {
'capabilities': [
'comments',
'headers',
'last-event-id',
'read-timeout'
]
}
return (json.dumps(body), 200, {'Content-type': 'application/json'})

@app.route('/', methods=['DELETE'])
def delete_stop_service():
global_log.info("Test service has told us to exit")
os._exit(0)

@app.route('/', methods=['POST'])
def post_create_stream():
global stream_counter, streams

options = json.loads(request.data)

stream_counter += 1
stream_id = str(stream_counter)
resource_url = '/streams/%s' % stream_id

stream = StreamEntity(options)
streams[stream_id] = stream

return ('', 201, {'Location': resource_url})

@app.route('/streams/<id>', methods=['POST'])
def post_stream_command(id):
global streams

params = json.loads(request.data)

stream = streams[id]
if stream is None:
return ('', 404)
if not stream.do_command(params.get('command')):
return ('', 400)
return ('', 204)

@app.route('/streams/<id>', methods=['DELETE'])
def delete_stream(id):
global streams

stream = streams[id]
if stream is None:
return ('', 404)
stream.close()
return ('', 204)

if __name__ == "__main__":
port = default_port
if sys.argv[len(sys.argv) - 1] != 'service.py':
port = int(sys.argv[len(sys.argv) - 1])
global_log.info('Listening on port %d', port)
app.run(host='0.0.0.0', port=port)
Loading