Skip to content

Http structured cloudevents #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Jun 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a124d61
Moved fields out of base & structured support
Jun 24, 2020
a704f9e
testing structured
Jun 25, 2020
fff8d4d
added tests for structured events
Jun 25, 2020
38ef53c
Added test valid structured cloudevents
Jun 25, 2020
09d9eb3
Created default headers arg in CloudEvent
Jun 25, 2020
919d162
Added http_events.py sample code
Jun 25, 2020
3b4441f
removed ../python-event-requests
Jun 25, 2020
78616b1
README.md nit
Jun 25, 2020
fa91be9
client.py nit
Jun 25, 2020
138a786
comment nits
Jun 25, 2020
72662ba
created __getitem__ in CloudEvent
Jun 25, 2020
dec013c
sample nits
Jun 25, 2020
2a09062
fixed structured empty data issue
Jun 25, 2020
d32da95
Added CloudEvent to README
Jun 25, 2020
431b15c
added http_msg to CloudEvent
Jun 25, 2020
98f9af5
implemented ToRequest in CloudEvent
Jun 25, 2020
b6bef8e
testing more specversions
Jun 25, 2020
b93ed54
Added sample code to README.md
Jun 26, 2020
88dff83
modified sample code
Jun 26, 2020
ac25f46
added datavalidation to changelog
Jun 26, 2020
f1565e5
updated README
Jun 26, 2020
6f3ca78
README adjustment
Jun 26, 2020
a050785
ruler 80 adjustment on http_events
Jun 26, 2020
50cedf5
style and renamed ToRequest to to_request
Jun 26, 2020
e8e875e
lint fix
Jun 26, 2020
e09bb1a
fixed self.binary typo
Jun 26, 2020
7595ab2
CHANGELOG adjustment
Jun 26, 2020
6623756
rollback CHANGELOG
Jun 26, 2020
590d33e
Added documentation to to_request
Jun 26, 2020
e2a1177
README.md adjustment
Jun 26, 2020
baa84fa
renamed event_handler to event_version
Jun 26, 2020
8bfa5d1
inlined field_name_modifier
Jun 26, 2020
8b10172
renamed test body data
Jun 26, 2020
68aed6c
removed unnecessary headers from test
Jun 26, 2020
3a763c1
removed field_name_modifier and fixed e.g. in client.py
Jun 26, 2020
84992e5
pylint fix
Jun 26, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#22]: https://github.com/cloudevents/sdk-python/pull/22
[#23]: https://github.com/cloudevents/sdk-python/pull/23
[#25]: https://github.com/cloudevents/sdk-python/pull/25
[#27]: https://github.com/cloudevents/sdk-python/pull/27
[#27]: https://github.com/cloudevents/sdk-python/pull/27
67 changes: 51 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,57 @@ This SDK current supports the following versions of CloudEvents:

Package **cloudevents** provides primitives to work with CloudEvents specification: https://github.com/cloudevents/spec.

Sending CloudEvents:

### Binary HTTP CloudEvent

```python
from cloudevents.sdk.http_events import CloudEvent
import requests


# This data defines a binary cloudevent
headers = {
"Content-Type": "application/json",
"ce-specversion": "1.0",
"ce-type": "README.sample.binary",
"ce-id": "binary-event",
"ce-time": "2018-10-23T12:28:22.4579346Z",
"ce-source": "README",
}
data = {"message": "Hello World!"}

event = CloudEvent(data, headers=headers)
headers, body = event.to_request()

# POST
requests.post("<some-url>", json=body, headers=headers)
```

### Structured HTTP CloudEvent

```python
from cloudevents.sdk.http_events import CloudEvent
import requests


# This data defines a structured cloudevent
data = {
"specversion": "1.0",
"type": "README.sample.structured",
"id": "structured-event",
"source": "README",
"data": {"message": "Hello World!"}
}
event = CloudEvent(data)
headers, body = event.to_request()

# POST
requests.post("<some-url>", json=body, headers=headers)
```

### Event base classes usage

Parsing upstream structured Event from HTTP request:

```python
Expand Down Expand Up @@ -68,22 +119,6 @@ event = m.FromRequest(
)
```

Creating a minimal CloudEvent in version 0.1:

```python
from cloudevents.sdk.event import v1

event = (
v1.Event()
.SetContentType("application/json")
.SetData('{"name":"john"}')
.SetEventID("my-id")
.SetSource("from-galaxy-far-far-away")
.SetEventTime("tomorrow")
.SetEventType("cloudevent.greet.you")
)
```

Creating HTTP request from CloudEvent:

```python
Expand Down
15 changes: 0 additions & 15 deletions cloudevents/sdk/event/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,6 @@
import json
import typing

_ce_required_fields = {
'id',
'source',
'type',
'specversion'
}


_ce_optional_fields = {
'datacontenttype',
'schema',
'subject',
'time'
}


# TODO(slinkydeveloper) is this really needed?
class EventGetterSetter(object):
Expand Down
15 changes: 15 additions & 0 deletions cloudevents/sdk/event/v03.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,21 @@


class Event(base.BaseEvent):
_ce_required_fields = {
'id',
'source',
'type',
'specversion'
}

_ce_optional_fields = {
'datacontentencoding',
'datacontenttype',
'schemaurl',
'subject',
'time'
}

def __init__(self):
self.ce__specversion = opt.Option("specversion", "0.3", True)
self.ce__id = opt.Option("id", None, True)
Expand Down
14 changes: 14 additions & 0 deletions cloudevents/sdk/event/v1.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@


class Event(base.BaseEvent):
_ce_required_fields = {
'id',
'source',
'type',
'specversion'
}

_ce_optional_fields = {
'datacontenttype',
'dataschema',
'subject',
'time'
}

def __init__(self):
self.ce__specversion = opt.Option("specversion", "1.0", True)
self.ce__id = opt.Option("id", None, True)
Expand Down
151 changes: 104 additions & 47 deletions cloudevents/sdk/http_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,14 @@
# under the License.
import copy

import io

import json
import typing

from cloudevents.sdk import converters
from cloudevents.sdk import marshaller

from cloudevents.sdk.event import base
from cloudevents.sdk.event import v03, v1


Expand All @@ -30,12 +32,14 @@ class CloudEvent():

def __init__(
self,
headers: dict,
data: dict,
data_unmarshaller: typing.Callable = lambda x: x
data: typing.Union[dict, None],
headers: dict = {},
data_unmarshaller: typing.Callable = lambda x: x,
):
"""
Event HTTP Constructor
:param data: a nullable dict to be stored inside Event.
:type data: dict or None
:param headers: a dict with HTTP headers
e.g. {
"content-type": "application/cloudevents+json",
Expand All @@ -45,65 +49,118 @@ def __init__(
"ce-specversion": "0.2"
}
:type headers: dict
:param data: a dict to be stored inside Event
:type data: dict
:param binary: a bool indicating binary events
:type binary: bool
:param data_unmarshaller: callable function for reading/extracting data
:type data_unmarshaller: typing.Callable
"""
self.required_attribute_values = {}
self.optional_attribute_values = {}
if data is None:
data = {}

headers = {key.lower(): value for key, value in headers.items()}
data = {key.lower(): value for key, value in data.items()}
event_version = CloudEvent.detect_event_version(headers, data)
if CloudEvent.is_binary_cloud_event(headers):

# Headers validation for binary events
for field in base._ce_required_fields:
ce_prefixed_field = f"ce-{field}"

# Verify field exists else throw TypeError
if ce_prefixed_field not in headers:
raise TypeError(
"parameter headers has no required attribute {0}"
.format(
ce_prefixed_field
))

if not isinstance(headers[ce_prefixed_field], str):
raise TypeError(
"in parameter headers attribute "
"{0} expected type str but found type {1}".format(
ce_prefixed_field, type(headers[ce_prefixed_field])
))

for field in base._ce_optional_fields:
ce_prefixed_field = f"ce-{field}"
if ce_prefixed_field in headers and not \
isinstance(headers[ce_prefixed_field], str):
raise TypeError(
"in parameter headers attribute "
"{0} expected type str but found type {1}".format(
ce_prefixed_field, type(headers[ce_prefixed_field])
))

else:
# TODO: Support structured CloudEvents
raise NotImplementedError
# returns an event class depending on proper version
event_version = CloudEvent.detect_event_version(headers, data)
self.isbinary = CloudEvent.is_binary_cloud_event(
event_version,
headers
)

self.headers = copy.deepcopy(headers)
self.data = copy.deepcopy(data)
self.marshall = marshaller.NewDefaultHTTPMarshaller()
self.event_handler = event_version()
self.marshall.FromRequest(

self.__event = self.marshall.FromRequest(
self.event_handler,
self.headers,
self.data,
headers,
io.BytesIO(json.dumps(data).encode()),
data_unmarshaller
)

# headers validation for binary events
for field in event_version._ce_required_fields:

# prefixes with ce- if this is a binary event
fieldname = f"ce-{field}" if self.isbinary else field

# fields_refs holds a reference to where fields should be
fields_refs = headers if self.isbinary else data

fields_refs_name = 'headers' if self.isbinary else 'data'

# verify field exists else throw TypeError
if fieldname not in fields_refs:
raise TypeError(
f"parameter {fields_refs_name} has no required "
f"attribute {fieldname}."
)

elif not isinstance(fields_refs[fieldname], str):
raise TypeError(
f"in parameter {fields_refs_name}, {fieldname} "
f"expected type str but found type "
f"{type(fields_refs[fieldname])}."
)

else:
self.required_attribute_values[f"ce-{field}"] = \
fields_refs[fieldname]

for field in event_version._ce_optional_fields:
fieldname = f"ce-{field}" if self.isbinary else field
if (fieldname in fields_refs) and not \
isinstance(fields_refs[fieldname], str):
raise TypeError(
f"in parameter {fields_refs_name}, {fieldname} "
f"expected type str but found type "
f"{type(fields_refs[fieldname])}."
)
else:
self.optional_attribute_values[f"ce-{field}"] = field

# structured data is inside json resp['data']
self.data = copy.deepcopy(data) if self.isbinary else \
copy.deepcopy(data.get('data', {}))

self.headers = {
**self.required_attribute_values,
**self.optional_attribute_values
}

def to_request(
self,
data_unmarshaller: typing.Callable = lambda x: json.loads(
x.read()
.decode('utf-8')
)
) -> (dict, dict):
"""
Returns a tuple of HTTP headers/body dicts representing this cloudevent

:param data_unmarshaller: callable function used to read the data io
object
:type data_unmarshaller: typing.Callable
:returns: (http_headers: dict, http_body: dict)
"""
converter_type = converters.TypeBinary if self.isbinary else \
converters.TypeStructured

headers, data = self.marshall.ToRequest(
self.__event,
converter_type,
data_unmarshaller
)
data = data if self.isbinary else data_unmarshaller(data)['data']
return headers, data

def __getitem__(self, key):
return self.data if key == 'data' else self.headers[key]

@staticmethod
def is_binary_cloud_event(headers):
for field in base._ce_required_fields:
def is_binary_cloud_event(event_version, headers):
for field in event_version._ce_required_fields:
if f"ce-{field}" not in headers:
return False
return True
Expand Down
Loading