Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: create abstract cloudevent #186

Merged
merged 51 commits into from
Aug 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
9422e62
fix: non-cloudevents values must not equal to cloudevents values (#171)
sasha-tkachev Jul 11, 2022
6c6508b
test: refactor move fixtures to beginning
sasha-tkachev Jul 11, 2022
494170b
test: cloudevent equality bug regression (#171)
sasha-tkachev Jul 11, 2022
202c987
style: remove redundent else
sasha-tkachev Jul 12, 2022
8f98d65
test: remove redundent test
sasha-tkachev Jul 12, 2022
781d914
test: refactor non_cloudevent_value into a parameterization
sasha-tkachev Jul 12, 2022
fe3a2d0
docs: update changelog
sasha-tkachev Jul 12, 2022
188e7b7
Merge branch 'master' into master
sasha-tkachev Jul 12, 2022
39c838a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 12, 2022
5c0af53
docs: fix bad merge
sasha-tkachev Jul 12, 2022
f55d247
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 12, 2022
aaa8098
Merge branch 'master' into master
sasha-tkachev Jul 13, 2022
207c4a8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 13, 2022
41a9af2
feat: abstract event
sasha-tkachev Jul 22, 2022
5f8399f
feat: add missing return type
sasha-tkachev Jul 22, 2022
2b64830
feat: create function
sasha-tkachev Jul 22, 2022
4488201
feat: any cloud event
sasha-tkachev Jul 22, 2022
f1ff009
refactor: move to abstract
sasha-tkachev Jul 22, 2022
c747f59
refactor: integrate abstract event
sasha-tkachev Jul 22, 2022
6588577
refactor: create abstract cloudevent package
sasha-tkachev Jul 22, 2022
01041e7
docs: abstract cloudevent
sasha-tkachev Jul 22, 2022
5648968
feat: simplify data attributes
sasha-tkachev Jul 22, 2022
09062e3
fix: intengrate data read model
sasha-tkachev Jul 22, 2022
a96bd6c
feat: define abstract methods
sasha-tkachev Jul 22, 2022
3845aa7
refactor: use anycloudevent for generics
sasha-tkachev Jul 24, 2022
fb4f993
docs: getitem documentation
sasha-tkachev Jul 24, 2022
1187600
docs: better cloudevent explenation
sasha-tkachev Jul 24, 2022
ecf9418
docs: explain read model
sasha-tkachev Jul 24, 2022
b9e8763
docs: not implemented errors
sasha-tkachev Jul 24, 2022
62595ff
docs: explain why impl has no public attributes property
sasha-tkachev Jul 24, 2022
2b3c0f1
docs: add missing comment to from_http
sasha-tkachev Jul 24, 2022
a22efbd
test: add abstract cloudevent coverage tests
sasha-tkachev Jul 24, 2022
89d30eb
refactor: rename abstract to generic
sasha-tkachev Jul 24, 2022
d303eae
Merge branch 'main' into feature/abstract-cloudevent
sasha-tkachev Jul 24, 2022
ba16cdd
refactor: cloudevent is no longer absctract
sasha-tkachev Jul 24, 2022
ea19f7d
test: fix broken test
sasha-tkachev Jul 24, 2022
f47087d
Revert "refactor: rename abstract to generic"
sasha-tkachev Jul 24, 2022
41c5f59
refactor: move all abstract conversion logic under conversion
sasha-tkachev Jul 24, 2022
5d0882d
test: rename badly named test
sasha-tkachev Jul 24, 2022
6a92016
refactor: add default value for conversions
sasha-tkachev Jul 24, 2022
1109bc5
docs: remove inconsistent types
sasha-tkachev Jul 24, 2022
c0b5413
refactor: remove mutation variables from contract
sasha-tkachev Jul 24, 2022
065ef91
refactor: expose data and attributes in class
sasha-tkachev Jul 24, 2022
a8872b9
test: remove broken tests
sasha-tkachev Jul 24, 2022
fbc0632
refactor: use classmethods
sasha-tkachev Jul 24, 2022
0c2bafc
refactor: remove optional type
sasha-tkachev Jul 24, 2022
4e00b55
refactor: convert get_data and get_attributes to private member funct…
sasha-tkachev Aug 5, 2022
96c41a1
build: ignore not-implemented functions in coverage
sasha-tkachev Aug 5, 2022
d129e6a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2022
48de8e1
docs: mentioned default branch change in the changelog
xSAVIKx Aug 6, 2022
26e551b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[report]
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover

# Don't complain if tests don't hit defensive assertion code:
raise NotImplementedError
15 changes: 15 additions & 0 deletions cloudevents/abstract/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Copyright 2018-Present The CloudEvents Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

from cloudevents.abstract.event import AnyCloudEvent, CloudEvent # noqa
xSAVIKx marked this conversation as resolved.
Show resolved Hide resolved
137 changes: 137 additions & 0 deletions cloudevents/abstract/event.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Copyright 2018-Present The CloudEvents Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

import typing
from abc import abstractmethod
from typing import TypeVar


class CloudEvent:
"""
The CloudEvent Python wrapper contract exposing generically-available
properties and APIs.

Implementations might handle fields and have other APIs exposed but are
obliged to follow this contract.
"""

@classmethod
def create(
cls,
attributes: typing.Dict[str, typing.Any],
data: typing.Optional[typing.Any],
) -> "AnyCloudEvent":
"""
Creates a new instance of the CloudEvent using supplied `attributes`
and `data`.

This method should be preferably used over the constructor to create events
while custom framework-specific implementations may require or assume
different arguments.

:param attributes: The attributes of the CloudEvent instance.
:param data: The payload of the CloudEvent instance.
:returns: A new instance of the CloudEvent created from the passed arguments.
"""
raise NotImplementedError()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we mark this method as abstract?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because it causes issues in the code coverage tests.
I cannot trigger the line to be called because an abstract method call fails before the function is called

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sure, but that's not a valid reason to make smth not abstract. You can just create a test no-op or error-throwing implementation directly in the test case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried testing the code directly but it did not actually executed the pass line of the function

@abstractmethod
def dostuff():
  pass

because the abstract method raised an exception before reaching the pass.

I will do the thing that is mentioned here
https://stackoverflow.com/questions/9202723/excluding-abstractproperties-from-coverage-reports


@abstractmethod
def _get_attributes(self) -> typing.Dict[str, typing.Any]:
"""
Returns the attributes of the event.

The implementation MUST assume that the returned value MAY be mutated.

Having a function over a property simplifies integration for custom
framework-specific implementations.

:returns: Attributes of the event.
"""
raise NotImplementedError()

@abstractmethod
def _get_data(self) -> typing.Optional[typing.Any]:
"""
Returns the data of the event.

The implementation MUST assume that the returned value MAY be mutated.

Having a function over a property simplifies integration for custom
framework-specific implementations.

:returns: Data of the event.
"""
raise NotImplementedError()
sasha-tkachev marked this conversation as resolved.
Show resolved Hide resolved

def __eq__(self, other: typing.Any) -> bool:
if isinstance(other, CloudEvent):
same_data = self._get_data() == other._get_data()
same_attributes = self._get_attributes() == other._get_attributes()
return same_data and same_attributes
return False

def __getitem__(self, key: str) -> typing.Any:
"""
Returns a value of an attribute of the event denoted by the given `key`.

The `data` of the event should be accessed by the `.data` accessor rather
than this mapping.

:param key: The name of the event attribute to retrieve the value for.
:returns: The event attribute value.
"""
return self._get_attributes()[key]

def get(
self, key: str, default: typing.Optional[typing.Any] = None
) -> typing.Optional[typing.Any]:
"""
Retrieves an event attribute value for the given `key`.

Returns the `default` value if the attribute for the given key does not exist.

The implementation MUST NOT throw an error when the key does not exist, but
rather should return `None` or the configured `default`.

:param key: The name of the event attribute to retrieve the value for.
:param default: The default value to be returned when
no attribute with the given key exists.
:returns: The event attribute value if exists, default value or None otherwise.
"""
return self._get_attributes().get(key, default)

def __iter__(self) -> typing.Iterator[typing.Any]:
xSAVIKx marked this conversation as resolved.
Show resolved Hide resolved
"""
Returns an iterator over the event attributes.
"""
return iter(self._get_attributes())

def __len__(self) -> int:
"""
Returns the number of the event attributes.
"""
return len(self._get_attributes())
xSAVIKx marked this conversation as resolved.
Show resolved Hide resolved

def __contains__(self, key: str) -> bool:
xSAVIKx marked this conversation as resolved.
Show resolved Hide resolved
"""
Determines if an attribute with a given `key` is present
in the event attributes.
"""
return key in self._get_attributes()

def __repr__(self) -> str:
return str({"attributes": self._get_attributes(), "data": self._get_data()})


AnyCloudEvent = TypeVar("AnyCloudEvent", bound=CloudEvent)
224 changes: 224 additions & 0 deletions cloudevents/conversion.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# Copyright 2018-Present The CloudEvents Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
import typing

from cloudevents import exceptions as cloud_exceptions
from cloudevents.abstract import AnyCloudEvent
from cloudevents.http import is_binary
from cloudevents.http.mappings import _marshaller_by_format, _obj_by_version
from cloudevents.http.util import _json_or_string
from cloudevents.sdk import converters, marshaller, types


def to_json(
event: AnyCloudEvent,
data_marshaller: types.MarshallerType = None,
) -> typing.Union[str, bytes]:
"""
Converts given `event` to a JSON string.

:param event: A CloudEvent to be converted into a JSON string.
:param data_marshaller: Callable function which will cast `event.data`
into a JSON string.
:returns: A JSON string representing the given event.
"""
return to_structured(event, data_marshaller=data_marshaller)[1]


def from_json(
event_type: typing.Type[AnyCloudEvent],
data: typing.Union[str, bytes],
data_unmarshaller: types.UnmarshallerType = None,
) -> AnyCloudEvent:
"""
Parses JSON string `data` into a CloudEvent.

:param data: JSON string representation of a CloudEvent.
:param data_unmarshaller: Callable function that casts `data` to a
Python object.
:param event_type: A concrete type of the event into which the data is
deserialized.
:returns: A CloudEvent parsed from the given JSON representation.
"""
return from_http(
headers={},
data=data,
data_unmarshaller=data_unmarshaller,
event_type=event_type,
)


def from_http(
event_type: typing.Type[AnyCloudEvent],
headers: typing.Dict[str, str],
data: typing.Union[str, bytes, None],
data_unmarshaller: types.UnmarshallerType = None,
) -> AnyCloudEvent:
"""
Parses CloudEvent `data` and `headers` into an instance of a given `event_type`.

The method supports both binary and structured representations.

:param headers: The HTTP request headers.
:param data: The HTTP request body. If set to None, "" or b'', the returned
event's `data` field will be set to None.
:param data_unmarshaller: Callable function to map data to a python object
e.g. lambda x: x or lambda x: json.loads(x)
:param event_type: The actual type of CloudEvent to deserialize the event to.
:returns: A CloudEvent instance parsed from the passed HTTP parameters of
the specified type.
"""
if data is None or data == b"":
# Empty string will cause data to be marshalled into None
data = ""

if not isinstance(data, (str, bytes, bytearray)):
raise cloud_exceptions.InvalidStructuredJSON(
"Expected json of type (str, bytes, bytearray), "
f"but instead found type {type(data)}"
)

headers = {key.lower(): value for key, value in headers.items()}
if data_unmarshaller is None:
data_unmarshaller = _json_or_string

marshall = marshaller.NewDefaultHTTPMarshaller()

if is_binary(headers):
specversion = headers.get("ce-specversion", None)
else:
try:
raw_ce = json.loads(data)
except json.decoder.JSONDecodeError:
raise cloud_exceptions.MissingRequiredFields(
"Failed to read specversion from both headers and data. "
f"The following can not be parsed as json: {data}"
)
if hasattr(raw_ce, "get"):
specversion = raw_ce.get("specversion", None)
else:
raise cloud_exceptions.MissingRequiredFields(
"Failed to read specversion from both headers and data. "
f"The following deserialized data has no 'get' method: {raw_ce}"
)

if specversion is None:
raise cloud_exceptions.MissingRequiredFields(
"Failed to find specversion in HTTP request"
)

event_handler = _obj_by_version.get(specversion, None)

if event_handler is None:
raise cloud_exceptions.InvalidRequiredFields(
f"Found invalid specversion {specversion}"
)

event = marshall.FromRequest(
event_handler(), headers, data, data_unmarshaller=data_unmarshaller
)
attrs = event.Properties()
attrs.pop("data", None)
attrs.pop("extensions", None)
attrs.update(**event.extensions)

if event.data == "" or event.data == b"":
# TODO: Check binary unmarshallers to debug why setting data to ""
# returns an event with data set to None, but structured will return ""
data = None
else:
data = event.data
return event_type.create(attrs, data)


def _to_http(
event: AnyCloudEvent,
format: str = converters.TypeStructured,
data_marshaller: types.MarshallerType = None,
) -> typing.Tuple[dict, typing.Union[bytes, str]]:
"""
Returns a tuple of HTTP headers/body dicts representing this Cloud Event.

:param format: The encoding format of the event.
:param data_marshaller: Callable function that casts event.data into
either a string or bytes.
:returns: (http_headers: dict, http_body: bytes or str)
"""
if data_marshaller is None:
data_marshaller = _marshaller_by_format[format]

if event["specversion"] not in _obj_by_version:
raise cloud_exceptions.InvalidRequiredFields(
f"Unsupported specversion: {event['specversion']}"
)

event_handler = _obj_by_version[event["specversion"]]()
for attribute_name in event:
event_handler.Set(attribute_name, event[attribute_name])
event_handler.data = event.data

return marshaller.NewDefaultHTTPMarshaller().ToRequest(
event_handler, format, data_marshaller=data_marshaller
)


def to_structured(
event: AnyCloudEvent,
data_marshaller: types.MarshallerType = None,
) -> typing.Tuple[dict, typing.Union[bytes, str]]:
"""
Returns a tuple of HTTP headers/body dicts representing this Cloud Event.

If event.data is a byte object, body will have a `data_base64` field instead of
`data`.

:param event: The event to be converted.
:param data_marshaller: Callable function to cast event.data into
either a string or bytes
:returns: (http_headers: dict, http_body: bytes or str)
"""
return _to_http(event=event, data_marshaller=data_marshaller)


def to_binary(
event: AnyCloudEvent, data_marshaller: types.MarshallerType = None
) -> typing.Tuple[dict, typing.Union[bytes, str]]:
"""
Returns a tuple of HTTP headers/body dicts representing this Cloud Event.

Uses Binary conversion format.

:param event: The event to be converted.
:param data_marshaller: Callable function to cast event.data into
either a string or bytes.
:returns: (http_headers: dict, http_body: bytes or str)
"""
return _to_http(
event=event,
format=converters.TypeBinary,
data_marshaller=data_marshaller,
)
Loading