Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve mismatch error #815

Merged
merged 3 commits into from
Oct 9, 2024
Merged

Conversation

JP-Ellis
Copy link
Contributor

@JP-Ellis JP-Ellis commented Oct 8, 2024

📝 Summary

Adds a Mismatch class and a number of subclasses.

This is done to facilitate the error handling, and error displays.

🚨 Breaking Changes

  • The type of the mismatches in the MismatchesError is now different. It was a dict[str, Any] and is now a Mismatch sub-class.

🔥 Motivation

Improve error messages, as the old error messages were clearly not appropriate.

🔨 Test Plan

At time of writing, yet to be done.

🔗 Related issues/PRs

@JP-Ellis JP-Ellis self-assigned this Oct 8, 2024
@JP-Ellis
Copy link
Contributor Author

JP-Ellis commented Oct 8, 2024

@individual-it This hopefully goes some way to address the visibility of the mismatches when a Pact test fails.

I'm still implementing the tests for this, but the actual implementation should mostly be there (pending confirmation from the tests I'm writing)

Copy link

codecov bot commented Oct 8, 2024

Codecov Report

Attention: Patch coverage is 79.41176% with 70 lines in your changes missing coverage. Please review.

Project coverage is 78%. Comparing base (258f701) to head (d86e54b).
Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
src/pact/v3/error.py 78% 70 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##           master   #815    +/-   ##
======================================
  Coverage      78%    78%            
======================================
  Files          29     30     +1     
  Lines        3060   3366   +306     
======================================
+ Hits         2395   2636   +241     
- Misses        665    730    +65     
Flag Coverage Δ
examples 59% <52%> (-1%) ⬇️
tests 75% <79%> (+<1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@individual-it individual-it left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
I has no time yet to try that with provider side tests, but with consumer side tests I found some issues in the from_dict function

Returns:
A new Mismatch object.
"""
if mismatch_type := data.pop("type"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when running into this error with consumer tests, data is a list not a dict
e.g.

[{'method': 'GET', 'path': '/users/123', 'request': {'body': '', 'headers': {'accept': '*/*', 'accept-encoding': 'gzip, deflate, zstd', 'connection': 'keep-alive', 'host': 'localhost:38499', 'user-agent': 'python-requests/2.32.3'}, 'method': 'GET', 'path': '/users/123'}, 'type': 'request-not-found'}, {'method': 'GET', 'path': '/user/123', 'request': {'method': 'GET', 'path': '/user/123'}, 'type': 'missing-request'}]

so this ends up in a TypeError: 'str' object cannot be interpreted as an integer

if mismatch_type == "BodyMismatch":
return BodyMismatch(**data)
if mismatch_type == "MetadataMismatch":
return MetadataMismatch(**data)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not provoke any of those types.
All types I could provoke so far (in consumer tests) are Kebab Case
request-not-found or missing-request
so they will always end up as GenericMismatch. Is that intended?

@JP-Ellis
Copy link
Contributor Author

JP-Ellis commented Oct 8, 2024

Yeah I'm still working on the breaking changes introduced for tests that inspect errors. It's also catching a number of other 'mismatches' such as the request not found you mentioned.

@JP-Ellis JP-Ellis force-pushed the feat/improve-mismatch-error branch 3 times, most recently from d5ca690 to 80fd458 Compare October 9, 2024 04:41
This adds a new `Mismatch` class, with specific different kinds of
mismatches being subclasses of this class.

BREAKING CHANGE: The `srv.mismatches` is changed from a `list[dict[str,
Any]]` to a `list[Mismatch]`.

Signed-off-by: JP-Ellis <josh@jpellis.me>
Signed-off-by: JP-Ellis <josh@jpellis.me>
With the `Matcher` accepting a generic typevar, there were a few
instances where the typevar was missing.

Signed-off-by: JP-Ellis <josh@jpellis.me>
@JP-Ellis JP-Ellis merged commit 1c5c5aa into master Oct 9, 2024
33 of 37 checks passed
@JP-Ellis JP-Ellis deleted the feat/improve-mismatch-error branch October 9, 2024 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add structured Mismatch class
2 participants