Repository navigation
Add property-based tests using Hypothesis - #194
Merged
Merged
Conversation
property_tests.py checks general properties of jsonpatch against generated documents and patches: - make_patch: diffing equal documents gives an empty patch, applying make_patch(src, dst) to src gives dst, inputs are not modified, the result shares no objects with dst - apply_patch: the document and the patch are not modified, only JsonPatchException / JsonPointerException are raised (also for malformed patches), in_place and string patches give the same result - single operations: whole-document targets and sources, moving into own children, replace of any existing location, 'test' using JSON equality - JsonPatch: string roundtrip, equal patches have equal hashes - a state machine applying random valid operations and comparing each result with a direct implementation of RFC 6902 Properties that do not hold yet are expected failures, each pinned to the minimal counterexample Hypothesis found. Among them are known issues (#124, #137, #138, #179, #180, #190) and some that appear to be unreported: make_patch mishandling object keys that int() accepts, replace rejecting the object member '-', hash() failing for patches with array/object values, 'test' treating 1 and true as equal, copy from the whole document failing, and moves into own children being allowed for array members. The tests run in CI and via "make test". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqGXptMsaz9bfrMoaMpZ1A
The whole-document fixes from #191 and #192 make some pinned counterexamples pass: - test_copy_whole_document now passes; drop its expected-failure marker - test_whole_document_as_target still fails, but only for scalar documents, where 'add' raises TypeError while 'replace' works - test_raises_only_documented_exceptions still fails, now pinned to 'move' from the whole document, 'copy'/'move' from the '-' of an array, and operations on a scalar whole document Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqGXptMsaz9bfrMoaMpZ1A
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds comprehensive property-based testing to jsonpatch using the Hypothesis framework, complementing the existing example-based tests.
Summary
A new test suite (
property_tests.py) has been added that uses Hypothesis to generate many test cases and verify general properties of the jsonpatch implementation. This helps uncover edge cases and ensures the library behaves correctly across a wide range of inputs.Key Changes
New property_tests.py module: Contains 600+ lines of property-based tests organized into several test classes:
MakePatchProperties: Tests for themake_patch()function, including roundtrip verificationApplyPatchProperties: Tests for patch application, document/patch immutability, and exception handlingOperationProperties: Tests for individual RFC 6902 operations (add, remove, replace, move, copy, test)JsonPatchProperties: Tests for JsonPatch class behaviorPatchOperationMachine: A stateful test machine that applies valid operations sequentially and verifies resultsTest utilities: Helper functions for JSON comparison (
json_equal), pointer manipulation (to_pointer), document traversal (locations,containers), and outcome trackingStrategy definitions: Hypothesis strategies for generating:
Known issues documented: Tests marked with
@unittest.expectedFailuredocument 8 known issues in the implementation with minimal counterexamples, allowing them to be tracked and fixed incrementallyUpdated build configuration:
Notable Implementation Details
json_equal()for RFC 6902-compliant equality (e.g., treating 1 and True as different)PatchOperationMachineverifies that sequential operations produce consistent results@exampledecorators with minimal counterexamples for deterministic testinghttps://claude.ai/code/session_01JqGXptMsaz9bfrMoaMpZ1A