store_entity rework to add selective overwrite #61

LukasGold · 2024-07-08T10:31:12Z

This pull request includes a rework of store_entity() to allow selective overwriting. Not all viable options (e.g. overwriting only property values included in a revision older than x) are implemented yet

coveralls · 2024-07-08T14:26:12Z

Pull Request Test Coverage Report for Build 9939539897

Details

191 of 244 (78.28%) changed or added relevant lines in 4 files are covered.
8 unchanged lines in 3 files lost coverage.
Overall coverage increased (+2.4%) to 43.142%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/osw/utils/wiki.py	5	6	83.33%
src/osw/wtsite.py	18	22	81.82%
src/osw/core.py	165	213	77.46%

Files with Coverage Reduction	New Missed Lines	%
src/osw/utils/strings.py	1	53.85%
src/osw/utils/templates.py	1	77.27%
src/osw/wtsite.py	6	36.28%

Totals
Change from base Build 9809956479:	2.4%
Covered Lines:	2222
Relevant Lines:	4129

💛 - Coveralls

SimonStier

see comments

SimonStier · 2024-07-09T11:25:28Z

src/osw/core.py

+    bool,  # False: never overwrite a property,
+    # True: always overwrite a property
+    Literal[
+        "only empty",  # only overwrite if the property is empty


We should introduce an enum here, otherwise typos are likely

Implemented with df2a469

SimonStier · 2024-07-09T11:26:58Z

src/osw/core.py

+        "keep existing",  # keep the entity, does not add or remove any properties,
+        # if the page exists, the entity is not stored
+    ],
+    None,  # Not an option to choose from, will be replaced by the default


'None' and 'false' should alse be enum values for consistancy and autocomplete support.

Implemented with df2a469

SimonStier · 2024-07-09T11:50:34Z

src/osw/core.py

+                    self._overwrite_per_class["by name"][model_name] = param
+                    self._overwrite_per_class["by type"][model_type] = param

    def store_entity(


Can we separate the application of the override policy in another (util) function?
e.g. apply_override_policy(org_page, new_page, policy)
This would improve clearness & testability of the code

A sketch of what could be done:
We could separate the application of the overwrite of a page (locally setting the slot content + calling page.edit()) into a separate function osw.core.OSW.apply_overwrite_policy(org_page, entity, policy) -> osw.core.wtsite.WtPage

a params object would be defined with a validator checking that org_page and entity have the same uuid (for cases where apply_overwrite_policy() is called independently of store_entity().)

the store_entity() funtion would still call get_page() to get the original page (if exists)

therefore the set_content_and_edit_page() method would need to be moved to e.g. wtsite.WtPage

If it is not desired to access apply_overwrite_policy() independently, we could also define it as a function inside store_entity(), as done for set_content_and_edit_page().

@SimonStier please comment on what you had in mind

@SimonStier now implemented with b073c50.

SimonStier · 2024-07-09T11:52:08Z

tests/integration/store_and_load_test.py

 # run with: tox -e test -- --wiki_domain domain --wiki_username user --wiki_password pass


 def test_store_and_load(wiki_domain, wiki_username, wiki_password):


Beside this integration test we should add a unit test for apply_override_policy()

@SimonStier This implies that apply_overwrite_policy() should be accessible independently of store_entity() - correct?

implemented in b073c50
test included with 22bac86

…ely typos

* added inplace option to osw.core.OSW._ApplyOverwriteParam * moved template handling to store_entity to make osw.core.OSW_apply_overwrite_policy a static method

* added unit test for osw.core.OSW_apply_overwrite_policy * modified test of osw.core.OSW.store_entity

…up_key counting for non-matching groups

…D in their title

LukasGold · 2024-07-15T15:42:38Z

@SimonStier Should be ready for merge.

Please check cc43216 for potentially unwanted silent passes on pages without osw id in title or without jsondata
Please check e1ca4b8 for the changes to WtPage.changed regarding compatibility with the overall design philosophy of slot changes / changed page

LukasGold added 2 commits July 8, 2024 12:26

draft of store_entity rewrite + adaptation of

d122366

added test for osw.core.OSW.query_instances

0857a59

LukasGold requested a review from SimonStier July 8, 2024 10:31

LukasGold self-assigned this Jul 8, 2024

LukasGold added 3 commits July 8, 2024 15:02

Added debugging option to suppress printouts

7f7c760

small fix to avoid limited query

eb99a07

testing to fix store_entity

c3e1ef2

LukasGold added 2 commits July 9, 2024 11:18

Fix: Error proofing osw.core.OSW.delete_entity

dfb10f1

Enh: Moving to WtSiteConfig from WtSiteLegacyConfig

8c43b16

SimonStier suggested changes Jul 9, 2024

View reviewed changes

Enh: introducing an enum for overwrite options to avoid otherwise lik…

df2a469

…ely typos

LukasGold marked this pull request as draft July 12, 2024 10:01

LukasGold added 12 commits July 12, 2024 17:27

refactoring to remove import osw.model.entity as model

6f45601

making use of WtPage.changed in the function edit() again

e1ca4b8

introducing osw.core.OSW.apply_overwrite_policy()

b073c50

fixing the type error occurring in the file controller test

9ede423

making apply_overwrite_policy and accompanying params class private

facf714

moved test_regex_pattern to utils_test and added more unit tests

8ccd82c

changes to osw.core.OSW_apply_overwrite_policy

b87a184

* added inplace option to osw.core.OSW._ApplyOverwriteParam * moved template handling to store_entity to make osw.core.OSW_apply_overwrite_policy a static method

improved test coverage

22bac86

* added unit test for osw.core.OSW_apply_overwrite_policy * modified test of osw.core.OSW.store_entity

osw.utils.strings.count_match_groups now includes a correction of gro…

c8cedb4

…up_key counting for non-matching groups

allowing the upload of entities without jsondata and without an OSW-I…

cc43216

…D in their title

cosmetics

5755e94

fix: main statement in test

de5df3d

LukasGold marked this pull request as ready for review July 15, 2024 12:40

simontaurus merged commit ca4bb17 into main Jul 16, 2024

LukasGold added the enhancement New feature or request label Jul 16, 2024

LukasGold deleted the store_entity-rework-add-selective-overwrite branch November 28, 2024 08:53

		# run with: tox -e test -- --wiki_domain domain --wiki_username user --wiki_password pass


		def test_store_and_load(wiki_domain, wiki_username, wiki_password):

store_entity rework to add selective overwrite #61

store_entity rework to add selective overwrite #61

Uh oh!

Conversation

LukasGold commented Jul 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Jul 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 9939539897

Details

💛 - Coveralls

Uh oh!

SimonStier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LukasGold Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LukasGold Jul 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LukasGold Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LukasGold commented Jul 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

LukasGold commented Jul 8, 2024 •

edited

Loading

coveralls commented Jul 8, 2024 •

edited

Loading

LukasGold Jul 10, 2024 •

edited

Loading

LukasGold Jul 12, 2024 •

edited

Loading

LukasGold Jul 10, 2024 •

edited

Loading

LukasGold commented Jul 15, 2024 •

edited

Loading