[MOB-2899] - Health Monitoring for Offline Processor #345

Ayyanchira · 2021-05-12T14:41:16Z

🔹 Jira Ticket(s) if any

https://iterable.atlassian.net/browse/MOB-2899

✏️ Description

Dont schedule for offline processing if row count exceeds max allowed number
Max offline operation is set to 1000
Added tests for Healthmonitoring and offline processing
DB error checks and fallback logic

1. Dont schedule for offline processing if row count exceeds max allowed number 2. Max offline operation is set to 1000 3. Added mock task storage in initializer for testing

codecov · 2021-05-12T16:29:45Z

Codecov Report

Merging #345 (8c62628) into master (05e5c95) will decrease coverage by 1.12%.
The diff coverage is 52.38%.

❗ Current head 8c62628 differs from pull request most recent head e32b4b5. Consider uploading reports for the commit e32b4b5 to get more accurate results

@@            Coverage Diff             @@
##           master     #345      +/-   ##
==========================================
- Coverage   70.57%   69.45%   -1.13%     
==========================================
  Files          60       61       +1     
  Lines        3715     3765      +50     
  Branches      433      438       +5     
==========================================
- Hits         2622     2615       -7     
- Misses        828      895      +67     
+ Partials      265      255      -10

Impacted Files	Coverage Δ
...va/com/iterable/iterableapi/IterableConstants.java	`0.00% <ø> (ø)`
.../com/iterable/iterableapi/IterableTaskStorage.java	`8.37% <18.51%> (-15.02%)`	⬇️
...a/com/iterable/iterableapi/IterableTaskRunner.java	`64.36% <40.00%> (-1.90%)`	⬇️
...n/java/com/iterable/iterableapi/HealthMonitor.java	`78.94% <78.94%> (ø)`
.../iterable/iterableapi/OfflineRequestProcessor.java	`81.03% <91.66%> (-1.32%)`	⬇️
...in/java/com/iterable/iterableapi/IterableUtil.java	`65.17% <0.00%> (-5.36%)`	⬇️
...in/java/com/iterable/iterableapi/IterableTask.java	`40.00% <0.00%> (-3.34%)`	⬇️
...terableapi/IterableNetworkConnectivityManager.java	`33.33% <0.00%> (-2.78%)`	⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05e5c95...e32b4b5. Read the comment docs.

1. Healthmonitor now access Iterable task storage to directly access db and to understand if its ready 2. Introducing Healthmonitorhandler with idea that whoever subscribes to healthmonitor (in our case Taskrunner), should understand if something goes wrong and make it immediately switch to online. This is not implemented yet 3. IterableTaskStorageHandler (may be not a good idea. Might have to remove this)

1. Interface at IterableTaskStorage which will notify all DB errors to Healthmonitor. This will be used by healthmonitor to keep its flag 'errored' updated. 2. canSchedule and canProcess will now be checked through this flag in healthmonitor 3. Updated existing test methods which were failing because of this changes as it needed healthmonitoring mock in them 4. Made some improvements where healthmonitor was being initialized reperatedly. Still pending: 1. should canProcess and canSchedule be based on just one flag - errored in healthMonitor? 2. Is there a technique to switch back to offline mode? after we set errored flag to true?

1. DB method numberOfTasks now throws exception. Although we can still add exception handling in other methods in TaskStorage as well, this should cover it all. This should suffice as it internally checks for if the database is ready and throws an exception if its not. This method is also called during canSchedule which makes it a busy node for every request being made during offline request processing. 2. Added test method for canSchedule. Canprocess test is still pending

1. DB initialization was taking wrong context instead of the one passed into it. 2. Above change enabled writing test for healthMonitor's canProcess function. 3. getRemoteCongifuration now looks for "SDKOfflineModeBeta" instead of "offlineModeBeta" to enable offline mode 4. Reverting back to our previous implementation of apiclient where we dont have different requestProcessor in apiclient. It will remain abstract in apiclient layer

iterableapi/src/main/java/com/iterable/iterableapi/IterableTaskStorage.java

iterableapi/src/main/java/com/iterable/iterableapi/IterableConstants.java

iterableapi/src/main/java/com/iterable/iterableapi/HealthMonitor.java

iterableapi/src/main/java/com/iterable/iterableapi/IterableTaskStorage.java

iterableapi/src/main/java/com/iterable/iterableapi/HealthMonitor.java

iterableapi/src/main/java/com/iterable/iterableapi/IterableTaskStorage.java

iterableapi/src/main/java/com/iterable/iterableapi/OfflineRequestProcessor.java

davidtruong

LFTM

Adding suggested changes from PR reviews Also added error check while creatin-> inserting task into DB

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch 2 times, most recently from 757597c to 4f0bd2d Compare May 12, 2021 16:01

[MOB-2899] - Health Montoring for Offline Processor

09e81a7

1. Dont schedule for offline processing if row count exceeds max allowed number 2. Max offline operation is set to 1000 3. Added mock task storage in initializer for testing

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch from 4f0bd2d to 09e81a7 Compare May 12, 2021 16:18

Ayyanchira marked this pull request as draft May 12, 2021 16:32

Ayyanchira requested review from roninopf and vbabenkoru May 12, 2021 16:32

Ayyanchira changed the title ~~[MOB-2899] - Health Montoring for Offline Processor~~ [MOB-2899] - Health Monitoring for Offline Processor May 12, 2021

roninopf changed the base branch from master to MOB-2854-health-monitor May 12, 2021 18:16

roninopf and others added 7 commits May 12, 2021 17:09

start adding healthmonitor

6348a74

make requestprocessor dynamically chosen

b7122e6

removee requestprocessors from apiclient lazy load

e598e44

remove unused requestProcessor variable

f75cea7

clarify error message

66a1c69

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch from 8f3949c to d5801fe Compare May 18, 2021 17:10

roninopf and others added 5 commits May 19, 2021 14:08

minor maintenance

ada6946

create healthmonitor unit tests skeleton

900221f

stub unit test

0fe43c1

follow naming convention for unit tests

d03f79a

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch 7 times, most recently from 329cdb7 to 3a3cc94 Compare May 24, 2021 21:17

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch 3 times, most recently from 0906eb8 to 2161969 Compare May 25, 2021 15:14

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch from 2161969 to 7525fb2 Compare May 25, 2021 16:19

Ayyanchira marked this pull request as ready for review May 25, 2021 16:54

Ayyanchira requested a review from davidtruong May 25, 2021 17:09

davidtruong reviewed May 25, 2021

View reviewed changes

iterableapi/src/main/java/com/iterable/iterableapi/IterableTaskStorage.java Outdated Show resolved Hide resolved

roninopf changed the base branch from MOB-2854-health-monitor to master May 25, 2021 20:16

roninopf and others added 2 commits May 25, 2021 16:32

remove incomplete test

abed13f

Adding PR suggestions

25bf7ee

davidtruong reviewed May 26, 2021

View reviewed changes

iterableapi/src/main/java/com/iterable/iterableapi/IterableTaskStorage.java Outdated Show resolved Hide resolved

remove extraneous error notification

ba681fe

vbabenkoru reviewed May 27, 2021

View reviewed changes

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch from 3102b30 to 102e5be Compare May 27, 2021 18:29

Ayyanchira requested review from davidtruong and vbabenkoru May 27, 2021 18:47

davidtruong approved these changes May 27, 2021

View reviewed changes

Revision 4

e32b4b5

Adding suggested changes from PR reviews Also added error check while creatin-> inserting task into DB

Ayyanchira force-pushed the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch from 102e5be to e32b4b5 Compare May 27, 2021 19:18

vbabenkoru approved these changes May 27, 2021

View reviewed changes

Ayyanchira merged commit 9ba4711 into master May 27, 2021

Ayyanchira deleted the MOB-2899-OnlineRequestProcessor-when-DB-exceeds-maxcount branch May 27, 2021 21:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[MOB-2899] - Health Monitoring for Offline Processor #345

[MOB-2899] - Health Monitoring for Offline Processor #345

Uh oh!

Ayyanchira commented May 12, 2021 •

edited

Loading

Uh oh!

codecov bot commented May 12, 2021 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davidtruong left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[MOB-2899] - Health Monitoring for Offline Processor #345

[MOB-2899] - Health Monitoring for Offline Processor #345

Uh oh!

Conversation

Ayyanchira commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔹 Jira Ticket(s) if any

✏️ Description

Uh oh!

codecov bot commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davidtruong left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Ayyanchira commented May 12, 2021 •

edited

Loading

codecov bot commented May 12, 2021 •

edited

Loading