[FEATURE] Added Auto Transaction Saver from incoming notifications while app is in background

[Droyder7]

Fixes #205

Hi @jameskokoska, as discussed, Here is the PR for implementing this feature.

This PR includes multiple file changes for implementing multiple sub-features, will raise separate PRs if needed

Code Changes:

1. Added flutter_notification_listener for notification listening in background
   • Added flutter_notification_listener package
   • Enabled background listener permission in AndroidManifest
2. Added Notification Controller for handling notification plugin
   • Added NotificationController interface for defining controller behaviors
   • Implemented daily & upcoming notification using NotificationController
3. Added Rich Custom Notifications using awesome_notifications package
   • Installed & setup awesome_notifications package
   • Implemented AwesomeNotificationController for using the plugin
4. Added Auto Transaction Saver
   • Added transaction parsing & saving from sms notifications
   • show notification on save of auto transaction & handled notification tap
   • Added wallet selection in scanner templates
5. Added Regex scanner template
   • Added regex & income fields in ScannerTemplates schema
   • Implemented regex & isIncome fields in ScannerTemplate usage
   • Added new transaction detection for adding scanner

Recording:

Cashew.-.Auto.Transaction.Saver.mp4

[jameskokoska]

Just some comments from the demo:

The awesome_notifications is a nice touch, however I think a simpler workflow would be send a notification when it has been automatically added and tap to edit it (then the awesome_notifications isn't needed) this will also simplify the implementation with the NotificationController?

I don't think the 'ignoring' of sections of a parsed notification is intuitive. I think the before and after should be used of the selected part, however if only one condition of 'before' or one condition of 'after' is satisfied, then parse the amount correctly. (If both are satisfied, great! but maybe only one is needed?)

When I get the chance, I will start porting over at least the background parsing of notifications and will use this as a reference. Thank you for your contributions!

[Droyder7]

The awesome_notifications is a nice touch, however I think a simpler workflow would be send a notification when it has been automatically added and tap to edit it (then the awesome_notifications isn't needed) this will also simplify the implementation with the NotificationController?

Actually I have initially thought 3 use cases for an advanced workflow of using Transaction Notification, as mentioned below:

1. Default notification tap will open the default transaction page for just checking the transaction details
2. Edit button tap will open the transaction page with Initial Add Transaction popups enabled for quick editing the transaction
3. Done button tap for notification dismissal

Simpler workflow is also not bad, but this approach gives the user more control over notification usability, which was the reason behind implementing it.

I don't think the 'ignoring' of sections of a parsed notification is intuitive. I think the before and after should be used of the selected part, however if only one condition of 'before' or one condition of 'after' is satisfied, then parse the amount correctly. (If both are satisfied, great! but maybe only one is needed?)

I agree, this 'ignoring' of sections is not very intuitive but there are some issues with the existing approach of using 'before' & 'after' substring to match the section.

Suppose, if there is some variable part in the message like date or serial no, and it presents on both the 'before' & 'after' substring of the matched section, then matching may fail for other messages as variable part will change for each message.

That's why I have gone with this approach where the user can ignore the variable part, so that it will not affect the pattern matching.

[jameskokoska]

The awesome_notifications is a nice touch, however I think a simpler workflow would be send a notification when it has been automatically added and tap to edit it (then the awesome_notifications isn't needed) this will also simplify the implementation with the NotificationController?

Actually I have initially thought 3 use cases for an advanced workflow of using Transaction Notification, as mentioned below:

1. Default notification tap will open the default transaction page for just checking the transaction details
2. Edit button tap will open the transaction page with Initial Add Transaction popups enabled for quick editing the transaction
3. Done button tap for notification dismissal

Simpler workflow is also not bad, but this approach gives the user more control over notification usability, which was the reason behind implementing it.

Hmm, I do appreciate the thoughts, but I think reworking all the notifications is a bit too much at the moment - especially testing support for all platforms. I do agree it needs a refactor but at a later time.

I don't think the 'ignoring' of sections of a parsed notification is intuitive. I think the before and after should be used of the selected part, however if only one condition of 'before' or one condition of 'after' is satisfied, then parse the amount correctly. (If both are satisfied, great! but maybe only one is needed?)

I agree, this 'ignoring' of sections is not very intuitive but there are some issues with the existing approach of using 'before' & 'after' substring to match the section.

Suppose, if there is some variable part in the message like date or serial no, and it presents on both the 'before' & 'after' substring of the matched section, then matching may fail for other messages as variable part will change for each message.

That's why I have gone with this approach where the user can ignore the variable part, so that it will not affect the pattern matching.

I'm wondering if there is a way of using a type of 'shifting' pattern matching. Where the entire original string is stored and the 'locations' of the amount, title etc is stored. Then the pattern matching can find the location by using a shirting window when the matched patterns align. Not sure if this makes sense, but it would allow for dynamic values to be inserted and therefore ignored.

For example:
Transaction for $50.00 added on January 24 for Food at account 1234
So we would know that all the words before $50.00 is Transaction for and after is added on January 24 for Food at account 1234 so we would find the location of the amount based on the most letters matched/aligned with the original.

Another question I have is that is there a way to only allow the background to run and therefore scan notifications in the background ONLY IF notification scanning is enabled? Otherwise this feature may not be a good idea - Cashew should not be an app that is constantly running in the background for an optional feature.

I also saw you have limited the packages of the notifications to scan. I think it's fine to scan all notifications but only parse ones of interest. This PR is most likely only going to be used as a reference for implementing this feature so it's not a big deal.

[Droyder7]

I'm wondering if there is a way of using a type of 'shifting' pattern matching. Where the entire original string is stored and the 'locations' of the amount, title etc is stored. Then the pattern matching can find the location by using a shirting window when the matched patterns align. Not sure if this makes sense, but it would allow for dynamic values to be inserted and therefore ignored.

For example: Transaction for $50.00 added on January 24 for Food at account 1234 So we would know that all the words before $50.00 is Transaction for and after is added on January 24 for Food at account 1234 so we would find the location of the amount based on the most letters matched/aligned with the original.

It seems to be a very good idea, though it needs some brainstorming, as this algorithms needs to handle all the edge cases that may appear while pattern matching on messages including dynamic values.

Another question I have is that is there a way to only allow the background to run and therefore scan notifications in the background ONLY IF notification scanning is enabled? Otherwise this feature may not be a good idea - Cashew should not be an app that is constantly running in the background for an optional feature.

Yes, I have already implemented this in this PR. I'm starting the notification listener service only when the notification scanning is being enabled, & stopping the service when it's being disabled. You can check the code here : budget/lib/pages/autoTransactionsPageEmail.dart line 213

I also saw you have limited the packages of the notifications to scan. I think it's fine to scan all notifications but only parse ones of interest. This PR is most likely only going to be used as a reference for implementing this feature so it's not a big deal.

Yes, I'm currently limiting the allowed packages for notification scanning for mainly 2 reasons:

1. Privacy & Transparency: The user should not be worried if the app is constantly listening to all the notification on the device, so we can assure the user that the app will listen to the app's notifications where they wants to be scanned themself. The package choosing feature is currently a TODO which can be implemented in future.
2. Easier development: While testing on physical device, I wanted to prevent parsing of unwanted notifications from other apps while development or debugging

[tiniwings]

Update: I have implemented similar feature into Cashew with Tasker app and I am happy with it now.

Hi @Droyder7,

I am also looking for similar transaction automation feature. If possible can you please share the app with your PR? I would like to use it and can provide the feedback needed.

Thank you.
Tini.

[saadulkh]

I'm wondering if there is a way of using a type of 'shifting' pattern matching.

this algorithms needs to handle all the edge cases that may appear while pattern matching on messages including dynamic values.

Hey @jameskokoska,

I can work on building an AI/ML solution for this problem, which could run locally (on-device), if you think we need a smarter solution. ML can be applied on the text/sms to extract data from it regardless of whatever the pattern is. Lemme know if you are interested.

Thanks for your great work.

[jameskokoska]

I don't think it would be worth making a model for that problem; it would take up quite a bit of storage for a simple problem to solve. I have already written a pattern matching algorithm actually that works quite well, just never got around to releasing the code for this feature yet. The main issue right now is that the notification listening service is unreliable in the background; at least in my local branch implementation based on this one.

[saadulkh]

As far as my experience with android development, background services require additional permissions like ignore battery optimizations and the worst part is the vendor specific battery optimizations. It would be good to use some solution (not sure if exists) where the system automatically notifies the app for new notifications like broadcasts, instead of constantly running in the background. Or else, go with the solution where it scans after regular intervals, like the Transaction Reminder feature which sends a notification at Alert Time. The app can scan for notifications when opened as well.

Best of luck and waiting for the new release.

[jameskokoska]

I think a major issue is that the flutter_notification_listener package is inconsistent in determining if the service is running and starting/ending the service. Once that becomes consistent, this feature can become production. I haven't had much issue with OS level battery optimizations when configured in at the system level which is outside the scope of Cashew's control anyways.

[jameskokoska]

go with the solution where it scans after regular intervals, like the Transaction Reminder feature which sends a notification at Alert Time. The app can scan for notifications when opened as well.

Also to add, it isn't possible to 'capture' notifications at a certain time or interval within Android. The listener can only listen to a notification event and therefore must always be running to ensure all notifications are processed.

[saadulkh]

The listener can only listen to a notification event

That makes sense.

One more thing, please provide an app link for the pattern matching feature, separate from notification listening service. This would allow the use of the Tasker app on Android for notification listening and then forward notification content to Cashew using the app link.

I would want just one less app running in the background :/ and that too for just one feature.

Thanks.

[jameskokoska]

I appreciate everyone's input on this matter so far. However, because App Links are very powerful, there seem to be some issues with Flutter's primary Notification Listener package on Android, and there isn't a great solution for parsing specific information coupled with an intuitive user experience - I am going to close this PR. There is a thread here: #127 (comment) which can show you how to harness the power of App Links to achieve a similar (if not better!) implementation of transaction automation albeit with a bit of setup.