Mails Randomly Marked as "Important", Also For Other IMAP Clients

[gohrner]

Expected behavior

Mails should not be randomly marked as "Important" - I could not find any documentation about how to specifically configure and/or disable automatic tagging of mails as "Important".

The only related information I could find was this announcement: https://nextcloud.com/blog/nextcloud-mail-introduces-machine-learning-for-priority-inbox/

We mainly use Thunderbird to read mails. Tagging of mails is basically solely performed in Thunderbird, and more or less for most of the users serves as a kind of "TODO" tag - there's something about "Important" mails that still needs to be dealt with.

Actual behavior

After upgrading the NextCloud Mail App to 1.10.0, mails in other IMAP clients seem to be randomly classified as "Important", even if the NextCloud Mail App was not "actively" used by a user.

This totally breaks the logic of using "Important" to mark mails which still need to be dealt with, does not seem to have happened in previous versions of NextCloud Mail (at least by for not as obvious), and is not desired.

Also, the NextCloud Mail App does not seem to "learn" which mails are "unmarked" as "not important" in other IMAP clients (specifically, Thunderbird).

Mail app

Mail app version: 1.10.0

Mailserver or service: self-hosted Dovecot server

Server configuration

Operating system: Debian 10 Buster

Web server: Apache httpd

Database: MariaDB

PHP version: 7.0.33

Nextcloud Version: 20.0.11 (most recent release currently offered through the Stable channel for this server)

Client configuration

Browser: N/A

Operating system: N/A

[ChristophWurst]

Your users can turn this off: #5260

Thea reason why the important messages now also show in Thunderbird is that we now are fully interoperable with Thunderbird and its tagging conventions. Anything marked as important here is also important there. Turn off the classification for manual workflows.

Also, the NextCloud Mail App does not seem to "learn" which mails are "unmarked" as "not important" in other IMAP clients (specifically, Thunderbird).

It actually does based on a naive bayes algorithm. Messages (or merely senders) with similar characteristics as previous email are classified respectively. The model is retrained once an hour. Therefore it should be possible for the algorithm to somewhat pick up your preferences over time. The more data it gets, the better it will be able to classify.

[ChristophWurst]

Please also see #3968 for a bit more discussion around this feature and its quality ✌️

[gohrner]

@ChristophWurst: Ok, thanks! From #5290 I get that disabling still does not really work for Mail 1.10.0 and 1.10.1, but will be fixed in 1.10.2 - correct?

Also, how can I imagine the "learning" - say I only use Thunderbird to (re)classify my mails as "Important" or "Not Important". However I also move my mails to other IMAP folders immediately after dealing with it, so the following will happen often:

• NextCloud Mail tags a mail as "Important" which I don't consider to be.
• I remove the "Important" flag in Thunderbird and immediately move the mail to one of my many topic-specific folders.
• Up to an hour later, the NextCloud Mail classifier will take the next look at my inbox.

Will NextCloud Mail still notice that I considered the "Important" tag as a misclassification in this constellation and update its statistics accordingly?

Or will it think I just deleted the mail from the inbox and the classification it did was correct?

[ChristophWurst]

@ChristophWurst: Ok, thanks! From #5290 I get that disabling still does not really work for Mail 1.10.0 and 1.10.1, but will be fixed in 1.10.2 - correct?

Correct :)

Will NextCloud Mail still notice that I considered the "Important" tag as a misclassification in this constellation and update its statistics accordingly?

It depends. If Nextcloud is configured to also synchronize the other mailbox at which the message is now stored for background syncs (checkbox in the ... menu in the sidebar) then it will have that message as non-important reference. Then it can learn your pattern.

If the mailbox is not indexed nor synchronized by Nextcloud Mail then it won't be able to learn from your feedback.

There is one other quirk you might encounter.

1. A new messages is received
2. Mail marks it as important
3. You unmark and move the message
4. The message is detected in the new mailbox, Mail marks it as important again

This will happen because from Mail's perspective we don't know the message was ever unmarked from being important, our application doesn't keep track of before/after states. It "just" finds this new message in a mailbox and then does its classification.

As opposed to $junk and $notjunk there is no negated IMAP flag for $label1 (important). If Thunderbird and other clients set a $notimportant or similar on IMAP then our application would be able to know that something was either already classified or changed by the user. Right now either no flag is set or $label1. If the flag is set then we don't classify, we assume that this state is already correct. In the absence of the flag we start our classifier.

[gohrner]

@ChristophWurst:

The message is detected in the new mailbox, Mail marks it as important again

Actually I have the impression that Nextcloud Mail also even marks mails in the inbox again which were misclassified and which I unmarked, without them being moved anywhere.

I assume this should not happen, should I file another bug for this? However I probably cannot provide much more information than this at the moment...

[ChristophWurst]

Well there is no code for that. Only when messages are indexed for the first time they are classified.

Could you observe it a little longer to find a pattern of when that happens exactly?

[gohrner]

(Shall I move this conversation to the forum or PM; it's probably not (yet) suitable for a bug tracker.)

Could this be related to re-marking messages as unread?

What I do is, I get a bunch of new mails, which wait in "unread" state in my mailbox and after a while Nextcloud Mail marks some mails as "important". I use Thunderbird to open the mails, reset the "important" flag to "not important" and re-mark the mail as "unread".

Could this cause Nextcloud Mail to tag the mails as "important" again?

In any case, this already happened again.

Mailserver is the Dovecot version contained in Debian 10 Buster.

Also a question - if

Only when messages are indexed for the first time they are classified.

is the case, how does Nextcloud Mail notice later changes to the "Important" flag for training? Or does it monitor already classified mails if the flag changes, but does the classification and tagging only once when adding the mail to the index?

[bjoernv]

I observe the same problem. Only new messages are marked as "Important".

The setup is more recent compared with the initial report:
Web Server: Apache 2.4.57
PHP: 8.2
MySQL: 5.7.42
IMAP: Dovecot IMAP 2.2.33.2
Nextcloud: 25.0.8

[NextTherapist]

Actually I have the impression that Nextcloud Mail also even marks mails in the inbox again which were misclassified and which I unmarked, without them being moved anywhere.

I can confirm this for Nextcloud 26.0.3 and PHP 8.1 with Thunderbird 102.11.

Why is this buggy feature activated by default? It should not be terrible, ok, but it is, since the bug occurs in a completely different app (Thunderbird) after having a mail account configured in Nextcloud.

[zulu64]

In case this helps anyone:

This is neither a bug in Thunderbird nor Nextcloud.

If your email server type is IMAP, all changes made by any of your email clients will be received by the other email clients. This doesn't mean those changes are always visible, but the "Important" tag is one feature that is shared by Thunderbird and the Nextcloud Mail app.

I confirm this still happens today with:
Thunderbird 128.8.0
Mail 4.2.5 on Nextcloud 31.0.0 (Hub 10)

If you don't want this to happen in Thunderbird, ensure that other email clients don't automatically set the "Important" tag - in my case, change the setting in the Nextcloud Mail app: Mail Settings - Assistance features - Mark as important - uncheck

As this issue is in Triage for 4 years now, and cannot be fixed (and doesn't need to be), I suggest it can be closed.

[Jacques-Olivier-Farcy]

Hi,

I really don't understand why in Nextcloud Mail this setting is activ by default.
Mail Settings - Assistance features - Mark as important should be uncheck by default.
I am using thunderbird for years with an IMAP server.
Today i had to configure mail in my nextcloud, i realised after that all my messages were marked as important when i opened thunderbird again.

I had no way of knowing that there was this option to tick before configuring mail in nextcloud.

I know have to reconfigure all my mail history to change the important tag.
I don't think it can be closed.