Mail (IMAP) source

Mail (IMAP) Source

The application offers also the possibility of uploading mail messages with their attachments to Microsoft SharePoint. For this purpose, multiple mail accounts may be configured as document sources.

During the time the source is active, it scans for new emails in the indicated accounts and folders and uploads them to the configured Microsoft SharePoint libraries/lists. Uploaded mail messages are moved to the configured archive folder, while messages which failed to upload are moved to the configured failed folder.

Uploaded messages receive as name a truncated subject (max. 100 characters), to which a unique ID is appended. This is by design, as uploading messages with the same subject should not overwrite/add a new version to the same message. Additionally, each attachment matching the configured pattern will be uploaded as a separate file having as name the name of the uploaded message with the name of the attached file appended to it. This is by design, as in this way the message with its attachments are subsequent records in the SharePoint list. Nevertheless, the original subject and attachment name are saved in the configured OriginalName field.

mailSettings.json file

Purpose

Contains server connection settings, attachment pattern, the source added fields and the lit of configured Microsoft Exchange accounts.

• serverName: Required. The Microsoft Exchange server name. For on-premise deployments this may be an IP address or a computer name, while for Microsoft Exchange Online this is the server URL.
• port: Required. The port for IMAP connection.
• fields: A list of fields to be applied to all uploaded messages in the configured accounts and folders.
• attachmentFields: A list of fields to be applied to all uploaded attachments in the configured accounts and folders.
  • Because attachments are part of a message, all message fields are also applied to the attachments. The attachments, however, may add new or overwrite them.
• attachmentPattern: A string containing a list of file name patterns separated by , or |, just like the operating system file patterns. If not provided, all attachments are uploaded.
  • Note: some HTML based mails contain images which should be typically ignored, like signature pictures. Use this option to include only relevant attachments.
  • Example: "attachmentPattern": "*.pdf,*.doc,*.docx,*.xls,*.xlsx,*.ppt,*.pptx,*.html,*.htm,*.msg,*.eml".
• redirectFilePattern: A string containing a file name pattern just like the operating system file patterns. If provided, all attachments with file names matching this pattern will not be uploaded as such, but redirected to another source for further processing. For example *.msg attachments will be redirected to the Mail source to unpack and upload the containing mail.
• strategy: a string containing the name of a strategy to be applied when handling emails. If not provided, the first in the ordered list of strategies is used.
  • Developer note: A strategy is a service implementing IMailStrategy. By default, the MoveToArchive strategy is used which moves the processed emails to the Archive folder, if successful, and to SyncFailed folder otherwise.
• account: An array of account configurations. For each account, the following options are available:
  • userName: the user name of the Microsoft Exchange account.
  • userEncryptedPassword: the encrypted password of the user configured above. Check the encrypt command for more information about how to get an encrypted password.
  • userPassword: the plain password of the user configured above.
    • Avoid using this value in the long run, use it only for the initial connection testing and, after that, switch to the encrypted password.
    • Do not use both userPassword and userEncryptedPassword settings at the same time. Even though the application will not complain, it is not ensured that a specific setting is used.
  • userAddress: the user mail address.
  • archiveFolder: the folder where messages should be moved after a successful upload. Default value: Archive.
    • If this folder does not exist, it will be created.
  • failedFolder: the folder where messages should be moved after a failed upload. Default value: SyncFailed.
    • If this folder does not exist, it will be created.
  • fields: A list of fields to be applied to all uploaded messages and attachments from this account.
  • attachmentFields: A list of fields to be applied to all uploaded attachments from this account.
  • syncFolders: An array of folder configurations. For each folder, the following options are available:
    • path: Required. The path to the folder. Nested folders will specify a path using the Exchange separator character /.
    • library: The library where the documents retrieved from this folder should be uploaded. If not provided, the library indicated in the global settings file is used.
    • preserveFolderStructure: Flag used to control the relative path of the uploaded file. If false, the file is uploaded in the list's root folder, otherwise a folder structure mirroring the source structure is created. Default value: false.
    • recursive: Indicates whether the mail folder should include all sub folders at any level. Default value: false.
      • Note: configured archive and failed folders are ignored.
    • fields: A list of fields to be applied to all uploaded messages and attachments in this folder.
    • attachmentFields: A list of fields to be applied to all uploaded attachments from this account.
    • redirectFilePattern: If specified, overrides the same setting at the root level.
    • strategy: If specified, overrides the same setting at the root level.
    • query: A string or JSON specifying a query for retrieving emails. See the query definition and the examples further in this document.
      • Note: if no query is specified, all emails are considered.

Example

{
    "serverName": "outlook.office365.com",
    "port": 993,
    "attachmentPattern": "*.pdf,*.doc,*.docx,*.xls,*.xlsx,*.ppt,*.pptx,*.html,*.htm,*.msg,*.eml",
    "fields": {
        "Source": "Mail"
    },
    "attachmentFields": {
        "RequireOCR": true
    },
    "accounts": [
        {
            "userName": "<invoices@company.com>",
            "userEncryptedPassword": "<the-encrypted-password>",
            "userAddress": "<my-user@company.com>",
            "archiveFolder": "Archive",
            "failedFolder": "SyncFailed",
            "fields": {
                "Partner": "Odoo"
            },
            "syncFolders": [
                {
                    "path": "Inbox",
                    "library": "Invoices",
                    "recursive": true,
                    "preserveFolderStructure": true,
                    "fields": {
                        "Kind": "Invoice"
                    }
                }
            ]
        }
    ]
}

Mail queries

Queries are built using predefined queries, query operators, and expression queries. Rules for building the query:

• The names for queries, query operator and expression queries is case insensitive.
• Predefined queries are specified as a string containing their name.
• The query operators and expression queries contain at least one parameter. Parameters are specified as an array of values.
• Date/Time values are provided as strings in the format "YYYY-MM-DD", optionally with the time: "YYYY-MM-DD hh:mm[:ss]".

Predefined queries

• All: Matches all messages in the folder.
• Seen: Matches messages with the Seen flag set.
• Recent: Matches messages with the Recent flag set.
• Old: Matches messages that do not have the Recent flag set.
• New: Matches messages with the Recent flag set but not the Seen.
• Flagged: Matches messages with the Flagged flag set.
• Draft:
• Deleted:
• Answered:
• NotSeen:
• NotRecent:
• NotFlagged:
• NotDraft:
• NotAnswered:
• NotDeleted:

Query operators

• And (binary): Combines two queries using the logical AND operator.
• Or (binary): Combines two queries using the logical OR operator.
• Not (unary): Negates the query condition.
• Fuzzy (unary): Applies a fuzzy matching algorithm to the specified query expression.

Expression queries

• DeliveredAfter(date)
• DeliveredBefore(date)
• DeliveredOn(date)
• FromContains(text)
• CcContains(text)
• BccContains(text)
• SubjectContains(text)
• BodyContains(text)

Mail query examples

Search only not seen emails.

{
   ...
   "query": "notseen"
}

Search only not seen emails and not flagged.

{
   ...
   "query": { "and": [ "notseen", "notflagged" ] }
}

Search only not seen emails and not flagged and delivered after 2020-05-01.

{
   ...
   "query": { "and": [
                 "notseen",
                 {
                     "and": [ "notflagged", { "deliveredAfter": [ "2020-05-01" ] } ]
                 }
              ] }
}