-
Notifications
You must be signed in to change notification settings - Fork 62
control-service: add amazon ses smtp mail notifications #3269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
13686b1
control-service: add aws ses smtp
yonitoo 682577a
Google Java Format
ac9706c
control-service: change gitlab ci to build a test image for ses
yonitoo 0bdb710
vdk-dag: add status checks functionalities (#3250)
duyguHsnHsn f09b1fc
vdk-core: env var precedence over secrets config (#3239)
DeltaMichael 105a418
vdk-core: add db_connection_after_operation (#3254)
duyguHsnHsn ea602ce
control-service: explicitly pass the test image tag
yonitoo 7fd1682
Merge branch 'main' into person/yonitoo/test-aws-smtp
yonitoo e1a8800
control-service: add AWS profile for the AmazonEmailNotification class
yonitoo aceecb9
add debug logs
yonitoo a0ae1ec
Google Java Format
b6899d8
vdk-oracle: remove secrets override (#3220)
duyguHsnHsn 71c6010
vdk-oracle: Pass ingestion payload rows in uniform batches (#3194)
DeltaMichael 75e3bbf
vdk-smarter: migrate decorate_operation to before_operation (#3255)
DeltaMichael 547f687
vdk-lineage: migrate decorate_operation to before_operation (#3253)
DeltaMichael e34d2bf
vdk-impala: adapt lineage to new db operations (#3258)
duyguHsnHsn dfc8e00
Merge branch 'main' into person/yonitoo/test-aws-smtp
yonitoo 9794e72
change to prints
yonitoo 720039e
control-service: add a few more logs
yonitoo 295d5f0
control-service: change dest due to lack of permissions
yonitoo f4a9655
Google Java Format
a041930
control-service: amazon ses uses real values.
murphp15 87ca661
Google Java Format
d037f98
Merge branch 'main' into person/murphp15/amazon_ses_use_real_config
murphp15 6e5dac2
control-service: add condition to prefer amazon when ARN is present
yonitoo 496a097
Merge branch 'main' into person/murphp15/amazon_ses_use_real_config
yonitoo 96296de
control-service: fix prefix
yonitoo cfd8e97
control-service: change smtp condition
yonitoo 6f39c2e
control-service: amazon ses uses real values.
murphp15 bb4a3ba
control-service: amazon ses uses real values.
murphp15 0997522
control-service: amazon ses uses real values.
murphp15 758cb10
control-service: cleanup the changes from redundant logs
yonitoo 3ff6061
control-service: remove unused logger
yonitoo 8e8d51b
control-service: change logs to debug
yonitoo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
76 changes: 76 additions & 0 deletions
76
...vice/src/main/java/com/vmware/taurus/service/notification/AmazonSesEmailNotification.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| /* | ||
| * Copyright 2023-2024 Broadcom | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package com.vmware.taurus.service.notification; | ||
|
|
||
| import com.amazonaws.auth.AWSStaticCredentialsProvider; | ||
| import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; | ||
| import com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider; | ||
| import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder; | ||
| import com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClientBuilder; | ||
| import com.amazonaws.services.simpleemail.model.*; | ||
| import com.google.common.collect.Lists; | ||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| import javax.mail.MessagingException; | ||
| import javax.mail.internet.InternetAddress; | ||
| import java.util.Arrays; | ||
|
|
||
| /** | ||
| * The AmazonEmailNotification class is responsible for implementing the email notifications | ||
| * functionality in AWS using SES. The single exposed send method accepts pre-build notification | ||
| * object. | ||
| * | ||
| * @see DataJobNotification | ||
| * @see NotificationContent | ||
| */ | ||
| public class AmazonSesEmailNotification implements EmailNotification { | ||
|
|
||
| static Logger log = LoggerFactory.getLogger(AmazonSesEmailNotification.class); | ||
| private final String roleArn; | ||
| private final String region; | ||
|
|
||
| public AmazonSesEmailNotification(EmailConfiguration emailConfiguration) { | ||
| this.region = emailConfiguration.smtpWithPrefix().get("mail.smtp.host"); | ||
| this.roleArn = emailConfiguration.getUsername(); | ||
| } | ||
|
|
||
| @Override | ||
| public void send(NotificationContent notificationContent) throws MessagingException { | ||
| String roleSessionName = "email-session"; | ||
|
|
||
| STSAssumeRoleSessionCredentialsProvider credentialsProvider = | ||
| new STSAssumeRoleSessionCredentialsProvider.Builder(roleArn, roleSessionName) | ||
| .withStsClient( | ||
| AWSSecurityTokenServiceClientBuilder.standard() | ||
| .withCredentials(new DefaultAWSCredentialsProviderChain()) | ||
| .withRegion(this.region) | ||
| .build()) | ||
| .build(); | ||
|
|
||
| var sesClient = | ||
| AmazonSimpleEmailServiceClientBuilder.standard() | ||
| .withCredentials(new AWSStaticCredentialsProvider(credentialsProvider.getCredentials())) | ||
| .withRegion(this.region) | ||
| .build(); | ||
| log.debug("Send AmazonSesEmailNotification"); | ||
| SendEmailResult sendEmailResult = | ||
| sesClient.sendEmail( | ||
| new SendEmailRequest() | ||
| .withSource(notificationContent.getSender().toString()) | ||
| .withDestination( | ||
| new Destination( | ||
| Lists.newArrayList( | ||
| Arrays.stream(notificationContent.getRecipients()) | ||
| .map(InternetAddress::toString) | ||
| .toList()))) | ||
| .withMessage( | ||
| new Message( | ||
| new Content(notificationContent.getSubject()), | ||
| new Body().withHtml(new Content(notificationContent.getContent()))))); | ||
| log.info("Email sent using Amazon SES. Message ID: {}", sendEmailResult.getMessageId()); | ||
| } | ||
| } |
114 changes: 114 additions & 0 deletions
114
...ervice/src/main/java/com/vmware/taurus/service/notification/DefaultEmailNotification.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| /* | ||
| * Copyright 2023-2024 Broadcom | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package com.vmware.taurus.service.notification; | ||
|
|
||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| import javax.mail.*; | ||
| import javax.mail.internet.MimeMessage; | ||
| import java.util.Arrays; | ||
| import java.util.Properties; | ||
| import java.util.stream.Collectors; | ||
|
|
||
| /** | ||
| * The DefaultEmailNotification class is responsible for implementing the email notifications | ||
| * functionality. The single exposed send method accepts pre-build notification object. | ||
| * | ||
| * @see DataJobNotification | ||
| * @see NotificationContent | ||
| */ | ||
| public class DefaultEmailNotification implements EmailNotification { | ||
|
|
||
| static Logger log = LoggerFactory.getLogger(DefaultEmailNotification.class); | ||
|
|
||
| private static final String CONTENT_TYPE = "text/html; charset=utf-8"; | ||
|
|
||
| private final Session session; | ||
| private final EmailConfiguration emailConfiguration; | ||
|
|
||
| public DefaultEmailNotification(EmailConfiguration emailConfiguration) { | ||
| this.emailConfiguration = emailConfiguration; | ||
| Properties properties = System.getProperties(); | ||
| properties.putAll(emailConfiguration.smtpWithPrefix()); | ||
| properties.setProperty("mail.transport.protocol", emailConfiguration.getTransportProtocol()); | ||
| session = Session.getInstance(properties); | ||
| } | ||
|
|
||
| public void send(NotificationContent notificationContent) throws MessagingException { | ||
| log.debug("Send DefaultEmailNotification"); | ||
| if (recipientsExist(notificationContent.getRecipients())) { | ||
| try { | ||
| sendEmail(notificationContent, notificationContent.getRecipients()); | ||
| } catch (SendFailedException firstException) { | ||
| log.info( | ||
| "Following addresses are invalid: {}", | ||
| concatAddresses(firstException.getInvalidAddresses())); | ||
| var validUnsentAddresses = firstException.getValidUnsentAddresses(); | ||
| if (recipientsExist(validUnsentAddresses)) { | ||
| log.info("Retrying unsent valid addresses: {}", concatAddresses(validUnsentAddresses)); | ||
| try { | ||
| sendEmail(notificationContent, validUnsentAddresses); | ||
| } catch (SendFailedException retriedException) { | ||
| log.warn( | ||
| "\nwhat: {}\nwhy: {}\nconsequence: {}\ncountermeasure: {}", | ||
| "Unable to send notification to the recipients", | ||
| retriedException.getMessage(), | ||
| "Listed recipients won't receive notifications for the job's status", | ||
| "There might be misconfiguration or outage. Check error message for more details"); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| private void sendEmail(NotificationContent notificationContent, Address[] recipients) | ||
| throws MessagingException { | ||
| if (emailConfiguration.isAuthEnabled()) { | ||
| sendAuthenticatedEmail(notificationContent, recipients); | ||
| } else { | ||
| sendUnauthenticatedEmail(notificationContent, recipients); | ||
| } | ||
| } | ||
|
|
||
| private void sendUnauthenticatedEmail( | ||
| NotificationContent notificationContent, Address[] recipients) throws MessagingException { | ||
| var mimeMessage = prepareMessage(notificationContent); | ||
| Transport.send(mimeMessage, recipients); | ||
| } | ||
|
|
||
| private void sendAuthenticatedEmail(NotificationContent notificationContent, Address[] recipients) | ||
| throws MessagingException { | ||
| Transport transport = session.getTransport(); | ||
| var mimeMessage = prepareMessage(notificationContent); | ||
| try { | ||
| transport.connect(emailConfiguration.getUsername(), emailConfiguration.getPassword()); | ||
| transport.sendMessage(mimeMessage, recipients); | ||
| } finally { | ||
| transport.close(); | ||
| } | ||
| } | ||
|
|
||
| private MimeMessage prepareMessage(NotificationContent notificationContent) | ||
| throws MessagingException { | ||
| MimeMessage mimeMessage = new MimeMessage(session); | ||
| mimeMessage.setFrom(notificationContent.getSender()); | ||
| mimeMessage.setRecipients(MimeMessage.RecipientType.TO, notificationContent.getRecipients()); | ||
| mimeMessage.setSubject(notificationContent.getSubject()); | ||
| mimeMessage.setContent(notificationContent.getContent(), CONTENT_TYPE); | ||
| return mimeMessage; | ||
| } | ||
|
|
||
| private boolean recipientsExist(Address[] recipients) { | ||
| return recipients.length > 0; | ||
| } | ||
|
|
||
| String concatAddresses(Address[] addresses) { | ||
| return addresses == null | ||
| ? null | ||
| : Arrays.stream(addresses).map(Address::toString).collect(Collectors.joining(" ")); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.