Error when sending an email in Java with CompletableFuture.runAsync

[disbrain]

I just migrated from Play 2.3 to Play 2.5 but now I can't send mails anymore.

• I Modified my conf file entry accordingly to the new format
• I created an Injector Constructor for injecting mailer

    @Inject
    public SmtpConnector(MailerClient mailer)
    {
        this.mailerClient = mailer;
        MailcapCommandMap mc = (MailcapCommandMap) MailcapCommandMap.getDefaultCommandMap();
        mc.addMailcap("text/html;; x-java-content-handler=com.sun.mail.handlers.text_html");
        mc.addMailcap("text/xml;; x-java-content-handler=com.sun.mail.handlers.text_xml");
        mc.addMailcap("text/plain;; x-java-content-handler=com.sun.mail.handlers.text_plain");
        mc.addMailcap("multipart/*;; x-java-content-handler=com.sun.mail.handlers.multipart_mixed");
        mc.addMailcap("message/rfc822;; x-java-content-handler=com.sun.mail.handlers.message_rfc822");
        CommandMap.setDefaultCommandMap(mc);
    }

• I execute the sending in an async task:

CompletableFuture.runAsync( () -> mailerClient.send(email) ).exceptionally(exc -> {exc.printStackTrace(); return null;});

But now I get the following exception:

java.util.concurrent.CompletionException: org.apache.commons.mail.EmailException: Sending the email to the following server failed : 127.0.0.1:25
    at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
    at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
    at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629)
    at java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1618)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: org.apache.commons.mail.EmailException: Sending the email to the following server failed : 127.0.0.1:25
    at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1421)
    at org.apache.commons.mail.Email.send(Email.java:1448)
    at play.api.libs.mailer.SMTPMailer$$anon$2.send(MailerPlugin.scala:100)
    at play.api.libs.mailer.CommonsMailer.send(MailerPlugin.scala:130)
    at play.api.libs.mailer.SMTPMailer.send(MailerPlugin.scala:110)
    at play.api.libs.mailer.SMTPDynamicMailer.send(MailerPlugin.scala:117)
    at play.api.libs.mailer.MailerClient$class.send(MailerPlugin.scala:56)
    at play.api.libs.mailer.SMTPDynamicMailer.send(MailerPlugin.scala:114)
    at assets.SmtpConnector.lambda$send_mail$1(SmtpConnector.java:57)
    at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
    ... 5 more
Caused by: javax.mail.MessagingException: IOException while sending message;
  nested exception is:
    javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/alternative; 
    boundary="----=_Part_0_1284684208.1469102367572"
    at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1177)
    at javax.mail.Transport.send0(Transport.java:195)
    at javax.mail.Transport.send(Transport.java:124)
    at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1411)
    ... 14 more
Caused by: javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/alternative; 
    boundary="----=_Part_0_1284684208.1469102367572"
    at javax.activation.ObjectDataContentHandler.writeTo(DataHandler.java:896)
    at javax.activation.DataHandler.writeTo(DataHandler.java:317)
    at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1485)
    at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1773)
    at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1121)
    ... 17 more

I touched NOTHING except the mailer/play version in build.sbt. Any clues please?

[ggrossetie]

Looks like a classpath issue but you already added MailcapCommandMap.
References:

• http://stackoverflow.com/questions/21856211/javax-activation-unsupporteddatatypeexception-no-object-dch-for-mime-type-multi
• http://stackoverflow.com/questions/7521027/dch-class-error-with-javamail

Do you have a dependency on javax.mail or commons-email in your build.sbt. Could you please provide a sample project to reproduce this issue ?

[disbrain]

No, I don't have such dependencies.
I attached a very minimal sample project, thanks a lot
IssueTest.zip

[ggrossetie]

I think this issue is related to the ForkJoinPool.commonPool()
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html#commonPool--

I don't know why but threads in this pool have a "special" classpath and javax.mail dependency is not loaded correctly.

If you use another Executor then the code is working. For instance I'm using a Executors.newSingleThreadExecutor():

CompletableFuture.runAsync(() -> mailerClient.send(email), Executors.newSingleThreadExecutor())
  .exceptionally(exc -> {
    exc.printStackTrace();
    return null;
});

The following is also working:

private static ForkJoinPool commonExecutor = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

CompletableFuture.runAsync(() -> mailerClient.send(email), commonExecutor)
  .exceptionally(exc -> {
    exc.printStackTrace();
    return null;
});

NOTE: In this cases you don't need to redefine MailcapCommandMap

CAUTION: I've never used this API before so maybe there's a better, simpler workaround

[disbrain]

Hi Mogztter, I tried, but the behavior is the same, I still get the exception
I added debug to javax.activation and as extra info now I get:

Can't load DCH com.sun.mail.handlers.multipart_mixed; Exception: java.lang.ClassNotFoundException: com/sun/mail/handlers/multipart_mixed

Could be all this related about the new play classloader? I use oracle java 8

[ggrossetie]

Are you running Play! in debug mode ? in production mode ? I'm running the application with activator run.

Here's the modified files:

build.bt

name := """IssueTest"""

version := "1.0-SNAPSHOT"

lazy val root = (project in file(".")).enablePlugins(PlayJava)

scalaVersion := "2.11.7"
routesGenerator := StaticRoutesGenerator
libraryDependencies ++= Seq(
  javaJdbc,
  cache,
  javaWs,
  "org.webjars" %% "webjars-play" % "2.3.0",
  "org.webjars" % "bootstrap" % "3.1.1-2",
  "com.typesafe.akka" % "akka-remote_2.11" % "2.4.8",
  "com.typesafe.play" %% "play-mailer" % "5.0.0"
)

SmtpConnector.java

package goodies;

import play.libs.mailer.Email;
import play.libs.mailer.MailerClient;

import javax.inject.Inject;
import java.util.Arrays;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ForkJoinPool;

public class SmtpConnector {

    private final static String DEFAULT_EMAIL_SENDER = "test@test.com";
    private final MailerClient mailerClient;

    private static ForkJoinPool commonExecutor = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

    @Inject
    public SmtpConnector(MailerClient mailer) {
        this.mailerClient = mailer;
    }

    public void send_mail(String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        send_mail(DEFAULT_EMAIL_SENDER, recipients, subject, textBody, htmlBody);
    }

    public void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null) {
            return;
        }
        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent()) {
            email.setBodyHtml(htmlBody.get());
        }
        CompletableFuture.runAsync(() -> mailerClient.send(email), commonExecutor)
                .exceptionally(exc -> {
                    exc.printStackTrace();
                    return null;
                });
    }
}

Could be all this related about the new play classloader?

Probably

I use oracle java 8

Me too

java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)

[disbrain]

I found that specifying executor works, but only if the email sending method is called directly from the Play controller method.
In my setting it's called in an async reply handlng method (to an akka request) and within a class instantiated through reflection:

.thenApplyAsync(new Function<Message, JsonNode>() {
  @Override
  public JsonNode apply(Message response) {
    try
    {
      Object json_reply = output_parametrized_constructor.newInstance(response, parameter.get(),http_ctx);
    }

and in the class construtor

EmailService.send_async_account_validation_email(...)

I don't know if even moving out the sending from the constructor and reflectively calling a method for that on the just built object would fix the problem because the classloader context should not be Play's anyway, right?

[ggrossetie]

This seems rather complex but I don't know your use case.
I think we need to find the root cause, so far we just found a workaround.

If we send an email synchronously everything is working fine. So I'm pretty sure that the classloader context is different when using CompletableFuture.
Could you please try without using runtime DI ? https://github.com/playframework/play-mailer/blob/master/user-manual.adoc#new-instances

[disbrain]

I tried, with and w/o explicitly setting MailcapCommandMap before sending the email, but it's still not working.

Here the class I used:

public class SmtpConnector {

    private final static MailerClient mailerClient = new SMTPMailer(
            new SMTPConfiguration(
                    "127.0.0.1",
                    25,
                    false,
                    false,
                    scala.Option.apply(null),
                    scala.Option.apply(null),
                    false,
                    scala.Option.apply(null),
                    scala.Option.apply(null),
                    false));
    private final static ForkJoinPool mail_pool = new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism());

    public static void send_mail(String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        send_mail(Preferences.DEFAULT_EMAIL_SENDER, recipients, subject, textBody, htmlBody);
    }

    public static void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null)
        {
            return;
        }

        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent())
            email.setBodyHtml(htmlBody.get());

        //async call for sending email
        CompletableFuture.supplyAsync( () -> mailerClient.send(email),mail_pool).exceptionally(exc -> {exc.printStackTrace(); return null;});

    }
}

[ggrossetie]

This is working for me. Are you testing with your sample projet or with your full project ? Did you tried to use Executors.newSingleThreadExecutor() instead of a ForkJoinPool ?

[disbrain]

Yeah, but nothing changed.
I was able to replicate the issue within a new test project. Now accessing the index page triggers an async email sending and it fails w/wo the MailcapCommandMap
IssueTest-v2.zip

EDIT
I attach the -verbose:class and getClass().getClassLoaders() dumps
classloader-dump.zip
verbose-class.zip

[ggrossetie]

I'm not familiar with this Java API but I think you are doing too much asynchronous:

• supplyAsync runs in a ForkJoinPool.
• thenApplyAsync runs in a ForkJoinPool.
• send_mail runs in a newSingleThreadExecutor.

Why not simplify:

HomeController.java

    public static CompletionStage<Result> index() {
        try {
            return CompletableFuture.supplyAsync(() -> build_me)
                    .thenApply(new Function<Constructor<?>, Result>() {
                        @Override
                        public Result apply(Constructor<?> constructor) {
                            try {
                                constructor.newInstance(null);
                            } catch (Exception exc) {
                                exc.printStackTrace();
                                return internalServerError("D'OH");
                            }
                            return ok("YAY");
                        }
                    });
        } catch (Exception exc) {
            exc.printStackTrace();
            return CompletableFuture.completedFuture(internalServerError());
        }
    }

SmtpConnector.java

    public void send_mail(String sender, String[] recipients, String subject, String textBody, Optional<String> htmlBody) {
        if (sender == null || recipients == null)
        {
            return;
        }
        Email email = new Email()
                .setFrom(sender)
                .setSubject(subject)
                .setBodyText(textBody);
        Arrays.stream(recipients).forEach(recipient -> email.addTo(recipient));

        if (htmlBody.isPresent()) {
            email.setBodyHtml(htmlBody.get());
        }
        mailerClient.send(email);
    }

Also why are you using reflection ?

[disbrain]

That was just an example on how to reproduce the issue, In my real setting I have a generic webservice handling system that validates requests, queries backend and build responses automatically and asynchrously, so I have no idea about the types that are going to be built/returned by the system. Basically every webservice have to specify only which kind of object it wants as a reply and parsing/validation/querying is all automatic:

    private final static String SET_TASK_PARAMETERS_ROUTER = play.Play.application().configuration().getString("actor.set_task_parameters");
    private final static GenericBrainAsyncer set_task_parameters_ws = new GenericBrainAsyncer(
            SetTaskParametersInput.class,
            GenericRestOutputGenerator.class,
            SET_TASK_PARAMETERS_ROUTER,
            WebServicesHotCache.GENERIC_TIMEOUT_REPLY);

    @CorsRest
    @VerboseRest
    @RequireAuthentication
    @BodyParser.Of(BodyParser.Json.class)
    public static CompletionStage<Result> set_task_parameters() {
        return set_task_parameters_ws.WS_behave(request().body().asJson());
    }

In this example, the ws request will be validated using SetTaskParametersInput class model, the resulting querying object will be sent to the backend, and the reply from the backend will be generated/analyzed using GenericRestOutputGenerator. All the async chaining stuff since is a pattern common to every service (validation, querying, analysis, output generation), is done be the generic ws_behave. But since I have very different validation models for every service, I need to be generic. Actually I could not use routes (I could create a service dispatcher with the same logic), I use them just for readability

Since the code of input builders and validators/output analyzers can be automatically generated too with a special compiler, in this way I can autogenerate the code of every new webservice regardless of what it takes, gives or what it does simply using a name convention in the actor conf file

In my case I have to be asynchronous with the email sending because I don't want to add the email-sending delay while generating the response object the user, and only inside the reply validator I will have the infos required for the email sending, since they are been validated and extracted from the request/reply by an automatic validator

[ggrossetie]

Thanks for this detailed explanation.
Just to make sure that this issue is related to classpath context in an async environment (and not specific to play-mailer), could you please try to send an email directly with commons-email ?