Jump to content
Yaron

Sending HTML based eMails with UTF8 encoding and SSL

By Yaron, in Indy

Recommended Posts

Yaron    53

Yaron
Posted (edited)

Using Delphi 10.3.2:

I use the function below to send HTML eMails through Amazon SES using Indy.

However, no matter what I tried, non-latin characters appear as "�" (for example, instead of "Oriënt Express", gMail displays "Ori�nt Express").

 

I need to  be able to include non-latin characters in both the subject and the eMail's HTML body, what am I doing wrong?

 

P.S.

Not sure if it matters, but the HTML does contain "<meta charset="UTF-8">" in the "<HEAD>" block and I verified the source was properly encoded by saving it to a local utf8 html file which displays the encoded characters correctly. 

procedure SendEmailIndy(
        const SMTPServer: string;
        const SMTPPort: integer;
        const SMTPUserName : string;
        const SMTPPassword : string;
        const FromName, FromAddress: string;
        const ToAddresses: string; //comma "," separated list of e-mail addresses
        const CCAddresses: string; //comma "," separated list of e-mail addresses
        const BCCAddresses: string; //comma "," separated list of e-mail addresses
        const Subject: string;
        const EmailBody: string;
        const IsBodyHtml: Boolean; //verses Plain Text
        const Attachments: TStrings;
              UseTLS : Boolean);
var
    smtp: TIdSMTP; // IdSmtp.pas
    TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support
    msg: TidMessage; // IdMessage.pas
    builder: TIdCustomMessageBuilder; //IdMessageBuilder.pas
    s: string;
    emailAddress: string;
begin
  TLSHandler := nil;
  msg := TidMessage.Create(nil);
  msg.ContentType := 'text/html';
  msg.CharSet     := 'UTF-8';
  msg.ContentTransferEncoding := '8bit';

  try
    if IsBodyHtml then
    begin
      builder := TIdMessageBuilderHtml.Create;
      TIdMessageBuilderHtml(builder).Html.Text := EmailBody;
    end
      else
    begin
      builder := TIdMessageBuilderPlain.Create;
    end;

    try
      Try
        if Attachments <> nil then for s in Attachments do builder.Attachments.Add(s);
        builder.FillMessage(msg);
      Except
        on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('Exception : '+E.Message){$ENDIF};
      End;
    finally
      builder.Free;
    end;

    msg.From.Name    := FromName;
    msg.From.Address := FromAddress;
    msg.Subject      := Subject;

    //If the message is plaintext then we must fill the body outside of the PlainText email builder.
    //(the PlainTextBuilder is unable to build plaintext e-mail)
    if not IsBodyHtml then msg.Body.Text := EmailBody;

    for s in ToAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then
      begin
        with msg.recipients.Add do
        begin
          //Name := '<Name of recipient>';
          Address := emailAddress;
        end;
      end;
    end;

    for s in CCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.CCList.Add.Address := emailAddress;
    end;

    for s in BCCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.BccList.Add.Address := emailAddress;
    end;

    smtp := TIdSMTP.Create(nil);
    try
      smtp.Host     := SMTPServer;
      smtp.Port     := SMTPPort;
      smtp.Username := SMTPUserName;
      smtp.Password := SMTPPassword;

      If UseTLS = True then
      Begin
        TLSHandler     := TIdSSLIOHandlerSocketOpenSSL.Create;
        smtp.IOHandler := TLSHandler;
        smtp.UseTLS    := TIdUseTLS.utUseRequireTLS;
      End;

      Try
        smtp.Connect;
        try
          try
            smtp.Send(msg)
          Except
            on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('SMTP Send Exception : '+E.Message){$ENDIF};
          End;
        finally
          smtp.Disconnect;
        end;
      Except
        on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('SMTP Connect Exception : '+E.Message){$ENDIF};
      End;
    finally
      smtp.Free;
      If TLSHandler <> nil then TLSHandler.Free;
    end;
  finally
    msg.Free;
  end;
end;

 

Edited by Yaron

Share this post

Link to post

microtronx    38

microtronx
Posted

What about assigning the Subject and Body as Utf8Encoded with 

  

msg.Subject      := UTF8Encode(Subject);
    if not IsBodyHtml then msg.Body.Text := UTF8Encode(EmailBody);

 

Share this post

Link to post

Yaron    53

Yaron
Posted

If I try to use UTF8Encode in Delphi 10.3 I get a warning "[dcc32 Warning] xxx.pas(683): W1057 Implicit string cast from 'RawByteString' to 'string'".

Share this post

Link to post

Yaron    53

Yaron
Posted

Nevermind, I figured it out, I had to move the charset code after setting the email body.

 

Share this post

Link to post

Remy Lebeau    1436

Remy Lebeau
Posted (edited)
On 9/19/2019 at 7:35 AM, Yaron said:

I need to be able to include non-latin characters in both the subject and the eMail's HTML body, what am I doing wrong?

In Unicode environments (Delphi/C++Builder 2009+, and FreePascal in UNICODESTRINGS mode), Indy encodes the Subject (and other headers) using UTF-8 by default (which can be overridden using the TIdMessage.OnInitializeISO event).  But for the message body, you need to manually set the TIdMessage.CharSet and/or TIdText.CharSet property to 'utf-8', as they default to 'us-ascii' if left blank and the corresponding ContentType property is set to a 'text/...' media type, like 'text/html' (I am open to consider changing this default to UTF-8 instead).  Note that the TIdMessageBuilder... classes already default the CharSet properties to UTF-8 in Unicode environments.

Quote

Not sure if it matters, but the HTML does contain "<meta charset="UTF-8">" in the "<HEAD>" block

No, it doesn't matter.  That has no effect on how Indy encodes the email.  And in fact, since the TIdMessage.Body and TIdText.Body properties are TStrings objects, you can't store UTF-8 text in the them anyway, only UTF-16 text.  So if you want the text encoded as UTF-8 during transmission, make sure the corresponding CharSet properties are set to 'utf-8'.

Quote

I verified the source was properly encoded by saving it to a local utf8 html file which displays the encoded characters correctly.

But, are you loading that file as UTF-8?  Does the file have a UTF-8 BOM?  If not, you must explicitly make use of TEncoding.UTF8 or equivalent when loading the file.  What does your loading code look like?

On 9/19/2019 at 7:35 AM, Yaron said:

If the message is plaintext then we must fill the body outside of the PlainText email builder. (the PlainTextBuilder is unable to build plaintext e-mail)

That is not true at all.  TIdMessageBuilderPlain has a PlainText property for the body, just as TIdMessageBuilderHtml has an Html property for the body.  I think you are just confused because the PlainText property is not declared in the TIdMessageBuilderPlain class, it is actually inherited from the TIdCustomMessageBuilder class (which means TIdMessageBuilderHtml also has the PlainText property, so that you can provide alternative plain text for HTML-unaware readers).  You should read my articles on this topic on Indy's website:

 

HTML Messages

 

New HTML Message Builder class

 

On 9/19/2019 at 8:58 AM, Yaron said:

According at this post https://stackoverflow.com/questions/9844250/not-able-to-send-utf-8-email-using-delphi-indy , I'm setting the parameters in the right order.

I have posted an update to my answer on that question.  As of 2 months ago, the order in which ContentType and CharSet proerties is set is no longer important.

22 hours ago, Yaron said:

Nevermind, I figured it out, I had to move the charset code after setting the email body.

No, you do not.  However, if you do need to specify a CharSet explicitly, you should do so on the MessageBuilder classes themselves.  TIdMessageBuilderPlain has a PlainTextCharSet property, and TIdMessageBuilderHtml has PlainTextCharSet and HtmlCharSet properties.

 

With that said, try something more like this: 

procedure SendEmailIndy(
        const SMTPServer: string;
        const SMTPPort: integer;
        const SMTPUserName : string;
        const SMTPPassword : string;
        const FromName, FromAddress: string;
        const ToAddresses: string; //comma "," separated list of e-mail addresses
        const CCAddresses: string; //comma "," separated list of e-mail addresses
        const BCCAddresses: string; //comma "," separated list of e-mail addresses
        const Subject: string;
        const EmailBody: string;
        const IsBodyHtml: Boolean; //verses Plain Text
        const Attachments: TStrings;
        UseTLS : Boolean);
var
  smtp: TIdSMTP; // IdSmtp.pas
  TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support
  msg: TidMessage; // IdMessage.pas
  builder: TIdCustomMessageBuilder; //IdMessageBuilder.pas
  s: string;
  emailAddress: string;
begin
  msg := TIdMessage.Create(nil);
  try
    if IsBodyHtml then
    begin
      builder := TIdMessageBuilderHtml.Create;
    end else
    begin
      builder := TIdMessageBuilderPlain.Create;
    end;

    try
      try
        if IsBodyHtml then
        begin
          TIdMessageBuilderHtml(builder).Html.Text := EmailBody;
          TIdMessageBuilderHtml(builder).HtmlCharSet := 'utf-8';
          TIdMessageBuilderHtml(builder).HtmlContentTransfer := '8bit';
        end else
        begin
          TIdMessageBuilderPlain(builder).PlainText.Text := EmailBody;
          TIdMessageBuilderPlain(builder).PlainTextCharSet := 'utf-8';
          TIdMessageBuilderPlain(builder).PlainTextContentTransfer := '8bit';
        end;

        if Attachments <> nil then
        begin
          for s in Attachments do
            builder.Attachments.Add(s);
        end;

        builder.FillMessage(msg);
      except
        {$IFDEF TRACEDEBUG}
        on E : Exception do
        begin
          AddDebugEntry('Email Builder Exception : ' + E.Message);
          Exit;
        end;
        {$ELSE}
        Exit;
        {$ENDIF}
      end;
    finally
      builder.Free;
    end;

    msg.From.Name    := FromName;
    msg.From.Address := FromAddress;
    msg.Subject      := Subject;

    for s in ToAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.Recipients.Add.Address := emailAddress;
    end;

    for s in CCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.CCList.Add.Address := emailAddress;
    end;

    for s in BCCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.BccList.Add.Address := emailAddress;
    end;

    smtp := TIdSMTP.Create(nil);
    try
      smtp.Host     := SMTPServer;
      smtp.Port     := SMTPPort;
      smtp.Username := SMTPUserName;
      smtp.Password := SMTPPassword;

      if UseTLS then
      begin
        TLSHandler     := TIdSSLIOHandlerSocketOpenSSL.Create(smtp);
        smtp.IOHandler := TLSHandler;
        if SMTPPort = 465 then
        begin
          smtp.UseTLS := TIdUseTLS.utUseImplicitTLS;
        end else
        begin
          smtp.UseTLS := TIdUseTLS.utUseExplicitTLS;
        end;
      end;

      try
        smtp.Connect;
      except
        {$IFDEF TRACEDEBUG}
        on E : Exception do
        begin
          AddDebugEntry('SMTP Connect Exception : '+E.Message);
          Exit;
        end;
        {$ELSE}
        Exit;
        {$ENDIF}
      end;
      
      try
        try
          smtp.Send(msg)
        except
          {$IFDEF TRACEDEBUG}
          on E : Exception do
            AddDebugEntry('SMTP Send Exception : '+E.Message);
          {$ENDIF}
        end;
      finally
        smtp.Disconnect;
      end;
    finally
      smtp.Free;
    end;
  finally
    msg.Free;
  end;
end;

Note that TIdMessageBuilderHtml CAN create plain-text emails as well, if no HTML is assigned, so you can alternatively use this instead: 

procedure SendEmailIndy(
        const SMTPServer: string;
        const SMTPPort: integer;
        const SMTPUserName : string;
        const SMTPPassword : string;
        const FromName, FromAddress: string;
        const ToAddresses: string; //comma "," separated list of e-mail addresses
        const CCAddresses: string; //comma "," separated list of e-mail addresses
        const BCCAddresses: string; //comma "," separated list of e-mail addresses
        const Subject: string;
        const EmailBody: string;
        const IsBodyHtml: Boolean; //verses Plain Text
        const Attachments: TStrings;
        UseTLS : Boolean);
var
  smtp: TIdSMTP; // IdSmtp.pas
  TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support
  msg: TidMessage; // IdMessage.pas
  builder: TIdMessageBuilderHtml; //IdMessageBuilder.pas
  s: string;
  emailAddress: string;
begin
  msg := TIdMessage.Create(nil);
  try
    builder := TIdMessageBuilderHtml.Create;
    try
      try
        if IsBodyHtml then
        begin
          builder.Html.Text := EmailBody;
          builder.HtmlCharSet := 'utf-8';
          builder.HtmlContentTransfer := '8bit';
        end else
        begin
          builder.PlainText.Text := EmailBody;
          builder.PlainTextCharSet := 'utf-8';
          builder.PlainTextContentTransfer := '8bit';
        end;

        if Attachments <> nil then
        begin
          for s in Attachments do
            builder.Attachments.Add(s);
        end;

        builder.FillMessage(msg);
      except
        {$IFDEF TRACEDEBUG}
        on E : Exception do
        begin
          AddDebugEntry('Email Builder Exception : ' + E.Message);
          Exit;
        end;
        {$ELSE}
        Exit;
        {$ENDIF}
      end;
    finally
      builder.Free;
    end;

    msg.From.Name    := FromName;
    msg.From.Address := FromAddress;
    msg.Subject      := Subject;

    for s in ToAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.Recipients.Add.Address := emailAddress;
    end;

    for s in CCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.CCList.Add.Address := emailAddress;
    end;

    for s in BCCAddresses.Split([',']) do
    begin
      emailAddress := Trim(s);
      if emailAddress <> '' then msg.BccList.Add.Address := emailAddress;
    end;

    smtp := TIdSMTP.Create(nil);
    try
      smtp.Host     := SMTPServer;
      smtp.Port     := SMTPPort;
      smtp.Username := SMTPUserName;
      smtp.Password := SMTPPassword;

      if UseTLS then
      begin
        TLSHandler     := TIdSSLIOHandlerSocketOpenSSL.Create(smtp);
        smtp.IOHandler := TLSHandler;
        if SMTPPort = 465 then
        begin
          smtp.UseTLS := TIdUseTLS.utUseImplicitTLS;
        end else
        begin
          smtp.UseTLS := TIdUseTLS.utUseExplicitTLS;
        end;
      end;

      try
        smtp.Connect;
      except
        {$IFDEF TRACEDEBUG}
        on E : Exception do
        begin
          AddDebugEntry('SMTP Connect Exception : '+E.Message);
          Exit;
        end;
        {$ELSE}
        Exit;
        {$ENDIF}
      end;
      
      try
        try
          smtp.Send(msg)
        except
          {$IFDEF TRACEDEBUG}
          on E : Exception do
            AddDebugEntry('SMTP Send Exception : '+E.Message);
          {$ENDIF}
        end;
      finally
        smtp.Disconnect;
      end;
    finally
      smtp.Free;
    end;
  finally
    msg.Free;
  end;
end;

 

Edited by Remy Lebeau
  • Like 4

Share this post

Link to post

Remy Lebeau    1436

Remy Lebeau
Posted (edited)
On 9/19/2019 at 8:05 AM, microtronx said:

What about assigning the Subject and Body as Utf8Encoded with 


msg.Subject      := UTF8Encode(Subject);
    if not IsBodyHtml then msg.Body.Text := UTF8Encode(EmailBody);

 

That will only work in non-Unicode environments (Delphi/C++Builder 2007 and earlier, and FreePascal when not in UNICODESTRINGS mode), where string is AnsiString.  In Unicode environments, string is UnicodeString instead, and so using UTF8Encode() is deprecated and unnecessary.

Edited by Remy Lebeau

Share this post

Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Go To Topic Listing Indy
×