Delphi - send mail from Office 365 with EWS protocol

[J23 0]

Hello,
does ICS allow to send mail from Office365 account using the Exchange Web Service protocol?
I want my app to be able to send emails from my Office 365 account.
To start with, I decided to try GMail and SMTP. Thanks to ICS it was easy.
I decided that now I will only change the configuration and send an e-mail from O365. Failed to.
For O365 SMTP does not work even in Outlook (desktop). A few days ago I found out from MS technical support that now I can only send mail using Exchange.
Is it true?
Thanks for any help.
J23

[Stéphane Wierzbicki 45]

I guess that MS Technical support was wrong.

There is several ways to send emails throught SMTP (you will need a licence).

Have a look at this URL https://learn.microsoft.com/en-us/exchange/mail-flow-best-practices/how-to-set-up-a-multifunction-device-or-application-to-send-email-using-microsoft-365-or-office-365

Edited March 5, 2023 by Stéphane Wierzbicki

[Stéphane Wierzbicki 45]

I've asked ChatGPT. He proposed this:

 

To authenticate with Microsoft 365 and send emails through Exchange Online, you can use the Microsoft Graph API to obtain an access token, and then include that access token in the HTTP headers when sending the email. Here's an example Delphi code that shows how to do this:

 

delphi

Copy code

 

uses

  System.Net.URLClient, System.Net.HttpClient, System.Net.HttpClientComponent;

 

procedure SendEmail;

var

  AuthToken: string;

  Http: TNetHTTPClient;

  Request: TNetHTTPRequest;

  Response: IHTTPResponse;

  Body: TStringStream;

begin

  // First, obtain an access token using the Microsoft Graph API.

  Http := TNetHTTPClient.Create(nil);

  try

    Body := TStringStream.Create('grant_type=client_credentials&client_id=<your_client_id>&client_secret=<your_client_secret>&scope=https://graph.microsoft.com/.default');

    try

      Response := Http.Post('https://login.microsoftonline.com/<your_tenant_id>/oauth2/v2.0/token', Body, nil);

      try

        if Response.StatusCode = 200 then

        begin

          AuthToken := TJSONObject.ParseJSONValue(Response.ContentAsString).GetValue<string>('access_token');

        end

        else

        begin

          // Handle error.

          Exit;

        end;

      finally

        Response := nil;

      end;

    finally

      Body.Free;

    end;

  finally

    Http.Free;

  end;

 

  // Now, send the email using the access token.

  Http := TNetHTTPClient.Create(nil);

  try

    Request := TNetHTTPRequest.Create(nil);

    try

      Request.Method := TNetHTTPRequest.TMethod.rmPOST;

      Request.URL := 'https://graph.microsoft.com/v1.0/me/sendMail';

      Request.ContentType := 'application/json';

      Request.CustomHeaders['Authorization'] := 'Bearer ' + AuthToken;

      Request.Source := TStringStream.Create('{ ' +

        '"message": {' +

          '"subject": "Test email",' +

          '"body": {' +

            '"contentType": "Text",' +

            '"content": "This is a test email."' +

          '},' +

          '"toRecipients": [{' +

            '"emailAddress": {' +

              '"address": "<recipient_email_address>"' +

            '}' +

          '}]' +

        '},' +

        '"saveToSentItems": "true"' +

      '}', TEncoding.UTF8);

      Response := Http.Execute(Request);

      try

        if Response.StatusCode <> 202 then

        begin

          // Handle error.

        end;

      finally

        Response := nil;

      end;

    finally

      Request.Free;

    end;

  finally

    Http.Free;

  end;

end;

 

Replace <your_client_id>, <your_client_secret>, <your_tenant_id>, and <recipient_email_address> with your own values.

 

Note that you'll need to register an application in the Azure Portal and grant it the appropriate permissions to use the Microsoft Graph API

[J23 0]

I read a lot of materials on this topic, and finally decided to try the simplest solution - Thunderbird and Outlook applications. In both cases, it's easy to set up IMAP - it works In both of these programs. But I was unable to send an e-mail using SMTP. I don't need any license for this.

[J23 0]

9 minutes ago, Stéphane Wierzbicki said:

I've asked ChatGPT. He proposed this:

Thank you for your reply.

I've tried so many options that until I configure any mail application I won't even believe Mr. GPT 😉

Please note that in Outlook, my account configures itself as Exchange instantly. This means that I am the legitimate owner of this account.

Why not SMTP?

[Angus Robertson 574]

Never heard of Exchange Web Service protocol, but ICS has a TIcsRestEmail component that uses the graph.microsoft.com/v1.0/me/sendMail API, there is a sample in OverbyteIcsHttpRestTst,dpr.

 

Angus

[J23 0]

53 minutes ago, Angus Robertson said:

Never heard of Exchange Web Service protocol

Maybe I didn't understand something?:

https://www.emailarchitect.net/easendmail/kb/delphi.aspx?cat=15

"Exchange Web Service (EWS)

Exchange Web Services (EWS), an alternative to the MAPI protocol, is a documented SOAP based protocol introduced with Exchange Server 2007. We can use HTTP or HTTPS protocol to send email with Exchange Web Services (EWS) instead of SMTP protocol. I only suggest that you use EWS protocol in Exchange 2007/2010/2013/2016 or later version. Office365 also supports EWS very well."

[mvanrijnen 123]

i believe EWS is deprecated, take a look at the MS Graph  API:

Microsoft Graph overview - Microsoft Graph | Microsoft Learn

 

[J23 0]

22 hours ago, Angus Robertson said:

ICS has a TIcsRestEmail component that uses the graph.microsoft.com/v1.0/me/sendMail API, there is a sample in OverbyteIcsHttpRestTst,dpr

Thanks for your reply.

OverbyteIcsHttpRestTst,dpr successfully sends mails as OAuth2 from GMail - not from O365. It uses two components: HttpRest and RestOAuth.

But I've not found TIcsRestEmail component. Would you be so kind as to tell me where to find it (and how to use it)?

Inside this example, there is no such component.

I've downloaded ICS-V8.58 (Nov, 2018).

J23

Edited March 6, 2023 by J23

[Angus Robertson 574]

The ICS download page is http://wiki.overbyte.eu/wiki/index.php/ICS_Download

 

The latest is V8.70 which added more features for Office 365 User Authorities. 

 

Angus

 

[J23 0]

53 minutes ago, Angus Robertson said:

The latest is V8.70

Thank you very much, I was looking there: http://www.overbyte.eu/frame_index.html

and and there the latest version is from 2018 (signed: latest release) ;-(

Now, I have downloaded the latest version and will check.

Thanks again,

J23

[J23 0]

On 3/5/2023 at 9:01 PM, Angus Robertson said:

OverbyteIcsHttpRestTst,dpr

Hi,

I tried but couldn't send an email.

In the Send Email tab, I entered:

- From:                                 my personal Outlook mail address

- Recipients:                          my another mail address

 

In the Email Settings tab, I entered:

- Email Provider:                   Microsoft REST APIs

- MS User Autority:              Consumers

- App Client ID:                    App value from Azure

- Client Secret:                      App value from Azure

- Email Account:                    my personal Outlook mail address

- OAuth Authentication:        Local Web Server

 

I left all other fields unchanged. Should I enter anything else?

 

I tried to run (Email Settings tab):

Test Redirect button - Test Redirect OK

 

Login to Email button - Browser shows message (without login screen):

"unauthorized_client: The client does not exist or is not enabled for consumers. If you are the application developer, configure a new application through the App Registrations in the Azure Portal at https://go.microsoft.com/fwlink/?linkid=2083908."

 

After change - MS User Autority: to common

Login to Email button - Browser shows login screen, but when I tried to log with my personal Outlook mail address I was rejected with message:

"You cannot log in here with your personal account. Use a work or school account instead."

 

What am I doing wrong? The user of my application should authenticate the sending of the email with his data.

Tried logging in with my Azure account details but of course no luck either.

Thanks in advance for any help.

J23

 

[mjustin 23]

On 3/5/2023 at 7:33 PM, J23 said:

I decided that now I will only change the configuration and send an e-mail from O365. Failed to.

According to https://support.microsoft.com/en-us/office/pop-imap-and-smtp-settings-8361e398-8af4-4e97-b147-6c6c4ac95353

sending e-mails via SMTP from Office 365 / Microsoft 365 should work with these settings:

• Server: smtp.office365.com
• Port: 587
• Encryption: STARTTLS

User name / Password: as given in https://account.microsoft.com/

 

[Angus Robertson 574]

Almost certainly your REST email problem is down to the complexity and multitude of Azure permission settings.  Hard to be specific without knowing the email account type, that must match the permissions. 

 

I've tried to explain it as best I can in the 'Google and Microsoft OAuth2 Email Application Accounts' comments in the OverbyteIcsSslHttpOAuth.pas unit,

 

Angus

 

[J23 0]

8 minutes ago, Angus Robertson said:

complexity and multitude of Azure permission settings

MS is the master of complications 😉

As I wrote earlier, sending an email with the help of ICS (SMTP, OAuth2) from the GMail account was successful without any problems.

I tried using MS Technical Support and got conflicting answers from various consultants.

[mjustin 23]

On 3/5/2023 at 10:26 PM, mvanrijnen said:

i believe EWS is deprecated

 

It will not receive any feature updates, see this note on https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

Quote

"In 2018, we announced that Exchange Web Services would no longer receive feature updates and we recommended that application developers switch to using Microsoft Graph. See Upcoming changes to Exchange Web Services (EWS) API for Office 365."

Original announcement is here:

https://techcommunity.microsoft.com/t5/exchange-team-blog/upcoming-changes-to-exchange-web-services-ews-api-for-office-365/ba-p/608055

Edited March 7, 2023 by mjustin
Updated links to announcements

[J23 0]

17 minutes ago, mjustin said:

sending e-mails via SMTP from Office 365 / Microsoft 365 should work with these settings

Thank you, I know that,

but problem is IMHO with modern authentication - OAuth2 or using the graph.microsoft.com

and as Angus said with Azure permission settings.

J23

 

[mjustin 23]

4 minutes ago, J23 said:

Thank you, I know that,

but problem is IMHO with modern authentication - OAuth2 or using the graph.microsoft.com

Can you be more specific, what is the exact problem? SMTP can still be used with basic auth. Modern Authentication is not required. (However, it is a security option, which can be enforced and configured by the organization)

Edited March 7, 2023 by mjustin

[J23 0]

55 minutes ago, mjustin said:

SMTP can still be used with basic auth. Modern Authentication is not required

Neither GMail nor Microsoft allow Basic authentication

https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

 

The exception is the use of Application password, but my University that uses Office 365 mail does not agree to this solution.

 

By the way: MS Technical Support says that the Outlook application cannot be configured to send mail (Microsoft account) using SMTP - i tried and i didn't make it.

Edited March 7, 2023 by J23

[mjustin 23]

45 minutes ago, J23 said:

Neither GMail nor Microsoft allow Basic authentication

https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

This page says for SMTP it is still accessible without modern authentication, and explains why (existing hardware which can't be updated):

Quote

The reason SMTP will still be available is that many multi-function devices such as printers and scanners can't be updated to use modern authentication.

(Yes, the text on this page may be hard to read, as it is related three types of authentication - Basic authentication, SMTP AUTH. and modern authentication). But regarding Basic authentication and SMTP AUTH, it gets clearer by reading the the linked article "Improving Security - Together" at https://techcommunity.microsoft.com/t5/exchange-team-blog/improving-security-together/ba-p/805892

Quote

Please note this change does not affect SMTP AUTH – we will continue supporting Basic Authentication for the time being.  There is a huge number of devices and appliances that use SMTP for sending mail, and so we’re not including SMTP in this change

 

 

But if SMTP AUTH is not permitted (and therefore not enabled) by the organization, there are little choices. Microsoft recommends using the Graph API:

Quote

In 2018, we announced that Exchange Web Services would no longer receive feature updates and we recommended that application developers switch to using Microsoft Graph.

 

 

Edited March 7, 2023 by mjustin

[Angus Robertson 574]

The ICS SMTP and MailQueue components will send email using Office365, again provided all the permissions are set in Azure.  Look at the OverbyteIcsMailQuTst.dpr sample which allows two different SMTP servers to be specified and will retry both multiple times to send email.

 

In Azure, set Supported account types to All Microsoft account users, then in Graph permissions enable all of these (probably too many...);

 

email, Mail.Read, Mail.ReadWrite, Mail.Send, MailboxSettings.Read, offline_access, openid, POP.AccessAsUser.All, profile, SMTP.Send, User.Read 

 

REST, SMTP and POP3 should then work with OAuth2.

 

Angus

 

 

[J23 0]

On 3/7/2023 at 5:22 PM, Angus Robertson said:

The ICS SMTP and MailQueue components will send email using Office365, again provided all the permissions are set in Azure.  Look at the OverbyteIcsMailQuTst.dpr sample

Hello,
It's been a few months and I've tried to get back on topic. However, now I managed to clear some doubts and my question is a bit more precise.
I used the trial version of the EASendMail control. I was able to send the mail, so now I am sure that my application registration and its settings in Azure are correct.
And here the problem arose. At first I thought I also verified all URIs, URLs, etc. are correct. However, the same settings applied to the Overbyte Demo did not allow me to get a Token.
In the version with the EASendMail control, I have:

    clientID = ....
    clientSecret = ...
    scope = 'https://graph.microsoft.com/Mail.Send%20offline_access%20email%20openid';
    authUri = 'https://login.microsoftonline.com/common/oauth2/v2.0/authorize';
    tokenUri = 'https://login.microsoftonline.com/common/oauth2/v2.0/token';

httpListener.Create1('127.0.0.1', 0);
redirectUri := Format('http://127.0.0.1:%d', [httpListener.ListenPort]);//http://127.0.0.1:51034
    authorizationRequest := authUri;
    authorizationRequest := authorizationRequest + '?response_type=code&scope=' + scope;
    authorizationRequest := authorizationRequest + '&redirect_uri=' + redirectUri;//http://127.0.0.1:51034
    authorizationRequest := authorizationRequest + '&client_id=' + clientID;
    authorizationRequest := authorizationRequest + '&prompt=login';
browserUi.OpenUrl(authorizationRequest);
httpListener.GetRequestUrl(-1)
requestUri := httpListener.RequestUrl;                             //have the AuthorizationCode!

httpRequest := TServerXMLHTTP60.Create(nil);
        tokenRequestBody := 'code=' + code;
        tokenRequestBody := tokenRequestBody + '&redirect_uri=' + redirectUri;
        tokenRequestBody := tokenRequestBody + '&client_id=' + clientID;
        tokenRequestBody := tokenRequestBody + '&grant_type=authorization_code';

        httpRequest.setOption(2, 13056);
        httpRequest.open('POST', tokenUri, true);
        httpRequest.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
        httpRequest.send(tokenRequestBody);

        while( httpRequest.readyState <> 4 ) do
            httpRequest.waitForResponse(1);

        status := httpRequest.status;  //=200!
        result := httpRequest.responseText;                           //have the Token!

*********************************************************************************

 

What am I setting wrong in OverbyteIcsMailQuTst?

When I try to "Login to App" with above Scope I get the message:

AADSTS650053: The application xxx asked for scope 'Mail.Send%20offline_access%20email%20openid' that doesn't exist on the resource '00000003-0000-0000-c000-000000000000'. Contact the app vendor.

 

Previously, I tried the default Scope setting: https://graph.microsoft.com/Mail.Send

Then I get the message:

Failed to Generate App Token

but now the Authorization Code appears!

 

Thank you in advance for any help. I feel like I'm going round in circles.

J23

Edited April 24, 2023 by J23

[Angus Robertson 574]

Just signed into my live.com account ok, try using the scope:

 

offline_access https://outlook.office.com/SMTP.Send https://outlook.office.com/POP.AccessAsUser.All

 

Note no escaped spaces, they probably got double encoded.

 

Angus

 

[J23 0]

1 hour ago, Angus Robertson said:

offline_access https://outlook.office.com/SMTP.Send

Thank you very much Angus for your support. Unfortunately, it didn't help.

In the meantime, I tried in that application using EASendMail to apply:

scope = 'https://graph.microsoft.com/Mail.Send'

and it works fine too.

 

I noticed a strange thing: authentication requests sent from these apps are slightly different:

(xxxx replaces long codes here)

EASendMail (browser bar):

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?response_type=code&scope=https://graph.microsoft.com/Mail.Send&redirect_uri=http://127.0.0.1:60432&client_id=xxxx&prompt=login

 

OverbyteIcsHttpRestTst: (Debugger -BrowserURL text from OverbyteIcsSslHttpOAuth unit)

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?response_type=code&amp;client_id=xxxx&amp;redirect_uri=http%3A%2F%2F127.0.0.1%3A8080&amp;state=ICS-17630046&amp;scope=https%3A%2F%2Fgraph.microsoft.com%2FMail.Send

 

but the browser bar shows much more:

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?response_type=code&amp;client_id=xxxx&amp;redirect_uri=http%3a%2f%2f127.0.0.1%3a8080&amp;state=ICS-17630046&amp;scope=https%3a%2f%2fgraph.microsoft.com%2fMail.Send&amp;sso_nonce=xxxx&amp;client-request-id=xxxx&amp;mscrid=xxxx

 

Browser behavior is also different:

EASendMail: Requests MailAddress, Password and SMS verification

OverbyteIcsHttpRestTst: It requests MailAddress, Password and without SMS verification shows message:

"Failed to Generate App Token............."

The "............." looks like it's Authorization Code

 

Even the login windows are slightly different graphically.

 

Completely lost,

J23

Edited April 25, 2023 by J23

[Angus Robertson 574]

You got a scope error, so that is the parameter to adjust until it works.  Try a single scope without any spaces. 

 

The scopes need to match the settings in the Azure account, but you only need those for the current task.

 

Angus