POP, IMAP and SMPT with OAuth2 (Microsoft Office 365 services)

[Milo G 0]

Hi all,

 

as you probably know Microsoft is disabling basic authentication for Microsoft Office 365 services on October 2022.
So we need to find solution for our integrated mail client using OAuth2 protocol that is compatible with Delphi 7.

Currently we use Indy 10 SMPT, POP and IMAP client components. Based on this git issue, I assume Indy doesn't have official release jet. Proposed solution is partial in a sense that user is responsible for obtaining the necessary token.

We tryed ICS 8.69 mail components and after minor changes, regarding single tenant scenario, we managed to send mail using SMTP client and receive mail using POP3 client. But they use quite a different approach and I don't see IMAP support.

 

If you are ( or were ) facing similar problem, would you share how did you handle change ?

 

Regards, Milo

[Angus Robertson 574]

ICS OAuth2 has been tested against smtp-mail.outlook.com and pop-mail.outlook.com, but they have several platforms, what changes did you need to make it work, so I can fix it for others. 

 

Sorry, ICS does not have IMAP4. 

 

Angus

 

[Remy Lebeau 1396]

2 hours ago, Milo G said:

Currently we use Indy 10 SMPT, POP and IMAP client components. Based on this git issue, I assume Indy doesn't have official release jet.

There is currently a sasl-oauth branch in Indy's GitHub repo which adds new SASL components for using OAuth tokens with the POP3/IMAP4/SMTP clients.

Quote

Proposed solution is partial in a sense that user is responsible for obtaining the necessary token.

Yes, because each service provides their OAuth tokens in their own ways.  So, outside of POP3/IMAP4/SMTP, you have to first obtain the tokens for the particular services you want to talk to, and then you can give those tokens to Indy for POP3/IMAP4/SMTP authentication.

Edited September 14, 2022 by Remy Lebeau

[mvanrijnen 123]

We went from sending mail through smtp (office365) to the MS Graph interface.

Basically, some rest services authenticated with OAUTH2 (register application in azure director), and sending message with json.

(simple messages can go in 1 call, messages with big attachments take a bit more work).

 

So with MSGRaph you can send as user with the user authenticating with OAUTH2, of send it from a registered application with clientid&secret etc. etc.

 

user: sendMail - Microsoft Graph v1.0 | Microsoft Docs

Edited September 14, 2022 by mvanrijnen

[Milo G 0]

Thank you all for quick response. I didn't know that Delphi community is so vibrant - kudos !

 

6 hours ago, Angus Robertson said:

what changes did you need to make it work, so I can fix it for others. 

 

As I wrote, it was a minor ad-hoc change ... maybe you could find a better place in code for fix. Login (CAppUrl) and token (CTokenUrl) endpoint must include Tenant ID insted of "consumers" or "common" (which Thunderbird use)
-> https://login.microsoftonline.com/{tenant}/v2.0/.
I added TenantID property next to FCliID in TIcsRestEmail class and string replace in CommonSettings function for type RestEmailMSRest or RestEmailMSSmtp:
"       
 if (FTenantID <> '') then
  begin
     FRestOAuth.FAppUrl := StringReplace(FRestOAuth.FAppUrl,'consumers',FTenantID,[rfReplaceAll, rfIgnoreCase]);
     FRestOAuth.FTokenUrl := StringReplace(FRestOAuth.FTokenUrl,'consumers',FTenantID,[rfReplaceAll, rfIgnoreCase]);
  end;
"
You could also expose RedirectURL part of login endpoint, because MS service returns error, if it's not the same as registrated in Azure AD.

 

4 hours ago, Remy Lebeau said:

Yes, because each service provides their OAuth tokens in their own ways.  So, outside of POP3/IMAP4/SMTP, you have to first obtain the tokens for the particular services you want to talk to, and then you can give those tokens to Indy for POP3/IMAP4/SMTP authentication.

 

I get that, it's general solution. But from user stand point it's quite complicated and each of us have to implement it.
There are few major playes regarding mailing service, so it would be nice to have common solution for key players.
In my case, regarding Microsoft service, is a metter of setting scope parameter when requesting token.
If POP then "https://outlook.office.com/POP.AccessAsUser.All" if SMTP "https://outlook.office.com/SMTP.Send" if IMAP "https://outlook.office.com/IMAP.AccessAsUser.All"
Or even better property where you can select all 🙂 along with commons like "offline_access" and "https://outlook.office.com/MailboxSettings.Read" for reading user profile.

I have couple of days to decide and will try out suggested sasl-oauth branch using ICS solution (web server redirect to localhost).

 

3 hours ago, mvanrijnen said:

We went from sending mail through smtp (office365) to the MS Graph interface.

Thank you for you pointer on Microsoft Graph is a viable option.

[Angus Robertson 574]

Quote

We went from sending mail through smtp (office365) to the MS Graph interface.

ICS has a TIcsRestEmail component that supports the MS Graph interface, just basic functions to read and send email at the moment, demo in the OverbyteIcsHttpRestTst sample.

 

Angus

 

[KenR 29]

Quote

ICS has a TIcsRestEmail component that supports the MS Graph interface, just basic functions to read and send email at the moment, demo in the OverbyteIcsHttpRestTst sample.

Is TIcsRestEmail documented anywhere as I can't get the demo to work.

[Angus Robertson 574]

Sorry, the only documentation is at the top of the OverbyteIcsSslHttpOAuth.pas unit. 

 

I tested TIcsRestEmail against Gmail recently, but Microsoft was always much harder due to multiple APIs and account types.  It was working when I initially wrote it, but my secrets are now expired and I need to refresh them to gain access again. 

 

Angus

[KenR 29]

I am using the "Email Settings" tab. The "Test Redirect" button works. The "Login to Email" button pulls up the OAuth screen and allows me to give permission for the application but then displays the following. Any ideas?

Failed to Generate App Token

App Authorization Code: M.R3_BL2.dd7acb9d-a4ab-a11d-a988-120004ca064b

[Angus Robertson 574]

I've just got it working with my live.com account, after accepting several application warning screens during the OAuth2 login process, I've previously had to increase the ICS timeout to waiting long enough for all this extra padding to be clicked past.  

 

Sorry, no idea about specific error, Microsoft does not any of this stuff easy for developers, at least not of my age.

 

Angus

 

[KenR 29]

On 9/14/2022 at 5:33 PM, Remy Lebeau said:

There is currently a sasl-oauth branch in Indy's GitHub repo which adds new SASL components for using OAuth tokens with the POP3/IMAP4/SMTP clients. 

Yes, because each service provides their OAuth tokens in their own ways.  So, outside of POP3/IMAP4/SMTP, you have to first obtain the tokens for the particular services you want to talk to, and then you can give those tokens to Indy for POP3/IMAP4/SMTP authentication.

I have an application which sends mail-merged emails on behalf of multiple users per installation. They configure the email server thereselves via options in the application and I know some are using Outlook 365. This is not the main function of the software but is very important as, besides sending emails in the application, a service also sends reminder emails during the night. I have always used Indy for this but looks like I am going to have problems come 22nd October and am looking for the best way forward. I really do not want to switch away from Indy as it has been very reliable for many years. Can you please give any further information on how I can do this using the sasl-auth branch?

[Remy Lebeau 1396]

On 9/17/2022 at 8:13 AM, KenR said:

I have always used Indy for this but looks like I am going to have problems come 22nd October and am looking for the best way forward.

The simplest solution that will not require any code changes is to have your Outlook/Exchange users turn on 2-step verification in their accounts and then generate application-specific passwords.  Otherwise, you will have to update your code to use OAuth2 when connected to Outlook/Exchange.

[omnibrain 15]

On 9/18/2022 at 11:34 PM, Remy Lebeau said:

application-specific passwords.

My Understanding is, that App passwords also will stop working: Office 365 App passwords | Blog | Limilabs

 

I tried to jump through a lot of hoops and use OAuth with Clever Internet Components, but it all lead to errors, but I might try again, because:

I tried the TMS FNC Cloud Outlook Mail component, but you can only read and send mail. I also need to delete or at least move mail.

 

Has someone found another component? I don't have the time to implement and maintain the Graph API by myself.