Mark Williams

The TTimeZone unit is probably fine for my purposes. All I need to be able to do is get the UTC time zone from the end of the email header.

The above is a quote from the Indy help file re TIDMessage LoadFromFile. I have tested loading .eml files with TIDMessage none of which contain the specified terminator sequence "CRLF.CRLF". They load without problem. I can see online that there are quite a few posts dealing with this, but I am wondering if it is still an issue for the latest versions of INDY. If so, how do you deal with it. I have tried adding the terminator sequence to emails before reading them and IDMessage fails to read the data. I have tried with the following code: var strStream : TStringStream; begin strStream:=TStringStream.Create; try strStream.LoadFromFile(FileName); strStream.seek(0, soEnd); strStream.writeString(sLineBreak+'.'+sLineBreak); strStream.position := 0; IDMessage1.LoadFromStream(strStream); finally strStream.Free; end; The email loads as expected, but my questions are: (1) is this is actually necessary and, if so, (2) is this the right way to go about it? I am dubious because if this was needed in every case to ensure that the email opens then I would have thought it would have been incorporated in TIDmessage's Load methods.

I've had a look at the TIDMessageHelper unit. I've been testing with EML files saved to file from Gmail. Reading your comments in the Github issue you have mentioned, I would have thought these files should not be using dot transparency. However, I can read them fine from file with TIDMessage using LoadFromFile and without the helper unit. But if I am dealing with email files coming from possibly any email client presumably there is no way of knowing whether the email uses dot transparency before loading it. If that is correct then is my best option: Load the file into stream Try IDMessage.LoadFromStream - if it doesn't load... Try the helper unit Although I recall reading somewhere that if you try and read an email without dot transparency via IDMessage it takes a while and eventually triggers a time out error. Is that correct and is there any way around it? That statement should have been deleted. I had tested the sample code included in my original post without positioning the stream back to 0 so IDMessage was processing from the end of the stream. I realised my mistake and corrected it and I thought I had deleted this statement.

I can't find any examples of how to do this online. Plenty of how to send rtf, but not read it. I can't seem to find any specific unit that deals with it. What mime types do I need to check for just 'text/rtf' or also 'text/enriched' and 'text/richtext'? How do I extract the raw rtf? And how are inline images handled? Are they accessible via the attachments list or in some other way? Thanks.

Awesome. Thank you so much for your help.

I have had a look at the functions in IDCoderTNef. I will be working with a TIDAttachment object. A couple of questions: With a TNef attachment does the entire email get wrapped in the .dat file (including any attachments) or just the rtf? In other words. is it safe to parse out the TNef attachment into a new IDMessage object and just ignore the remainder of the original IDMessage? I can see there is a function for IsFileNameTNef which basically checks to see if the file is in the form mentioned by you above. But if the 'application/ms-tnef' is it possible it could be in any other form and, if so what?

I have mistakenly thought that the Body would contain plain text only. Presumably, it's also therefore possible for an html body to be contained in the body itself if the ContentType is text/html? I have yet to come across any examples of this although I can now see it would be possible. I would prefer to deal with an html body if it exists, if not then a rtf body and finally plain text. With that in mind can you see any problems with the following approach: Check the IDMessage ContentType. If it is text/???? then that is the body type I have to work with. If the ContentType is multipart/???? iterate through the attachments and find the one that best suits my purpose. Is it possible for the ContentType to be multipart and the body to be stored in TIDMessage body rather than in the message parts? I thought they were just alternate names for RTF. Having looked at the specification(s), I see not. What a pain! I have not come across either of these formats and from what I can see online they're not used that much, but I would still wish to be able to handle them if my app comes across them. I suppose the options are to display the body text as plaint text with tags and all or write a converted to convert to html and handle in same way as I handle html body. I have checked online to see if anyone has already written anything to handle the conversion and drew a blank. However, it doesn't look like it would be too difficult to write some code to convert the existing tags to their html equivalents at least the most common ones. AFAIK RTF usually inserts the raw image data into the RTF document (at least that is what WordPad does). I have created RTF emails with Outlook for text purposes. However, when I send these to a Google account they are getting converted to html along the way. I don't know if this is done at the Outlook or Google end. Outlook retains the original email as rtf and when I used Outlook's MailItem object to read it, the body is provided in RTF format and any images are handled as OLE attachments and incorporated by referenced in the rtf. But this may just be how Outlook handles them internally. Without any examples to play with I will work on the basis that the image data is embedded in the raw rtf.

Outlook's MailItem gives access to the RTFBody property for RTF formatted emails. This is described in the MS docs I need to get the RTFBody as a string that I can then write to file. I can't work out how to do this in Delphi 10.4. If found the following example from a post 8 years ago: System.Text.Encoding.UTF8.GetString(mailItem.RTFBody); This doesn't compile in 10.4. Can anyone please advise what function should be used with 10.4?

I can call LoadFromFile for TIDMessage on a MSG file. However, it has no return value to advise if it has successfully loaded the email. You only seem to know whether it has failed when you attempt to read certain properties eg Date which is returned as a 0 value. Is it therefore safe to assume that an email is in a format TIDMessage cannot read if it returns a 0 value for Date or is there a more reliable way of ascertaining if TIDMessage can read the particular email format?

It may possibly be unsafe, but it may be the only option and therefore the safest one, albeit not 100% safe. However, I'm asking for alternative and hopefully better solutions from users familiar with TIDMessage who may have encountered this issue. I have tried reading a large number of different emails including MSG. I've also submitted pdf, xml, avi, jpg, pas etc. It will have a crack at anything without complaint. Consistently (and unsurprisingly) it returns 0 for Date for files in an unreadable format (or possibly I should say incorrect format). Every email I have submitted to it in the expected format it has returned a non 0 Date value. However, it is possible (though I suspect unlikely or at least very rare) that there may be emails otherwise in the expected format, but with an incorrect Date header. I would rather my procedure does not return with an unread result in such cases. Unless someone can suggest a more accurate solution, I will go with what I have, but I would just like to hear from someone who has been here before me

I have worked through the code. Basically, it loads the file (whatever format it may be in) and attempts to parse the headers. If it cannot find a header it move on. So even with an outlook MSG file (which is in a proprietary format that IDMessage cannot read), it gives it a go and outputs something albeit not terribly useful. For the Date header, it calls a separate function that tries to parse the header value and if this is not in the right format it throws up and catches an exception and exits the function return a nil datetime value. I thought that might be the safest thing to use working on the assumption that ALL emails will have a Date header formatted in compliance with the standards and if it cannot be read that must mean that TIDMessage cannot read the format of that particular email. However, I am also aware that the standards are not strictly adhered to in all case and so my assumption is possibly unsafe. If it is unsafe then I am hoping there might be someone who has come up with a more reliable solution.

I want to extract the UTC date time that an email was sent. I cannot work out whether I can just read TIDMessage's Date property or if I have to read the Date header. I am in a time zone that is currently aligned with UTC so I can't tell whether the Date property is UTC or local!

To answer my own question "Yes I am". I can now see the function is converting the datetime to local time after deducting the offset. My error was to change the date on my pc back to the current date before reading the email with TIDMessage. The conversion from UniversalToLocal was being done at today's date when UTC is same as current date time. If I reset my clock back to August and read it again the IDMessage.Date reads the time as 10:02:026 which was the correct local time in August in UK. All very confusing! Of course, the same is also true for my original test with an unfaked email. Oh, the joys of living in a country which insists on changing the time twice a year!

I have tested with an email from September when UK was + 1 hour ahead of UTC. I get the following results: IDMessage1.Date =25/09/2020 08:31:34 Date header: Fri, 25 Sep 2020 09:31:34 +0100 Both parties in UK. The email was received by me just after 09:31 local time. Of course, that doesn't mean it was sent at 9.31. It could have been sent at 8.31 I suppose. I just changed the clock on my pc to a date where UTC different from local time and I then sent myself an email. The results were: IDMessage.Date=14/08/2020 09:02:26 Date Header: Fri, 14 Aug 2020 10:02:26 +0100 The pc clock was at 10:02 when I sent the email. So the date header is showing the local time. IDMessage date seems to me to be showing UTC time. The function for reading the date header in IDGlobalProtocols is: function GMTToLocalDateTime(S: string): TDateTime; var DateTimeOffset: TDateTime; begin if RawStrInternetToDateTime(S, Result) then begin DateTimeOffset := GmtOffsetStrToDateTime(S); {-Apply GMT and local offsets} Result := Result - DateTimeOffset; Result := {$IFDEF HAS_UniversalTimeToLocal}UniversalTimeToLocal(Result){$ELSE}Result + OffsetFromUTC{$ENDIF}; end; end; As I read it, it is converting the date header to a datetime and then deducting the value of the DateTimeOffSet. As the date header (at least in my case) is showing the local time in any event, deducting the offset is going to produce the UTC time. Am I misunderstanding something?

I am trying to work out the best way to batch read emails dropped into a windows explorer folder. I can think of three possibilities, but I can't get any of them to work! MAPI & IMessage This would probably do the job save for the fact that I cannot figure out how to open a file from explorer to use with the IMessage interface. Outlook An outlook MailItem would also probably do the job, but it has no LoadFromFile method and no other obvious method that I can find to load a file into a MailItem object. INDY TIDMessage I have tried with the following code: IDMessage1.LoadFromFile(FileName); with IDMessage1 do begin label6.caption := From.Text; label7.Caption := Recipients.EmailAddresses; label9.Caption := CCList.EmailAddresses; label10.Caption := BCCList.EmailAddresses; Label12.Caption := Subject; Label14.Caption := MsgId; Memo1.Text := Body.Text; end; Filename is a msg file dropped from outlook into explorer. It loads without complaint, but the output is wrong. Eg From.Text returns W is the correct initial for the sender. Recipients.EmailAddresses returns Again M is the correct initial. Subject returns This time with no leading double quote. I assume I am missing something, but I do not know what and I can find no helpful examples. I am looking for something that can reliably read emails in most formats. Don't know if TIDMessage is designed for this purpose. Hopefully someone can help re any of the above or even suggest any other possible solutions.

Easy solution. Just read an older email when UK time was not aligned with UTC! The answer for anyone interested is that TIDMessage's date property shows the UTC time.

So presumably I would have to reinstall INDY every time I install a new version of Delphi? Is there anything wrong with my alternative solution ie copying IDCoderTNef.pas to a separate folder along with IdCompilerDefines.inc. This compiles, although I have not yet tested if it works as I don't have any winmail.dat files.

I tried that route. Added to the protocols folder and then tried to compile IndyProtocols270.bpl. Got an error "Cannot compile package 'IndyProtocols270' which is currently required by Delphi 10.4"

Adding TIDCoderNef.pas to the protocols folder installed with Delphi doesn't work. Compiling fails to locate the unit. Saving it in a different location causes compile to fail whilst trying to find IDCompilerDefines.inc. I have copied the latter from protocols folder. to my new location, which compiles fine. However, is this the right approach or should I be dealing with this in a different way?

No. Definitely version 10. I have a host of IDCoder... units in the protocols folder, but no TIDCoderTNEF. Indy was installed same time as I installed Delphi 10.4. Just been back over the last couple of installs 18.0, 19.0, 20.0 and latest is 21.0. IDCoderTNef is not in the protocols folder of any of these installs.

@Remy Lebeau I have found reference to TIDCoderTNef, but I can't this component nor can I find an IDCoderTNef unit. I can't find anything in the help files either. Has this been replaced? If not, where can I find it please and is there any help on or any examples of how to use it that you are aware of?

I don't want to import the emails into Outlook as it is an unnecessary and added overhead (it's also not as straightforward to do so for EML files as for MSG files). My app needs to deal with EML files on a folder in Explorer. I can read the properties I need quickly using TIDMessage. I then post that data and the file to the server/database. I don't need the overhead of loading them into Outlook before doing this. If TIDMessage could handle MSG files I'd do it the same way. However, it doesn't so I am considering using Outlook to do it for me. But that comes at a cost. So it may be better for me to do as you suggest and use StgOpenStorage just for MSG files. Haven't quite made my mind up yet.

In case anyone is interested, it is fairly straightforward to open a MSG file in an Outlook MailItem (if you know how). Code below (thanks to Dmitry Streblechenko posted on StackOverflow): var App, NS, Msg:Variant; begin App := CreateOLEObject('Outlook.Application'); NS := App.GetNamespace('MAPI'); NS.Logon; Msg := NS.OpenSharedItem(fileName); I've also found a VB example of how to import EML files into Outlook: https://www.howto-outlook.com/howto/import-eml-files.htm. This shows how to import them into an Outlook folder.

I am looking to handle email messages saved to disk in whatever format. I need to open the emails, read certain properties from them and then post the file and the data to a database. If I can do this with just the file (as I can with EML using TIDMessage) then there is no need to fire up Outlook and copy the files into Outlook and then process them.

I am processing the email files for upload to a server. I don't need to add them to Outlook to do this. If I can open and handle them via TIDMessage it will save a significant amount of time.