TIdTime: datetime-method is not returning in local time for Android and iOS

[philipp.hofmann 4]

Hi,

 

I'm using TIdTime to synchronize the time for an online process but it's not working as expected:

 

Documentation TIdTime.DateTime:
- DateTime is a read-only TDateTime property that reflects the estimated current date and time according to a Time server.
- DateTime is expressed in the timezone for the local computer.

 

procedure TicTrainerF.syncTime();
var idTime:TIdTime;
    dtLocal1,dtLocal2,dtServer:TDateTime;
begin
  idTime:=TIdTime.create(nil);
  idTime.Host:='time.nist.gov';
  dtLocal1:=NOW;
  dtServer:=idTime.dateTime;
  dtLocal2:=NOW;
  mlog.info('syncTime: '+DateToStr(dtLocal1,myFormatSettings)+'/'+DateToStr(dtLocal2,myFormatSettings)+' vs '+DateToStr(dtServer,myFormatSettings));
  ...

It's fine for my windows-computers, so I can use it here.

 

1. But it's wrong for Android and iOS as the time is returned in UTC instead of the local time zone.

-> Question A: Is there a fixed rule for this so that it's always in UTC for this OS?

 

2. On my Amazon FireHD-10 tablet (FireOS/Android) I get something completly different. 

-> Question B: Was is the reason for completly different results here?

incorrect (timezone):
---------------------
syncTime (Android):        2021-01-18 16:24:50.789-16:24:51.169 vs 2021-01-18 14:24:51.000
syncTime (Android):        2021-01-18 16:24:48.071-16:24:48.428 vs 2021-01-18 14:24:49.000
syncTime (iOS):               2021-01-18 16:24:49.658-16:24:49.996 vs 2021-01-18 14:24:50.000

 

incorrect (at all):
-------------------
syncTime (FireOS/Android): 2021-01-18 16:24:48.606-16:24:49.028 vs 2021-01-18 14:40:02.000

correct:
--------
syncTime (Windows):        2021-01-18 16:24:47.612-16:24:48.359 vs 2021-01-18 16:24:48.000
syncTime (Windows):        2021-01-18 16:24:46.014-16:24:46.771 vs 2021-01-18 16:24:47.000
 

It's important for me that I can rely on the synchronization, else it's better to skip it and ask the user to take care of it.

 

Are there idea how to improve this synchronization?

 

Best regards, Philipp

[Lars Fosdal 1792]

IMO, it would be natural to use the UTC time as your comparison time, and not the local time. 

The user's time zone may vary with travel. Wherever the user is at the moment - only UTC timestamps are comparable.

 

However - if you are talking to a central server, the only way to ensure comparable timestamps is that you create them on the server, and not on the clients.

This is because you cannot rely on multiple devices time engines to be 100% in sync with eachother.

 

So - When you connect to the server, the server gives you a timestamp for the data you request. 

This you store locally. The next time you connect to the server, you compare the stored timestamp with the server's timestamp to determine if there was a change.

[philipp.hofmann 4]

For me it's fine to compare with UTC time but the documentation of TIdTime says "DateTime is expressed in the timezone for the local computer". 
And for Windows it's matching the documentation and for Android/iOS it's not matching the documentation. That's the point. 

 

In my case I need to know the difference between the local timestamp and the server timestamp. That's the reason to use TIdTime.
On Windows it's working fine, if the local time of computer A is 90 seconds ahead and of computer B it's 10 behind, I know that both computer's have a difference of 100 seconds and I have to keep this in mind for all synchronizations between both computers (it's a synchronization of video-play and it's should be insync with an accepted difference of +/-3 seconds.

That's working as expected with my routine.

[Remy Lebeau 1394]

On 1/18/2021 at 9:32 AM, philipp.hofmann said:

But it's wrong for Android and iOS as the time is returned in UTC instead of the local time zone.

That means Indy's IdGlobal.OffsetFromUTC() and/or IdGlobalProtocols.TimeZoneBias() function is returning a time zone offset of 0.0.  So, either the function(s) are failing to query the OS for the correct time zone, or the time zone is actually being reported as UTC.  You are going to have to debug into Indy's code to see which is actually the case.

On 1/18/2021 at 9:32 AM, philipp.hofmann said:

Is there a fixed rule for this so that it's always in UTC for this OS?

No.  The local time zone is always queried dynamically.  It just happens to be coming back as offset 0 in your situation.

On 1/18/2021 at 9:32 AM, philipp.hofmann said:

On my Amazon FireHD-10 tablet (FireOS/Android) I get something completly different. 

Was is the reason for completly different results here?

I can't answer that without more details about what exactly is failing.

[philipp.hofmann 4]

The error occurs in IdGlobalProtocols.TimeZoneBias:

 

function TimeZoneBias: TDateTime;
{$IFNDEF FPC}
  {$IFDEF UNIX}
var
  T: Time_T;
  TV: TimeVal;
  UT: {$IFDEF USE_VCL_POSIX}tm{$ELSE}TUnixTime{$ENDIF};
  {$ELSE}
    {$IFDEF USE_INLINE} inline; {$ENDIF}
  {$ENDIF}
{$ELSE}
  {$IFDEF USE_INLINE} inline; {$ENDIF}
{$ENDIF}
begin
{$IFNDEF FPC}
  {$IFDEF UNIX}
                                                                     
  {from http://edn.embarcadero.com/article/27890 }
  gettimeofday(TV, nil);
  T := TV.tv_sec;
  localtime_r({$IFNDEF USE_VCL_POSIX}@{$ENDIF}T, UT);
// __tm_gmtoff is the bias in seconds from the UTC to the current time.
// so I multiply by -1 to compensate for this.
  Result := (UT.{$IFNDEF USE_VCL_POSIX}__tm_gmtoff{$ELSE}tm_gmtoff{$ENDIF} / 60 / 60 / 24);
  {$ELSE}
  ...

 

If I replace this with the following it's fine:

function TimeZoneBias: TDateTime;
{$IFNDEF FPC}
  {$IFDEF UNIX}
    var nowDt:TDateTime;
  {$ELSE}
    {$IFDEF USE_INLINE} inline; {$ENDIF}
  {$ENDIF}
{$ELSE}
  {$IFDEF USE_INLINE} inline; {$ENDIF}
{$ENDIF}
begin
{$IFNDEF FPC}
  {$IFDEF UNIX}
  nowDt:=NOW;
  Result := TTimeZone.Local.ToUniversalTime(nowDt)-nowDt;
  {$ELSE}
  ...  

So it's fine for iOS and Android. For Android/FireOS there seems an additional issue with Delphi 10.3.3.

Edited January 21, 2021 by philipp.hofmann

[Remy Lebeau 1394]

9 hours ago, philipp.hofmann said:

The error occurs in IdGlobalProtocols.TimeZoneBias:

I assume, then, that gettimeofday() is failing?  That might be related to this: https://github.com/IndySockets/Indy/issues/245

Quote

If I replace this with the following it's fine

Thanks.  I'll look into incorporating TTimeZone into Indy.  There are quite a number of code fragments that can make use of it.

Quote

For Android/FireOS there seems an additional issue with Delphi 10.3.3.

What issue is that?  Is it an Indy issue, or just a Delphi issue?

Edited January 21, 2021 by Remy Lebeau

[Remy Lebeau 1394]

12 hours ago, philipp.hofmann said:

If I replace this with the following it's fine

Would one of these work for you (not sure whether UtcOffset returns positive or negative for, say, UTC-08:00)?

Result := TTimeZone.Local.UtcOffset.TotalMinutes / 60 / 24;
or:
Result := -1 * (TTimeZone.Local.UtcOffset.TotalMinutes / 60 / 24);

This would avoid having to query the system clock at all.

 

I'm thinking of updating IdGlobal.OffsetFromUTC() to use TTimeZone internally when available, in which case TimeZoneBias() under Delphi UNIX can just be:

Result := -OffsetFromUTC;

Like it is on all other platforms.

[philipp.hofmann 4]

Hi Remy, it's

Result := -1 * (TTimeZone.Local.UtcOffset.TotalMinutes / 60 / 24);

[Remy Lebeau 1394]

5 hours ago, philipp.hofmann said:

Hi Remy, it's

Result := -1 * (TTimeZone.Local.UtcOffset.TotalMinutes / 60 / 24);

In IdGlobalProtocols.TimeZoneBias(), correct?

 

I'm thinking of updating IdGlobal.OffsetFromUTC() to use TTimeZone as well.  And since most platform implementations of TimeZoneBias() are just returning -OffsetFromUTC(), that means OffsetFromUTC() should return (+1 * (TTimeZone.Local.UtcOffset.TotalMinutes / 1440)), and TimeZoneBias() should return (-1 * (TTimeZone.Local.UtcOffset.TotalMinutes / 1440)), is that correct?  Ideally, I would like to simplify TimeZoneBias() to just returning -OffsetFromUTC() on all platforms.  I don't know why it singles out Unix on Delphi but not FreePascal.

 

In which case, since OffsetFromUTC() and TimeZoneBias() are basically doing the same thing, just returning the inverse of each other, and all uses of TimeZoneBias() are only being used to adjust TDateTime values between local and UTC times, I'm thinking of just deprecating TimeZoneBias() and introducing a couple of new LocalTime<->UTC conversion functions in IdGlobal.pas to make things easier.  Then I can make better use of TTimeZone.Local.ToUniversalTime() and TTimeZone.Local.ToLocalTime() in Delphi, and LocalTimeToUniversal() and UniversalTimeToLocal() in FreePascal.

 

That would eliminate TimeZoneBias() completely, and leave only 2 remaining uses of OffsetFromUTC() - in IdGlobal.LocalDateTimeToGMT() and IdFTPCommon.MinutesFromGMT().

[philipp.hofmann 4]

Yes, I've changed it in IdGlobalProtocols.TimeZoneBias. I've tested it only with Android but I'm very sure that it's fine for MacOS/iOS also.

I can test a new version of the Indy-implementation without huge effort, if this helps you. You have to send me only the pas-file to replace.

[Remy Lebeau 1394]

1 hour ago, philipp.hofmann said:

I can test a new version of the Indy-implementation without huge effort, if this helps you. You have to send me only the pas-file to replace.

I have checked in the code as a branch on GitHub (https://github.com/IndySockets/Indy/tree/TTimeZone). Overall, there were 22 files affected by adding this new feature.

 

Edited February 3, 2021 by Remy Lebeau