ScanTime issue

[Attila Kovacs 617]

in System.SysUtils.ScanTime() (Berlin U2)

 

there is a line 

 

if ScanChar(S, Pos, AFormatSettings.DecimalSeparator) then

 

is this nonse still the same in recent Delphi versions?

 

 

 

 

 

[Uwe Raabe 1890]

Why is that nonsense? It is probably used to get the MSec part.

[Attila Kovacs 617]

Just now, Uwe Raabe said:

Why is that nonsense?

So you mean it should be "08:00:00,00000000" in a german environment?

And what about a mixed environment? Where I'm getting the values with "." from the database and I do not want to change the decimalseparator of the application?

[Uwe Raabe 1890]

33 minutes ago, Attila Kovacs said:

So you mean it should be "08:00:00,00000000" in a german environment?

Actually I don't know the rules.

 

37 minutes ago, Attila Kovacs said:

I do not want to change the decimalseparator of the application

You can always pass a FormatSettings parameter fitting your needs.

[Uwe Raabe 1890]

BTW, if you find the current behavior is wrong, please file a bug report. Otherwise the chances of changing it are pretty low.

[Attila Kovacs 617]

1 minute ago, Uwe Raabe said:

You can always pass a FormatSettings parameter fitting your needs.

No it's not true. It's in the DAC and it fetches with point and converts with decimalseparator.

I don't know who to blame, but it's obvious that it would be easy to solve with an extra field in the formatsettings.

 

1 minute ago, Uwe Raabe said:

BTW, if you find the current behavior is wrong, please file a bug report. Otherwise the chances of changing it are pretty low.

Thank you. Now you understand my question slowly.

Before I'm reporting this, I wanted to ask if it's still the same because I'm also bored reporting things doesn't exists anymore.

 

[Uwe Raabe 1890]

1 hour ago, Attila Kovacs said:

Now you understand my question slowly.

I found that the other way round it uses a dot when formatting the milliseconds. So either way is obviously wrong.

[Attila Kovacs 617]

With FormatSettings.LongTimeFormat I can set anything I want, like 'hh:mm:ss,zzz'.

Where did you see the hardcoded dot?

[Uwe Raabe 1890]

It seems to use whatever you put into the format string. My expectation was that it replaces the dot with the current DecimalSeparator, which would be logical as secs/msecs are decimals, but that isn't the case.

 

After some searching I wasn't able to find some docs for milliseconds in local time formats. There are only international norms.

 

I suggest to file a bug report and see what they have to say about it.

[Attila Kovacs 617]

Done.

Thank you Uwe.

[Remy Lebeau 1260]

Are you referring to this ticket?

 

[RSP-41434] StrToDateTime uses Decimalseparator for microseconds

 

There is not much useful information provided in that ticket, and certainly not a reproducible example.  I can imagine Embarcadero closing the ticket as "not reproducible" or "works as designed".  Also, you filed the ticket as a bug (which it is not, IMHO, at least not the way you described it), but you are proposing a new feature at the end, so you should have filed the ticket as a Feature Request instead of as a Bug.

 

In any case, I don't understand what the actual problem is.  You say in the ticket:

Quote

Also, while it is possible to use any separator I want for DateTimeToStr() by passing it in FormatSettings.LongTimeFormat (like 'hh:mm:ss!zzz'), it does not work in reverse except with the current decimal separator.

ScanTime() uses TFormatSettings.DecimalSeparator to parse milliseconds, so what is stopping you from specifying a custom separator when parsing a string?

var Fmt: TFormatSettings;
Fmt := TFormatSettings.Create;
Fmt.TimeSeparator := ':';
Fmt.DecimalSeparator := '!';
StrToDateTime('12:34:56!789', Fmt);

Quote

My proposal is to introducing a microsencondseparator for Formatsettings.

I don't see that being necessary at all.  Especially since most platforms don't define such a separator in their respective environments anyway, AFAIK.

Edited April 17, 2023 by Remy Lebeau

[Uwe Raabe 1890]

I guess, the bug is that DateTimeToStr uses whatever is given in the format string for the separator, but StrToDateTime/ScanTime uses the DecimalSeparator. So when you provide a dot like "ss.zzz", I would expect to see the given DecimalSeparator like it is when formatting a float, but I see a dot. While there might be ways to workaround this, it can be difficult when the calls are made inside a library using the global FormatSettings and you cannot or are not allowed to tweak that code.

 

To make conversion be consistent in both directions, DateTimeToStr should replace a dot with the DecimalSeparator as well. If a dot is replaced with DecimalSeparator when formatting a float, why is it not also replaced when formatting fractional seconds? If I really want a dot I still can escape it.

 

Unfortunately there seems to be no formal specification which character to use for different locales and I neither support the proposal for a MicrosecondsSeparator. So we should either handle fractional seconds like a float or constantly use a dot for all locales. Currently the implementation between formatting and scanning differ.

 

As ever so often it happens to work on the English locale...

[Remy Lebeau 1260]

38 minutes ago, Uwe Raabe said:

I guess, the bug is that DateTimeToStr uses whatever is given in the format string for the separator, but StrToDateTime/ScanTime uses the DecimalSeparator. So when you provide a dot like "ss.zzz", I would expect to see the given DecimalSeparator like it is when formatting a float, but I see a dot.

DateTimeToStr() calls DateTimeToString() with its Format parameter set to a blank string, which will set the Format to 'C' as a default:

Quote

Displays the date using the format given by the ShortDateFormat global variable, followed by the time using the format given by the LongTimeFormat global variable. The time is not displayed if the date-time value indicates midnight precisely.

When formatting a time value, milliseconds will be output only if the format string includes the 'Z' placeholder.  But there is currently no logic to handle the decimal separator, so yes, if the format string has something like '.ZZZ' then the '.' will be output as-is as a literal character, not translated to TFormatSettings.DecimalSeparator.

Quote

To make conversion be consistent in both directions, DateTimeToStr should replace a dot with the DecimalSeparator as well. If a dot is replaced with DecimalSeparator when formatting a float, why is it not also replaced when formatting fractional seconds? If I really want a dot I still can escape it.

Sounds reasonable.  That kind of information should be stated in the bug report.  I have added a comment about it.

Edited April 17, 2023 by Remy Lebeau

[Attila Kovacs 617]

1 hour ago, Remy Lebeau said:

ScanTime() uses TFormatSettings.DecimalSeparator to parse milliseconds, so what is stopping you from specifying a custom separator when parsing a string?

 

I'm not parsng anything. Where did you get the information that I'm parsing anything? Could you cite it please?

[Attila Kovacs 617]

1 hour ago, Uwe Raabe said:

Unfortunately there seems to be no formal specification which character to use for different locales and I neither support the proposal for a MicrosecondsSeparator.

Very funny. What about LongTimeFormat then? 

[Remy Lebeau 1260]

38 minutes ago, Attila Kovacs said:

What about LongTimeFormat then? 

What about it?

[Uwe Raabe 1890]

7 hours ago, Attila Kovacs said:

What about LongTimeFormat then? 

AFAIK, it doesn't contain milliseconds unless you change it yourself.

 

 

[Attila Kovacs 617]

15 minutes ago, Uwe Raabe said:

AFAIK, it doesn't contain milliseconds unless you change it yourself.

 

It doesn't matter what it's containing by default, righ now.

If you can set it for *ToStr then you have to be able to set it to StrTo* too.

And for this reason are the other separators there. There is no string magic needed to try to figure out what would be the time separator in the format string and change it to the decimal searator.

Nonsense.

It's not even mandatory to look both the same.

Without a setting for that separator it's just a mess.

Format strings are used for *ToStr and DateSeparator/TimeSpearator/Proposed are used for StrToI*.

Why even do we debate over this? It's clear as day.

Are you both bored?

Edited April 18, 2023 by Attila Kovacs

[Uwe Raabe 1890]

35 minutes ago, Attila Kovacs said:

Format strings are used for *ToStr and DateSeparator/TimeSpearator/Proposed are used for StrToI*.

I'm not with you here. Format strings are interpreted as described in the documentation (Note: For the overloaded versions the AFormatSettings parameter is used instead of the mentioned global variable).

Quote

FormatDateTime formats the date-and-time value given by DateTime using the
format given by Format.

...

  /       Displays the date separator character given by the DateSeparator
          global variable.

  :       Displays the time separator character given by the TimeSeparator
          global variable.
 

So even on a German system you get 31.12.1986 while the format string is 'dd/mm/yyyy'. If the format string is 'dd/mm/yyyy hh:mm:ss.zzz' my expectation would be a result of 31.12.1986 23:55.32,456, because ss.zzz describes the seconds as a real number with 3 decimals.

 

If this expectation is flawed because there is some rule how seconds and milliseconds have to be separated in each country, then your proposal may be valid. Unless you can provide such a standard I keep offering my suggestions from above: Either use the DecimalSeparator or always use a dot.

[Attila Kovacs 617]

  /       Displays the date separator character given by the DateSeparator
          global variable.

  :       Displays the time separator character given by the TimeSeparator
          global variable.

 

Great! Let's keep the pattern!

 

  .       Displays the proposed separator character given by the ProposedSeparator
          global variable.

 

Why would you restrict it for . or decimalseparator?

I just can't get it, why would you prefer a hardcoded version (which would be implemented in the same way by the way) by your flavour against an universal version.

All the other separators can have any value, even an emoji. What makes this ms separator so unique that we are spending hours to discuss it?

 

Edited April 18, 2023 by Attila Kovacs

[Uwe Raabe 1890]

All the other separators have a default value provided by the operating system according to the current locale.

[Attila Kovacs 617]

49 minutes ago, Uwe Raabe said:

All the other separators have a default value provided by the operating system according to the current locale.

So what? There is no ms separator in the "operating system" and you will define it as a dot? Who gives you the right to do so?

[Attila Kovacs 617]

@Uwe Raabe

I found that in RFC3339, this separator it's called "time-fraction" and restricted to comma or dot.

However, we do not have here an RFC3339 implementation as we can set anything to decimalseparator. The question is, is it necessary to separate the two or not.

If not, it should be decimalseparator for both directions, indeed.

However, I'm still not comfortable with combining the two and it gives less freedom.

Edited April 18, 2023 by Attila Kovacs

[Attila Kovacs 617]

https://learn.microsoft.com/en-us/sql/t-sql/data-types/time-transact-sql?view=sql-server-ver16#supported-string-literal-formats-for-time

 

ISO 8601 hh:mm[:ss][.fractional seconds]

 

It seems that Microsoft did not fully comply with the ISO 8601 standard, which allows for either a dot or a comma as the fractional seconds separator, as seen in RFC 3339. This has resulted in issues with the DAC, and simply changing the decimal separator will not resolve the problem. Microsoft's SQL Server consistently uses a dot as the fractional seconds separator.

 

But using a hardcoded dot as the fractional seconds separator in the time format could cause issues when integrating with other sources that use a comma. Therefore, the best solution may be to introduce a new separator specifically for the time fraction, which can be left open or limited to either comma or dot.

 

So we are there where we started.

Edited April 19, 2023 by Attila Kovacs

[Attila Kovacs 617]

or just scanning for both