StrToFloat () all combinations of decimal separator and lang. settings

[bernhard_LA 0]

need a good and flexible solution to  convert string to floats

 

the string  can be 1,99  vs. 1.99   the OS can be  german or english or .... 

 

not working  code goes like this :-( 

 

internalFormatSettings := TFormatSettings.Invariant();

 

Result := StrToFloat(FloatStr, internalFormatSettings); 

 

[Stefan Glienke 2002]

function MyStrToFloat(const S: string): Extended;
const
  Komma: TFormatSettings = (DecimalSeparator: ',');
  Dot: TFormatSettings = (DecimalSeparator: '.');
begin
  if not TryStrToFloat(S, Result, Komma) then
    Result := StrToFloat(S, Dot);
end;

 

[David Heffernan 2345]

I prefer to replace dot or comma with the locale's decimal sep and then use StrToFloat. 

 

Although in reality I don't use the Emba conversion functions because they are defective, i.e. not accurate. 

Edited August 6, 2021 by David Heffernan

[Stefan Glienke 2002]

27 minutes ago, David Heffernan said:

I prefer to replace dot or comma with the locale's decimal sep and then use StrToFloat. 

Unless you do that in place with PChar pointer magic that's a heap allocation! 😉

[David Heffernan 2345]

1 hour ago, Stefan Glienke said:

Unless you do that in place with PChar pointer magic that's a heap allocation! 😉

Yeah I don't do a heap allocation. I use a local fixed length array. 

[David Heffernan 2345]

2 hours ago, David Heffernan said:

Yeah I don't do a heap allocation. I use a local fixed length array. 

That's probably not true. I think I dtoa accepts both, or if not then I modified it. Since I'm in control of the conversion code thats the right way. 

[Tommi Prami 130]

When Thousand separators come into play, it starts to get almost impossible, (At least in here).

1,006.66 or 1 006,66 or 1 006.66 etc...

 

And if sometimes there are no decimals. How to fix US version without decimals 1,006 and Finnish ones with them 6,66. As far as I know there can't be universal routine to rule them all. 

 

US I think thousand separator is Comma, here it is decimal separator. THis is one of the places there would have been nice to have global standard, and only one 🙂

 

-Tee-

Edited August 9, 2021 by Tommi Prami

[Pat Foley 51]

Quote

US I think thousand separator is Comma, here it is decimal separator. THis is one of the places there would have been nice to have global standard, and only one 🙂

 

 

  numeric keypad shows . writes , 

Edited August 9, 2021 by Pat Foley
fixed link

[David Heffernan 2345]

3 hours ago, Tommi Prami said:

When Thousand separators come into play, it starts to get almost impossible, (At least in here).

1,006.66 or 1 006,66 or 1 006.66 etc...

 

And if sometimes there are no decimals. How to fix US version without decimals 1,006 and Finnish ones with them 6,66. As far as I know there can't be universal routine to rule them all. 

 

US I think thousand separator is Comma, here it is decimal separator. THis is one of the places there would have been nice to have global standard, and only one 🙂

 

-Tee-

So far as I am aware, routines for conversion from string to float do not accept strings with thousand seps so this is just not an issue. 

[Tommi Prami 130]

8 minutes ago, David Heffernan said:

So far as I am aware, routines for conversion from string to float do not accept strings with thousand seps so this is just not an issue. 

Has to be handled anyways.

 

-Tee-

[David Heffernan 2345]

19 minutes ago, Tommi Prami said:

Has to be handled anyways.

 

-Tee-

Try passing such a string to StrToFloat and see how it works out for you. 

[Fr0sT.Brutal 900]

You could also prepare input for processing: remove ^[\.,\d], replace , to . and convert invariantly

[Tommi Prami 130]

On 8/9/2021 at 11:14 AM, David Heffernan said:

Try passing such a string to StrToFloat and see how it works out for you. 

That seems to be correct, thousands separators are not accepted, so have to first handle them self.

[David Heffernan 2345]

38 minutes ago, Tommi Prami said:

That seems to be correct, thousands separators are not accepted, so have to first handle them self.

Well, not really. Certainly in my experience when dealing with floating point data, you can just mandate that it doesn't have thousand separators.

 

Perhaps if you are dealing with currency then you'd need to handle thousand separators but for other data it's not necessary. 

[Lars Fosdal 1792]

These things are perpetual headaches, as is the time and date separators. 

The default Norwegian Windows language setting is using period for both, which confuses the hell out of the Delphi string to datetime decoders.

[Fr0sT.Brutal 900]

I'd mark deprecated all functions that use global FormatSettings and this variable itself. Otherwise it's quite hard to track and kill all hidden usages

[Bill Meyer 337]

2 hours ago, Lars Fosdal said:

The default Norwegian Windows language setting is using period for both, which confuses the hell out of the Delphi string to datetime decoders.

Not surprising it would. Can't help wondering what the reasoning may have been for such a default.

[Fr0sT.Brutal 900]

2 hours ago, Lars Fosdal said:

The default Norwegian Windows language setting is using period for both, which confuses the hell out of the Delphi string to datetime decoders.

Very nice idea for testing BTW - change system settings to something wild and see if software stands it.

[David Heffernan 2345]

Seriously veering off topic here. Date strings don't go through StrToFloat. 

[Lars Fosdal 1792]

The common trait is that both floats and dates have separator character challenges.

 

For floats, the only reliable solution is to KNOW the input format and do the necessary stripping/replacement before passing the string to the converter.

 

In some of my older input parsers, I stripped spaces, then checked for the presence of , and . and did the following processing

- if only one exists, don't touch it

- if more than one of a kind exists, remove them all

- if both exists - remove all but the last one

Which still is hopeless if the 1,000 is 1000 and not 1 with three decimals.

[David Heffernan 2345]

2 hours ago, Lars Fosdal said:

For floats, the only reliable solution is to KNOW the input format and do the necessary stripping/replacement before passing the string to the converter.

Correct.

 

And as a developer that does a lot of parsing of scientific / engineering data, I just don't recognise the issue of dealing with thousand seperators. In the data that I process, they just don't appear. You don't bother trying to strip them, you just require the data not to have them.

 

2 hours ago, Lars Fosdal said:

In some of my older input parsers, I stripped spaces, then checked for the presence of , and . and did the following processing

- if only one exists, don't touch it

- if more than one of a kind exists, remove them all

- if both exists - remove all but the last one

Which still is hopeless if the 1,000 is 1000 and not 1 with three decimals.

Exactly.  It's a mugs game trying to handle data whose formatting is ambiguous, so don't do it.

[Lars Fosdal 1792]

I was working with various kinds of financial data, weather data and power data (prices, volumes, etc), and thousand separators usage was variable. Spaces, commas, dots, the lot.

It was a hodge-podge of formats since very few standard exchange formats existed at the time.

 

Even vendors that you had contractual agreements with, would change the format on the fly, without notice. 
"Yeah, we changed the format. Nobody told you?"

[skyzoframe[hun] 4]

function StrToInt00(sString:String; out f : Float32):Boolean;
  var
    fs:TFormatSettings;
begin
  try
    try
      GetLocaleFormatSettings(LOCALE_SYSTEM_DEFAULT, fs);
        // remove space
        if ContainsText(sString, ' ')
        then sString := StringReplace(sString,' ','',[rfReplaceAll, rfIgnoreCase]);      

        if ContainsText(sString, ',') then 
        begin
          if fs.DecimalSeparator='.' 
          then sString :=  StringReplace(sString,',','.',[rfReplaceAll, rfIgnoreCase]);
        end
        else if ContainsText(sString, '.') then 
        begin
          if fs.DecimalSeparator=',' 
          then sString :=  StringReplace(sString,'.',',',[rfReplaceAll, rfIgnoreCase]);
        end; 

    finally
      f :=  StrToFloat(sString);	
      Result := True;
    end;
  except
    Result := False;
  end;
end;

 

Edited August 23, 2021 by skyzoframe[hun]

[skyzoframe[hun] 4]

 

if you have problems with "Thousand Separators", remove it from the string!

if   (ContainsText(sString, ',')) 
 and (ContainsText(sString, '.')) 
then
  begin
  
  if  (fs.ThousandSeparator=',') then sString := StringReplace(sString,',','',[rfReplaceAll, rfIgnoreCase])
  else if (fs.ThousandSeparator='.') then sString := StringReplace(sString,'.','',[rfReplaceAll, rfIgnoreCase]); 

  end;

hm.. it is stupid... if string="1,1546.98" and ThousandSeparator=','  then result="1,154698"

Edited August 23, 2021 by skyzoframe[hun]

[Rollo62 536]

On 8/17/2021 at 9:48 AM, Lars Fosdal said:

Which still is hopeless if the 1,000 is 1000 and not 1 with three decimals.

I agree to that.

Maybe the best way, to catch any of the remaining formats is,

to seek the string reversely, from end to start.

 

1. Then take the last separator (first found when searching reversely) in the string, as preliminary decimal separator.

2. If there are more separators in the string, they could be safely ignored, they can only be thousands separators.

3. If separator 1.) is the only separator in the string, only then you might run into an unclear situation, as you explained above.

4. But you still could try to count the number of decimals after the last separator

    if there are <> 3, then you still could be very sure that it's NOT a thousands separator, but a decimal separator.

5. Even if you have >= 6 decimals, it should be a decimal separator, as I rarely expect having a million separator without a thousands separator.

    ( Still somebody could do nasty things like that )

6. Only if there are exactly 3 decimals, then you better know your format source.

 

 

 

Edited August 23, 2021 by Rollo62