Jump to content
Tommi Prami

Test your product with UTF-8 beta setting

Recommended Posts

Problem very poorly explained at https://quality.embarcadero.com/browse/RSP-21814

Please vote for it.

 

Now we have seen many cases of this that customers have bought new computer and this setting has been on as default. And then at least our software stops working.

 

Hope that you'll test this in on your own environments and products. and report your findings on that Ticket. I'll try to get on this also day, and report more exact findings.

 

-Tee-

  • Thanks 1
  • Confused 1

Share this post


Link to post

Never seen this option. Thanks for the heads up.

P.S. Is this done to make non-Unicode programs Unicode-compatible? I think Microsoft was late with this solution for 15 years at least.

Share this post


Link to post
Posted (edited)

Interesting option. I wonder what it changes from the app's POV.

3 minutes ago, Kryvich said:

P.S. Is this done to make non-Unicode programs Unicode-compatible? I think Microsoft was late with this solution for 15 years at least.

I doubt ancient apps will be able to use Unicode flawlessly even with this option.

Edited by Fr0sT.Brutal

Share this post


Link to post
10 minutes ago, Kryvich said:

P.S. Is this done to make non-Unicode programs Unicode-compatible? I think Microsoft was late with this solution for 15 years at least.

True dat.

Share this post


Link to post

This was tested in 10.3 But not with the latest. Anyone in 10.4 Beta should test this, on different kinds of apps.

 

And if there are problems, report issues inside beta-channels.

Share this post


Link to post

To support this option, every PAS file in a project should be saved in UTF-8 format (not ANSI!). Otherwise you'll get a bunch of compiler warnings.

 

ANSI-source-file.jpg

Share this post


Link to post
23 minutes ago, Kryvich said:

To support this option, every PAS file in a project should be saved in UTF-8 format (not ANSI!).

For new units you can enforce that with this registry setting (example for Delphi  10.3):

Quote

HKEY_CURRENT_USER\Software\Embarcadero\BDS\20.0\Editor]
"DefaultFileFilter"="Borland.FileFilter.UTF8ToUTF8"

 

  • Like 1
  • Thanks 2

Share this post


Link to post

This could use a QP to expose it in the regular Options dialogs.

  • Like 1

Share this post


Link to post
56 minutes ago, Uwe Raabe said:

"DefaultFileFilter"="Borland.FileFilter.UTF8ToUTF8"

It's strange that "Borland" still here in Delphi 10.3.3. 🙂 

Share this post


Link to post
45 minutes ago, Lars Fosdal said:

This could use a QP to expose it in the regular Options dialogs.

I think this option is not necessary, if Delphi 10.4 will set UTF-8 encoding by default for all new source files.

Share this post


Link to post
7 minutes ago, Kryvich said:

I think this option is not necessary, if Delphi 10.4 will set UTF-8 encoding by default for all new source files.

Known fact or hoping for that?

 

-Tee-

Share this post


Link to post
Posted (edited)

@Tommi Prami Just a hope. I haven't access to the 10.4 beta.

 

Note: When UTF-8 is used as a default code page in Windows, if you use national characters in a AnsiChar constant, they should be encoded in #$nn format. Otherwise you'll get "W1061 Narrowing given WideChar constant (#$0401) to AnsiChar lost information".

program AnsiCharConst;
{$APPTYPE CONSOLE}
{$R *.res}
const
  AnsiCharYo1: AnsiChar = 'Ё';
  AnsiCharYo2: AnsiChar = #$A8; // 'Ё' in ANSI 1251 Code Page;
begin
  Writeln('#', Byte(AnsiCharYo1), ', #', Byte(AnsiCharYo2));
  Readln;
end.

This program shows: "#129, #168".

 

Edited by Kryvich
  • Like 1

Share this post


Link to post
44 minutes ago, Kryvich said:

It's strange that "Borland" still here in Delphi 10.3.3.

So is CodeGear

Share this post


Link to post

Thanks god the new managements didn't kill all old references in code and IDE as first step, causing all kinds of weired issues.

  • Like 1
  • Thanks 1

Share this post


Link to post
Posted (edited)

Final conclusion after setting of UTF-8 as a default code page for Windows: if your program actively uses ANSI strings with national characters, you cannot just recompile the program and hope that it will work correctly. You will have to carefully check the code, and use AnsiStrings with a code page (i.e. AnsiString(1251)) or RawByteStrings where appropriate.

Edited by Kryvich

Share this post


Link to post
Posted (edited)
1 hour ago, Kryvich said:

When UTF-8 is used as a default code page in Windows, if you use national characters in a AnsiChar constant, they should be encoded in #$nn format.

Those who use chars > #127 in AnsiChar constant are digging a trap for themselves anyway.

I think this option is a good reason to check usage of Ansi encodings in projects. BTW, I recently found a very sly bug caused by ACP that only revealed itself on a system with an ACP different than expected. Luckily the check for reproducibility didn't require any OS change, just setting System.DefaultSystemCodePage to another value.

Edited by Fr0sT.Brutal
  • Like 2

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×