Jump to content
Max Terentiev

Control TDataSet strings encoding (auto encode to utf-8)

Recommended Posts

Hi,

 

I moving my project from D2007 to Rio. My project uses ElevateSoftware DBISAM database engine. I was very happy with it but it's not support unicode and I got problems with saving string data from new unicode-enabled delphi controls to TDBISAMTable: i got ???? instead of unicode symbols. I can handle this problem by manually encode/decode string fields values to/from UTF8. But I need to do it in several thousands places in my code :((((

 

Maybe somebody already found elegant solution for this problem ? Maybe overriding some methods in TDataSet ?

 

I found TStringField.Transliterate and TDataSet.Translate it's almost that I need but function

 

function TDataSet.Translate(Src, Dest: PAnsiChar; ToOem: Boolean): Integer;

 

have PAnsiChar params and I have data loss (???? instead of unicode) here, so I can't use it for background UTF8 conversions. But I need something similar !

 

Thanks for help !

Edited by Max Terentiev

Share this post


Link to post

maybe you can override a TStringField and tell your program to use that as the default.

i, on the other hand am very curious for the solution you seem to have found

 

René

 

  TUnicodeStringField = class(TStringField)
  protected
    function GetAsString: string; override;
    procedure SetAsString(const Value: string); override;
  end;

implementation

{ TUnicodeStringField }

function TUnicodeStringField.GetAsString: string;
begin
  // your solution here
end;

procedure TUnicodeStringField.SetAsString(const Value: string);
begin
  inherited;
  // your solution here
end;
    DefaultFieldClasses[ftString] := TUnicodeStringField;

 

image.png

Edited by rcaspers

Share this post


Link to post

@rcaspers I had no idea you can "override" default TField types. Would have made my life a lot easier a couple of years ago!

 

@Max Terentiev You can easily do the conversion by TEncoding.Convert, GetBytes and GetString. However, the easiest solution would be to upgrade the component you are using to one, which actually supports Unicode. If they are all TDataSet descendants, you should face no issues.

If upgrading is not an option, I'd create a TFieldHelper with an .AsUnicodeString property.

Getter would read the fields value as a byte array, and return it as a String with TEnoding.Unicode.GetString(fieldbytearray);

Setter would accept a string, convert it to a byte array with TEncoding.Unicode.GetBytes and strore this instead of the string.

Share this post


Link to post

I solve problem by overriding setting/getting fields content like this:

 

TUTF8StringField = class(TStringField)
private
  FAutoUtf8Conversion:     Boolean;
protected
  function                 GetAsString: string; override;
  function                 GetAsVariant: Variant; override;
  procedure                SetAsString(const Value: string); override;
public
  constructor              Create(AOwner: TComponent); override;
published
  property                 AutoUtf8Conversion: Boolean read FAutoUtf8Conversion write FAutoUtf8Conversion;
end;
//=============================================================

 

constructor TUTF8StringField.Create(AOwner: TComponent);
begin
  FAutoUtf8Conversion:=false;
  inherited Create(AOwner);
end;
//------------------------------------------------------------------------------

function TUTF8StringField.GetAsString: string;
begin
  if FAutoUtf8Conversion then
    Result:=UTF8ToString(GetAsAnsiString)
  else
    Result:=inherited GetAsString;
end;
//------------------------------------------------------------------------------

function TUTF8StringField.GetAsVariant: Variant;
begin
  if FAutoUtf8Conversion then
    Result:=UTF8ToString(AnsiString(inherited GetAsVariant))
  else
    Result:=inherited GetAsVariant;
end;
//------------------------------------------------------------------------------

procedure TUTF8StringField.SetAsString(const Value: string);
begin
  if FAutoUtf8Conversion then
    inherited SetAsAnsiString(UTF8Encode(Value))
  else
    Inherited SetAsString(Value);
end;
//------------------------------------------------------------------------------

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×