DB Blob being converted to Unicode and should not be

[Bill Meyer 337]

A colleague in the office is having an odd problem. We have long stored data in blob fields and now in moving from D2007 to Tokyo, we find that reading out of the blob has converted Ansi strings in the blob to Unicode. That's annoying for the string contend, but the string is simply a version identifier and the rest of the content is compressed data. What would seem obvious is to treat the content as an array of bytes, but somewhere in the chain, and before he pulls it, the conversion has occurred.

 

Opening the table in Database Workbench, the blob editor shows it is correct in the blob. Unicode should not come into play here. The Transliterate property does not appear to alter the behavior. What are we missing?

[Attila Kovacs 629]

does the blob editor showing it in raw hex?

do you read the blob as blobfield and stream or as string?

Edited February 17, 2021 by Attila Kovacs

[Bill Meyer 337]

21 minutes ago, Attila Kovacs said:

does the blob editor showing it in raw hex?

do you read the blob as blobfield and stream or as string?

Yes, blob editor shows us raw hex.

Tried TBlobField, and same problem. The conversion appears to occur before the call to CreateBlobStream. Using the answer given here: https://stackoverflow.com/questions/44365344/how-to-read-data-from-a-tblobfield-using-ado-in-delphi

In debug with this:

function ReadBlobField(DataSet: TDataSet; Field: TField): TBytes;
var
  Stream: TStream;
begin
  Stream := DataSet.CreateBlobStream(Field, bmRead);
  try
    SetLength(Result, Stream.Size);
    if Stream.Size>0 then
      Stream.ReadBuffer(Result[0], Stream.Size);
  finally
    Stream.Free;
  end;
end;

Right after Stream.ReadBuffer, Result shows characters with alternating zeroes.

Edited February 17, 2021 by Bill Meyer

[Attila Kovacs 629]

What kind of DB and Field type and DAC?

Can you read the data fine with the D2007 app?

[Bill Meyer 337]

Just now, Attila Kovacs said:

What kind of DB and Field type and DAC?

Can you read the data fine with the D2007 app?

Firebird, Blob, and TDataset.

The concern now is to read from data written out of D2007, where it has worked fine for years. 

[Attila Kovacs 629]

I'm not an IB expert but IB BLOB has a default subtype = text.

Maybe it was the whole time stored with that subtype but it was fine with ansistrings?

[Bill Meyer 337]

Just now, Attila Kovacs said:

I'm not an IB expert but IB BLOB has a default subtype = text.

Maybe it was the whole time stored with that subtype but it was fine with ansistrings?

We were just discussing that. I am not an IB or FB expert either, but we now suspect that the subtype may be the root cause. It should be 0 for binary, but is now set to 1. If that is the issue, we're looking at a lot of scripts to update DBs.

[Attila Kovacs 629]

depending on the DAC you are using you could try "Data Type Mapping"

[Bill Meyer 337]

4 minutes ago, Attila Kovacs said:

depending on the DAC you are using you could try "Data Type Mapping"

Will look at that next. Thanks.