aehimself

24 minutes ago, Remy Lebeau said:

You can't send window messages to a non-windowed class.  Well, unless you are calling TObject.Dispatch() directly, that is.  Are you?

Why are you not posting (not sending) the message to an actual window?

Well.. technically I am. I have one allocated with AllocateHwnd. I guess in this case all messages are sent there, even if it's filtered.

30 minutes ago, Remy Lebeau said:

144 is in the range of system-reserved message IDs, however there is no known system message with an ID of 144.  Message ID 2 is WM_DESTROY, though.

Yes, I managed to find 2, but there was nothing on 144 though which I could find.

Hello,

I have a TObject descendant which has a message handler defined like:

 TProgressContent = Class
 strict private
  Procedure ProcessWindowMessage(Var Msg: TMessage); Message UM_WORKERENDED;
 End;

This message gets sent when a worker thread finished so I extract some data and free it up. I started to get some strange results and some application freezes, and it quickly turned out that lots of other messages arrive in this handler. I have some 144's and 2's entering up here.

The solution is easy - I just made an if Msg.Message = UM_WORKERENDED inside the handler and now all the issues are solved but isn't explicitly defining the message should already filter the rest out?

Also, out of curiosity, what is a window message with the ID of 144? Couldn't find anything on Google so far.

Cheers!

On 2/13/2022 at 12:21 PM, PeterBelow said:

I had a similar problem with Delphi Alexandria, McAfee would detect a freshly build Win32 debug EXE as virus

Issue is not Delphi Alexandria but that piece of ga... wonderful McAffee.

I remember removing viruses by hand from multiple servers because the freshly updated McAfee failed to recognize a ~5 yo worm. We knew it worked because it always quarantined our patching tool.

At work ESET, at home M$ Defender (AntiMalware, Security... idk how it's called nowadays) works just fine. The only reason we excluded Delphi executables is to speed up the building process on dev PCs.

It seems I found and fixed the problem. For a quick recap: Using UTF16 character encoding I often experienced ORA-01406 fetched column value was truncated errors and when this went away by manually increasing the buffer size text data got malformed.

As it turns out these two were independent errors and for the first my workaround is actually the solution.

Text corruption was a bit trickier. There is a setting which controls how LOB data is cached and internally this chooses what kind of resultset is being created. The difference between the two of them was how the columns were defined: the one working properly checked if the codepage is set to Unicode and if yes it changed String types to UnicodeString.

Moving the same block to the other resutset fixed the corruption.

15 hours ago, Uwe Raabe said:

The OS thread scheduler will take care of running it on whatever core is available.

I might be wrong here now as I believe this was on D7, but I think I met the same issue as @DavidJr. once.

I lost my key to my own password database so I put together a simple brute-force program to try and open it with all possible combinations, using as many TThread descendants as physical CPU cores (which was 4 in my laptop back then). During the first test run it was immediately clear that something is off, as the total amount of CPU used was 25%; I couldn't max out the CPU.

I seem to remember having some tries with BeginThread but I ended up running separate, single-threaded processes to utilize the rig's full protentional I managed to recover my passwords on (we had an unused 32 core Dell server which I could "borrow" for this task).

P.s.: It seems yes, it was solved one day. Using the following test I can increase the total CPU usage count all the way up just by adding more instances to the list:

program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, System.Classes, System.Generics.Collections;

Type
 TMyThread = Class(TThread)
 protected
  Procedure Execute; Override;
 End;

Var
 threads: TObjectList<TMyThread>;
 a: Integer;

Procedure TMyThread.Execute;
Var
 a, b: Integer;
Begin
 b := 13;
 For a := Integer.MinValue To Integer.MaxValue Do
  b := b XOr a;
 a := b;
End;

begin
 threads := TObjectList<TMyThread>.Create(True);
 Try
  threads.Add(TMyThread.Create(False));
  {...}

  For a := 0 To threads.Count - 1 Do
   threads[a].WaitFor;
 Finally
  FreeAndNil(threads);
 End;
end.

My laptop has only 1 CPU though, it's possible that the problem still exists, just on multiple CPUs?

@Attila Kovacs you are right. I extracted the data behind RawByteString, which was 195,129,108,108,97,110,100,195,179,32,108,97,107,99,195,173,109. This is indeed UTF8, and it just gave me an idea on where to look...

@Dany Marmur The setting I'm talking about controls whether downloaded LOB data is cached or not. Inside the component this causes a different type of resultset created. On 3 PCs this setting was off (default), on one I turned it on. This is why one machine acted differently, as it was using a different type of resultset (the one which is faulty).

1 hour ago, Fr0sT.Brutal said:

and the issue is...?

...that the journey didn't end just yet 🙂

What I found is that the two things I thought are connected aren't. The original "column value is truncated" Oracle issue is a simple case of not enough buffer allocated and my workaround (forcing to use a larger buffer) is the solution.

However, this has nothing to do with the corruption issue; which I just found out is NOT memory corruption but wrongly transformed character array:

Here, the method's return value is PAnsiChar, fRawTemp is a RawByteString:

RawByteString is correct, but in the moment it's being casted to a PAnsiChar it is immediately messed up:

This then is stored in the resultset, so the dataset and the components display this mess.

...so a little update, the issue is in the component 🙂

The behavior difference between the 4 PCs are caused by a setting in my application which is changing a property on the connection, which is causing the corruption. This setting by default is off and wasn't changed on 3 PCs.

Took me a while to figure this out, as I had to import half of my main program bit-by-bit to the simple test app for the issue to finally occur.

At least it also made it clear what I thought is one issue is actually two; and fortunately both have a workaround.

4 minutes ago, Der schöne Günther said:

How do you actually detect that something has now gone wrong? Some component (like a DBGrid) displays a TDataSet that contains gobbledegook? Or is there more to detect it?

Messed up text in components. This is why I explicitly mentioned that it's not a display issue, data in the dataset is garbage too. I guess this never worked on my side, for a long time I simply got a "Fetched value truncated" error from Oracle until I found out changing a property gets rid of that (see the prev, thread mentioned in the linked one). Data is still corrupted, though.

7 minutes ago, Der schöne Günther said:

How do you know it's memory corruption (like writing out of bounds), and not generally using a stale pointer? Have you tried compiling the application with a memory manager like full FastMM and enabled runtime checks?

I don't know, but based on the initial Oracle error I guess it is going to be a memory corruption. I already checked MadExcept's buffer under / overrun protection, did not fire off.

6 minutes ago, Attila Kovacs said:

Why is it relevant what's in the dataset after an error in the client lib?

After increasing the initial_buffer_size, Oracle doesn't throw an error anymore.

7 minutes ago, Attila Kovacs said:

Could you do some selects where the actual data is the half of the maximum data etc.. to see the threshold for the error?

Yes, I did that, but it's not consistent. I have a feeling that it is going to be a sweetspot, when a specific amount of character data was returned the error appeared. Simply removing 3 random text columns already returned all data, formatted correctly afaik.

9 minutes ago, Attila Kovacs said:

Btw. are the actual RDBMS's also the same?  (You and your collegue)

Failed to mention this, thanks! Yes, we are connecting to the exact same DB.

12 minutes ago, Attila Kovacs said:

One side, you were mentioning VARCHAR somewhere and the other side you are setting the clientcodepage to UTF16. I don't know oracle but this is interesting.

Maybe zeos calculates for varchar and server delivers in nvarchar?

This is something I can not tell. I don't have access to the DB create scripts, PL/SQL says all the textual fields are VARCHARs only.

Unfortunately my knowledge in private life of RDBMSs is very limited, I consider myself a user only, so I don't know what is (should be) the difference between normal varchar and nvarchar.

4 hours ago, Joseph MItzen said:

No more #13#10 in the era of Unicode...

That's a strange way to write sLineBreak 🙂

I experimented with Delphi (I think 10.2) and Wine but never got it to work completely. Everything more complex than a single form caused AVs and other funky things.

Maybe things already changed at Wine, but my latest experience is that it won't fully work, ever.

All,

I have a really strange situation here and I'm running out of ideas on what differences to check. Long story short: I have an application to access data in a database (Connection -> DataSet -> DataSource -> DBGrid, using Zeos and VCL DBGrid descendant for some custom features). In one specific codepage setting it seemingly starts to corrupt memory on ONE PC only (that I know of, that is). Details can be read here.

Now, given the complexity the issue might be Oracle-related, PC environment-related, component-related, moon phase related or simply I messed something up in my code. My issue is that I'm running out of ideas to pinpoint the culprit.

I'm not asking how to fix the issue (I know it's not possible without seeing the code), I'm asking for diagnostic steps; basically how would you start to debug this?

What I know:

- OS, Oracle client locale doesn't seem to affect the behavior

- It's not thread-related, confirmed by moving all logic to VCL

- Oracle client and server version seems to be irrelevant, the issue appeared with 12 and 19, client and server respectively

- It is not a display issue, datasets actually contain garbage

What I think:

- I doubt the issue is in the component as the same binary acts differently on one PC only

- Because of the same reason I would say that my code is good, too

What I will do next:

- Simple test app. Only a connection, query and a DBGrid

- On a PC where it seems to work I'll use said codepage to see if just by some miracle corruption only happens at a later stage

Thank you all for your input!

2 hours ago, Daniel said:

Thanks for the support - I am a step further.

Actually OpenSSL handles all these algorithm-stuff by itself. It was just my code that was writing a superfluous zero-byte to a stream. And when it comes to cryptography, a single byte can destroy everything. 😉

Oooooh, how many of sleepless nights I caused myself when due to encoding issues I changed some string routines to TBytes...

I can feel the satisfaction of your "oh, damn it" moment when you realized what went wrong 🙂

No need for a launcher, a single .EXE can do it - at least on Windows.

Windows allows to rename the executable even if your application is running. So the steps are...

- Download the updated .exe, and save it e.g. .exe.new

- Rename the current application (Application.ExeName) to .exe.old

- Rename the updated file .exe.new -> .exe

- Launch the .exe and immediately quit

- It's a good practice to wait for the previous process to actually die before doing any actual work, like reading settings up, etc.

- Upon startup check and delete any .exe.old files

I'm using this method and it works like a charm.

The value is in pixels, not percent.

Use Form.ClientWidth Div 2 instead.

Based on what I know about local variable initialization, luck was the only logical explanation. But considering the amount of chances it could have broken (15+ years, ~100 clients, software is running on 10 machines at each, minimum, in daily use) I started to have doubts.

If this is pure luck, I'll advise our CEO to start to play the lottery 🙂

Hello,

Not that long ago we managed to fix up the codebase enough so our large legacy application could be compiled to Win64. Since then this is the second time when something causes an AV on Win64 which worked for 15+ years on Win32.

Pseudocode:

Procedure TForm1.Button1Click(Sender: TObject);

 Procedure DoStuff(Var sl: TStringList);
 Begin
  If Not Assigned(sl) Then
    sl := TStringList.Create;
  sl.Text := 'Just add' + sLineBreak + 'some data';
 End;

Var
 sl: TStringList;
Begin
 Try
  If condition Then
    Exit;

  DoStuff(sl);

  [...]

 Finally
  sl.Free;
 End;
End;

There are two points where said code can fail:

- sl wasn't initialized to nil before passing it to DoStuff. Win32 happily created the TStringList, Win64 didn't and therefore the next method doing anything with it threw an AV

- Before even having a chance to initialize sl, we might break out of the method, but sl.Free is called anyway. On Win32 this was fine, Win64 threw an AV first chance

As far as I'm aware Delphi is not initializing local variables and I also don't think we were that lucky for 15+ years that these variables were always reserved on a previously cleared memory area (therefore, pointing to nil).

Before you say anything, I know that this is a coding issue and was corrected quickly, I'm interested why it behaves differently; maybe some compiler differences...?

What can be the explanation?

Cheers!

1 hour ago, Der schöne Günther said:

• You cannot fine-grain access (public read access, protected write access)

However I disagree with the opinion that properties are useless, but the above is driving me nuts, too. To be able to achieve this, I have a public, readonly ex. Status property, and a protected writeable InternalStatus property, which point to the same variable. It works, but I hate even to look at it.

Maybe this could be solved if internally we refer to Self as an interface... idk.

Security patches WOULD be important, if there's no firewall whatsoever between the machine and the Internet (WinXP 30 sec countdown, anyone?) OR if you are visiting shady websites OR if any machine on the network is already infected.

I have an ESXi at home with VMs of all Windows Server versions from Win2k up so I can test the product compatibility. For none of these I have blocked Internet access and with only a router inbetween I had no issues (that I know of, anyway :)). in the past years.

I'd say it's safe to have EOL OSes connected to the Internet if you know exactly what you are doing.

As far as I am aware, SQLite was designed to be a small, "engineless", flatfile data storage which can be queried just like any RDBMS. It never meant to have all the functionality of those, it simply provided an alternate, standardized way to store, and quickly access information.

Therefore, limitations. Quite few, and easily workaround-able, imo, though.

13 minutes ago, Fr0sT.Brutal said:

And don't forget to make it 64-bit otherwise rollover will bite very hard

I started to use databases after the 32bit UNIX timestamp panic was already all over the Internet. Therefore I quickly learned to define all UNIX timestamps as UInt64s 🙂

Nonetheless very true.

From Wiki if someone is interested:

Quote

At 03:14:08 UTC on Tuesday, 19 January 2038, 32-bit versions of the Unix timestamp will cease to work, as it will overflow the largest value that can be held in a signed 32-bit number (7FFFFFFF16 or 2147483647). Before this moment, software using 32-bit time stamps will need to adopt a new convention for time stamps, and file formats using 32-bit time stamps will need to be changed to support larger time stamps or a different epoch. If unchanged, the next second will be incorrectly interpreted as 20:45:52 Friday 13 December 1901 UTC. This is referred to as the Year 2038 problem.

Quote

At 06:28:15 UTC on Sunday, 7 February 2106, Unix time will reach FFFFFFFF16 or 4294967295 seconds, which, for systems that hold the time on 32-bit unsigned integers, is the maximum attainable. For some of these systems, the next second will be incorrectly interpreted as 00:00:00 Thursday 1 January 1970 UTC. Other systems may experience an overflow error with unpredictable outcomes.

Maybe you think of FTP, but HTTP does not work this way. You need a receiver script / application on your web server which will receive and save that file.

4 hours ago, Stano said:

When I found SQLite and found how complicated it was to work with dates, I quickly rejected it.

Use UNIX timestamp and convert in the application. SQLite will be more than happy to store that number for you.

Lately I'm even storing dates as UNIX timestamps in local settings files too.

It's really convenient.

2 hours ago, DavidJr. said:

 

  repeat
    if MsgWaitForMultipleObjects(0, Pointer(nil)^, FALSE, (TickTime - Present), QS_ALLINPUT) <> WAIT_OBJECT_0 then Break;
    Application.ProcessMessages;
    Present := (GetTickCount - Past);
  until ( Present >= LongInt(TickTime) );

 

I wanted to save time before to convert blocking methods to background threads but still blocking the calling method while it runs... used a cycle like this. Once issues started to appear I spent months getting rid of this "shortcut". It became way more messy and way more complicated.

Even if you don't have issues at the moment... remember, this cycle is the devil reincarnated.

3 minutes ago, Ian Branch said:

Bad news then!  Nothings been done about it since 10.3!  😞

My comment was meant to be ironic. Sounded more ironic before I pressed send anyway.

I'd rephrase "nothing" to "nothing in this matter". There has been a significant improvement in speed and accuracy of error insight, suggestions - at least in small to midsize projects. The IDE got more stable (once you disable LiveBindings and the welcome screen) and some convenience features were introduced (e.g. Ctrl+click on inherited). Hell, my IDE did not crash by itself since D10.1, not even at work with an endless amount of (even custom) components installed.

Delphi is getting better in lots of ways. Alas, this still doesn't mean that the speed / willingness of Emba fixing bugs is not tragicomic.