Thank you @Dalija Prasnikar, I learned some new things about window messages! Unfortunately though, none of the above applied to my case.
After the comment of @Lars Fosdal I got rid of the TTimer and used the same dummy window to catch the now manually fired SetTimer messages too. I also included a counter which increases each time a window message is received (whether processed or not) to see how swarmed application is. With a timer of 1 second, 70 seconds of runtime I got 72 messages. 1 is UM_WORKERENDED, I count the extra 1 as an acceptable discrepancy due to the timer's accuracy (or something unprocessed).
So no, it was not the window messages.
The solution is so painfully trivial I'm almost afraid to share...
In the TComponent my timer was set to 750 msec to update the elapsed time on the UI. As I'm using TStopWatch, I manually converted the .ElapsedMilliSeconds to some readable format... using dividing and substracting.
First time the timer fired at 0,75 seconds, UI was updated with the rounded 0 seconds.
Second time the timer fired at 1,5 seconds, UI was updated with the rounded 1 seconds.
Third time the timer fired at 2,25 seconds, UI was updated with the rounded 2 seconds.
Fourth time the timer fired at 3 seconds, UI was updated with the rounded 3 seconds.
Fourth time the timer fired at 3,75 seconds, UI was updated with the rounded 3 seconds.
Even if the timer component is spot on it can be seen that every 3 seconds, for 2 seconds the elapsed time does "not update"... thus the feeling of variable speed.
After upping the sampling to 1000 msec instead of 750 and some small tweaks to the conversion everything seems to be working fine.
Why it was working on the TPanel...? Because originally I forgot to change the default 1000 msec interval to 750...
What a nice way to waste a day by debugging something, which is working as it should 🙂