Jump to content

Pawel Piotrowski

  • Content Count

  • Joined

  • Last visited

  • Days Won


Pawel Piotrowski last won the day on November 24 2019

Pawel Piotrowski had the most liked content!

Community Reputation

15 Good

Technical Information

  • Delphi-Version
    Delphi 10.3 Rio

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Pawel Piotrowski

    Example of wasteful, innefficient string manipulation

    Actually, your code is wrong. Now you are doubling the memory usage. SetLength creates a big string, then in the loop, you add more to it... If you wish to pre-allocate the length of the string, then you need to do it this way: Function MultiplyStrWithPreallocate(aStr: String; aMultiplier: Integer): String; Var i, pos: Integer; Begin SetLength(Result, Length(aStr) * aMultiplier); pos := 1; For i := 1 To aMultiplier Do Begin move(aStr[1], Result[pos], Length(aStr) * sizeOf(Char)); Inc(pos, Length(aStr)); End; End; Here is what is going on behind the scene, when you call: Result := Result + aStr; the memory manager creates a new temporary string, assigns the content of "result" and "aStr" to it. That new temporary Strings is then assigned to "result" which in turn decreases the reference counter to the String that result is refering... in that case to 0, which causes the memory manager to release that String. So for a short period of time, both, the original "result" and the "result"+"aStr" are in memory. And for each for-loop-iteration, you basically have a copy operation of the whole string. My above optimization using move reduces both of those concerns. There is just one memory allocation, and only a few bytes are copied in each for-loop-iteration
  2. if you are more comfortable using strings, then just use rawByteString. TBytes are fine, too. There will be no real performance drawback using them. RawByteString have a possible drawback, depending how you will use them. There are some functions, that expect a string, but will happyly accept a rawByteString, doing a implicit conversion. And that conversion from rawByteString to String and then back to rawByteString can cost some performance... You will not have such a problem with TBytes. SO if you go with rawByteString, keep a close look on your compiler warnings 🙂
  3. Pawel Piotrowski

    Dynamic arrays and copying

    The copy on write rule is only for strings. Arrays are reference counted and that's it. If you need to copy the content of the array, you need to use the copy() function. It will create a new instance of your array.
  4. Maybe have a look at the DEB an Event Bus framework for Delphi from Daniele Spinetti. I think you will like it. It should help you achieve your decoupling without too much added complexity. here is the blog entry about this: http://www.danielespinetti.it/2016/02/deb-event-bus-framework-for-delphi.html and here the git repo: https://github.com/spinettaro/delphi-event-bus
  5. Pawel Piotrowski

    multi-media question

    I suppose I would write a power point automation. You can control Word and Excel with delphi I suppose you could control power point as well. From there it is simple and fast. Insert your text/music/images into your ppt a template then let power point export it as a video file.
  6. Good or bad memory doesn't matter. You need to be able to navigate and understand the code even if you didn't wrote it yourself. I would guess, your biggest problem lays in the hard to find coupling of controls with the actual code. In other words, bad code navigation capabilities. Code navigation is for me personally more important then separation. Well, I know, controversial. And don't get me wrong, I'm all for separation, but try to find a way, that allows you to navigate your code nicely and have still your separation. When you have a dozen of projects, each with hundreds of units... well, you will be thankful to be able to navigate the code with ease. Do not try to follow the hype just because. There should be a reason to do it. Over complicating things doesn't solve anything. You wrote, that your mainform references the frame, maybe it shouldn't? Maybe it should reference the presentation layer instead. Maybe you should couple the frame with that presentation layer there, or at least have a call to the coupling there? I know the unpleasant WTF moment, when you look at a form, and have no idea where the code is, that supposed to be there... then you go on a search, wasting a lot of time...just because someone decided to implement a fully decoupled system just for the sake of it. I would suggest two solutions: 1. Hide the frame from yourself. the mainform (or better said, your entry point) should reference the presentation layer, where you can find the coupling of the controls and the frame that is used. or.. 2. let the frame couple itself with the presentation layer. There is something beautiful to be able to click in the form designer on a button and see where the call goes. You need to do the coupling somewhere anyway. There is little reason to remove *all* the code from the frame. And if you do that, ask yourself what is the real benefit for you. Are you solving a problem or are you introducing one?
  7. I agree. But others before me pointed that already out 🙂 I was trying to explain (not justify) the differences between the two samples given, the one with 17% more memory and the second with 68% more. You can explain the +17% with the missing SetLength, but not the +68% more memory. More knowledge doesn't hurt, and if someone uses the task manager to compare memory usage, I suppose it is nice to know about the memory manager itself, isn't it?
  8. Like others said, TList has a capacity growing strategy. The second, and more important thing is, that the memory manager doesn't release the memory back to windows. So what you might getting is this: when you add items to the list, and it tries to resize, it might happen, that there is not sufficient memory in the current place to just in place resize. so the memory manager copies the whole array to a new location, that can hold the size of the new array. but the old memory space is still kept reserved by the memory manager. So when you use the task manager, windows sees both. Andd BTW, the memory manager doesn't request the exact size either. Like TList, it has a growth strategy, too.
  9. Pawel Piotrowski

    Changing label text in thread leads to weird display

    Thanks 🙂 The answer to your question is yes and no 😉 No: not with the default Delphi settings and not in a regular record or class, no a word can not cross 32bit boundaries. The compiler ensures that for you by default. Yes: You can force it to cross the 32bit boundaries. Delphi gives you this option. Like with packed records. Or by changing the compiler settings, or by using the compiler directive... and maybe something else 😉 But that is usually a conscious choice.
  10. Pawel Piotrowski

    Changing label text in thread leads to weird display

    I might unintentionally confused you... sorry for that. The header in my test saying "aligned" should rather say "starts with memory divisible by 4". But that was too long for the header. I tried - and failed it seems 😉 - to clarified that at the bottom of that same post. even if a variable doesn't start right away aligned (so its memory allocation is not divisible by 4), but doesn't cross the 32bit boundary, it is guaranteed to be atomic. Basically, a word is atomic, no matter if the 2 bytes that have to be discarded are at the start of the memory or at the end of it.
  11. Pawel Piotrowski

    Changing label text in thread leads to weird display

    The situation is not that grim and hopeless 🙂 dummzeuch is correct. Delphi already performs aligning. Not only for global variables, but also for fields inside a record or a class. See my post above for measurements of the sample record. You can see, that all of those fields are aligned, Delphi already injects dummy bytes to ensure this. This is the default setting. So you only need to be aware of special cases, like packed records and such.
  12. Pawel Piotrowski

    Changing label text in thread leads to weird display

    I know 🙂 But the number of threads doesn't matter to the correctness or falsehood of the assumption that strings are thread safe 😉 The high number of threads just helps to increase the probability of bad things to happen 🙂 And we want bad things to happen while we write and test the code, and not after we ship the software. When working with threads, one of the many problems is... it can work just fine... even, if there are problems in the code. At least for a while, on the development machine. The problems start rolling in, when you ship the software to some hundred customers. Then you will start to get strange bug reports from them. That is a very unlucky position to be in.
  13. Pawel Piotrowski

    Changing label text in thread leads to weird display

    I'm not sure if y I'm not sure if your testcase not crashing is a proof that strings are now thread safe. remember how strings are designed: http://www.marcocantu.com/epascal/English/ch07str.htm so the string is just a pointer to the string content and before that you will find the length and the reference. In order to increase the reference, the CPU needs to get the address of the content. Then decrease it, and only then has it access to the reference. And only then can it safely increase the reference. That are multiple read/write operations right there. This is not atomic. Increase the number of threads that perform the writing and reading. Mayby then it will crash. I've prepared a small test app myself. It crashes almost instantanously. Try it out. And even if it doesn't crash, it shows errors in the string length. But if you enable the critical section, all is fine again. Unit Unit1; { .$DEFINE UseCS } Interface Uses Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics, {$IFDEF UseCS } syncObjs, {$ENDIF} Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, Vcl.ExtCtrls; Type TForm2 = Class(TForm) Timer1: TTimer; StaticText1: TStaticText; Procedure FormCreate(Sender: TObject); Procedure Timer1Timer(Sender: TObject); Procedure FormDestroy(Sender: TObject); Private fErrorCounter: integer; fTerminated: boolean; {$IFDEF UseCS} fCS: TCriticalSection; {$ENDIF} Procedure asyncWrite; Procedure asyncRead; Public End; Var Form2: TForm2; GlobalString: String; TempString: String; Implementation {$R *.dfm} Procedure TForm2.asyncRead; Var len: integer; x: integer; s: String; Begin Repeat {$IFDEF UseCS} fCS.Enter; Try {$ENDIF} s := GlobalString; len := length(s); {$IFDEF UseCS} Finally fCS.Leave; End; {$ENDIF} For x := 0 To 99999 Do Begin {$IFDEF UseCS} fCS.Enter; Try {$ENDIF} If len <> length(s) Then Begin inc(fErrorCounter); break; End; {$IFDEF UseCS} Finally fCS.Leave; End; {$ENDIF} End; Until fTerminated; End; Procedure TForm2.asyncWrite; Var x: integer; Begin Repeat {$IFDEF UseCS} fCS.Enter; Try {$ENDIF} If random(2) = 0 Then GlobalString := '' Else GlobalString := StringOfChar('a', random(124)); {$IFDEF UseCS} Finally fCS.Leave; End; {$ENDIF} Until fTerminated; End; Procedure TForm2.FormCreate(Sender: TObject); Var x: integer; Begin randomize; {$IFDEF UseCS} fCS := TCriticalSection.create; {$ENDIF} For x := 0 To 9 Do Begin TThread.CreateAnonymousThread(asyncWrite).Start; TThread.CreateAnonymousThread(asyncRead).Start; End; End; Procedure TForm2.FormDestroy(Sender: TObject); Begin fTerminated := true; End; Procedure TForm2.Timer1Timer(Sender: TObject); Begin StaticText1.Caption := IntToStr(self.fErrorCounter); End; End.
  14. Pawel Piotrowski

    Changing label text in thread leads to weird display

    not at all. Delphi Strings are NOT thread Safe. You need to use a CriticalSection or similar. see here: https://stackoverflow.com/questions/30412147/it-is-safe-to-change-variable-values-of-a-thread-from-the-main-thread http://blog.synopse.info/post/2011/08/28/Multi-threading-and-Delphi
  15. Pawel Piotrowski

    Changing label text in thread leads to weird display

    For a Delphi string, it's never threadsafe if you have one thread writing and one thread reading. What is safe is the reference counting of strings. Copy-On-Write of Delphi strings is not a threadsafe operation. if you need a multithreaded read/write access to the same string you generally should use some synchronization, otherwise you are potentially in trouble. Example of what could happen without any lock. String is being written: it should become bigger than it was, so new memory is allocated. But pointer is not yet modified, it points to old string. At the same time reading thread got a pointer and began to read old string. Context switched again to writing thread. It changed pointer, so now it is valid. Old string got refcount 0 and was immediately freed. Context switch again: reading thread continues to process old string, but now it is access to deallocated memory which may easily result in access violation. dummzeuch is correct. Usually, you do not need to worry, Delphi plays nice and takes care for you. But if you want to know why non aligned memory access is not atomic, here is why: The problem is not limited to CPU instructions. In fact it has more to do with the data bus. When you have a 32 bit wide data bus, a read from memory is aligned on that boundary. So if you were to perform a 32 bit read from address 0x02, then two memory cycles are required, a read from address 0x00 to get two of the bytes and a read from 0x04 to get the other two bytes. You see, the first read fetches all 4 bytes from the address 0x00, discards the first 2 bytes. The second read fetches all 4 bytes from 0x04, and similar discards the second two bytes. After that the remaining bytes are combined to give you your 32bit data that you requested. This is not guaranteed to be atomic. A different CPU core could get its chance in between the above two reads to change the memory at address at 0x04, just before you read it. This is why you can not assume atomicity with non aligned variables. You might get lucky, because the CPU has multiple caches, and it might happen to be safe. But it is not guaranteed to be atomic. On a similar note, aligned memory is twice as fast to read/write, and this is why Delphi (and other compilers) align instructions and data in memory. I've build a small test, to see, how the following record will be aligned: Just for fun. TmyNiceRecord = Record i1: Integer; i64: int64; b1, b2, b3: byte; w1, w2: word; b4: byte; i2: Integer; b5: byte; b6: byte; b7: byte; End; who can guess which fields are aligned properly? Which are guaranteed to be atomic? Here are the results: for win32 build, with delphi 10.3.2 Address*| Aligned | VarName | VarSize 0 | Yes | i1 | 4 8 | Yes | i64 | 8 16 | Yes | b1 | 1 17 | No | b2 | 1 18 | No | b3 | 1 20 | Yes | w1 | 2 22 | No | w2 | 2 24 | Yes | b4 | 1 28 | Yes | i2 | 4 32 | Yes | b5 | 1 33 | No | b6 | 1 34 | No | b7 | 1 (* as an offset to the record itself) For win64 Address | Aligned | VarName | VarSize 0 | Yes | i1 | 4 8 | Yes | i64 | 8 16 | Yes | b1 | 1 17 | No | b2 | 1 18 | No | b3 | 1 20 | Yes | w1 | 2 22 | No | w2 | 2 24 | Yes | b4 | 1 28 | Yes | i2 | 4 32 | Yes | b5 | 1 33 | No | b6 | 1 34 | No | b7 | 1 interesting, isn't it? So, is delphi playing nice? Which fields can be assumed to be atomic? The answer is: All of them. Even b2 and b3. Yes, they do not start with a aligned address, but they do not cross the boundary either. This means, the read/write is still atomic. I hope that helps to better understand the topic. MemoryAlignedOrNot.zip