David Schwartz

I cannot change the infrastructure, per se, but I can change how this app works for its own needs. The means of sending data between the thread and the form is of no consequence outside the app. The form does not communicate with the thread once the thread is initiated. The thread does, however, communicate with some supporting services and I can't change that part. But as far as I can tell, they're working fine. Just one question ... mostly what I need to pass is strings, 25-250 characters in length, mostly single lines under 80 chars. Using this mechanism, would I need to pass them as arrays of bytes rather than as pointers? (I think this is where my current problem lies in that the pointers to the objects or strings being sent are getting zapped.) Since these are Records, are they passed by value (ie, copied)? If so, would it make sense to break things up into, say, 127-char blocks and send them that way?

I've been trying to install MadExcept and not having much luck so far. We have severe security restrictions to deal with in our environment. In particular, our normal logins do not have Admin privs. We have a separate login "a-<username>" that DOES have Admin privs, but it's only temporary. It's kinda like sudo but more restrictive. The install is putting the data in to the Admin's registry, not the normal user's registry. So it's there if I run Delphi as Admin but not otherwise. When i try to add the BPLs to Delphi with my normal login, I get an error saying I don't have permission to read the files. Not even the Help files! We can't change ownership, and are restricted in terms of granting wider visibility to files. We're even blocked from running RegEdit. Getting a slow response from IT Dept isn't helping. Grrrr....

I know there are a couple of very long and detailed books that have been written that amount to "The Tao of Multi-Threading in Windows (and Delphi)". I'll leave it to others to master this topic since I have not really needed to do so up to this point in my career. My experience with multi-threading is with kernels that take perhaps 10 pages to explain the entire model. They're very simple, succinct, consistent, and designed to support the needs of (soft) real-time interrupt-driven programming. I cut my teeth on the "Manager vs. Monitor" debates that flared up in the 1980s as they applied to multi-processing, and I did a lot with both tightly-coupled and loosely-coupled multi-processing. Then DOS showed up and people mostly laughed. Then Windows showed up and people gagged. Need I mention some of the high-profile failures Windows NT caused during the 90's when it was used to control real-time process-control applications? (Does anybody remember the fiasco around the baggage handling system at Denver International Airport that delayed its opening by over a year? The popular press blamed the DBMS they were using, but I used the same DBMS and it worked flawlessly; the guys I talked with there said it was how Win NT was so inconsistent in how it processed things that made it keep locking up.) I think the numerous episodes like this speak quite eloquently for themselves. Several people have pointed out that the original developers of this code may have also lacked a depth of understanding based on what I've shown here. From comments in the code, I'd say the threading was added around 2004 and it does not appear to have been significantly altered since then, except for a few odd bits here and there given the comments that were left. The threading model that this app uses is implemented in several units that form the core of a couple dozen other apps, and I'll get skewered if I make any changes that affect them. My task is simply to migrate this thing from Win XP to Win 10, and there are issues showing up in Win 10 that may have been around forever but weren't manifesting in XP. All I know is that with minimal calls to the tracing & logging methods, everything works fine. When I send more and more tracing data out, I hit a point where I'm getting AVs in unpredictable areas. I can tell there are some race conditions going on. I just can't tell exactly where they're happening or why, and how to either fix them or circumvent them. I'm trying different things people have suggested, but to little avail so far. I do appreciate the support and suggestions, tho.

It seems to me that if the string is gone at the time this method is called, then copying it is only going to trigger the same AV fault, right? If it's not, then it's unnecessary.

Well, there's a problem right off the bat. procedure StatusOut(const stat: string); var AMsg:string; begin AMsg := UniqueString(stat); 'stat' is a const arg, and UniqueString is looking for a 'var' arg. procedure UniqueString(var str: UnicodeString); overload; Perhaps the 'const' argument is part of the problem here?

I just created a new VM with a fresh copy of Win10 and a new license, then loaded in some common utilities and Delphi 10.3.3 Rio via the ISO. (I realized I forgot to run the setup as Admin. Will that pose any problems down the line?) It looked like all of the various options are enabled by the installer, so I just loaded everything up. I started installing various libraries, and got to some TMS libs. They have the ability to select which of the different platforms you want to use. I seem to have bad luck with them a lot because all I get mostly is Win32 and Win64. All the others generate errors. In this case, the MacOS platforms fail because there's a folder named Import that's not found. Earlier versions of Delphi have some kind of Platform Manager that let you install different options that I guess aren't installed by default, but I can't find it in 10.3.3. I loaded up the REST multi-platform demo and it let me add every platform other than Linux64. I didn't try to build anything, but in the past if you could load it, it would build. Still the TMS installer chokes, but I don't think this is TMS' fault. What changed from 10.3.2 --> 10.3.3 vis a vis the platform installation / setup / selections ? I've had this problem with previous versions but was able to fix it by poking and prodding the Delphi installer and Platform Manager settings. What am I missing here? (Right now, I'm mainly interested in MacOS64, iOS Simulator, and Android64. All I can successfully install is Win32 and Win64.)

I'll give it a try and let you know.

can anybody guess why they'd both write to the TMemo and use a separate logger to save the same data to disk instead of just calling memo.Lines.SaveToDisk( ) periodically? In a sense, the memo is a queue, although it's not flushed and cleared. Is there a chance the logger is the source of these AVs? (I tried to set up a TStringList as a buffer instead of the memo and logger, but I couldn't get it to flush when the program terminated.)

FWIW, I'd like to think I have a better-than-average understanding of multi-threading. I spent the first 10 years of my career in the real-time machine-control world and even spent a couple of years working on a real-time multitasking kernel. The thing is, these platforms were all designed from the ground-up to do this, and they were very consistent and predictable. In contrast, I find Windows to be more of a swampy mess when it comes to multi-threading. And when you layer Delphi on top of that, it only gets worse. Thankfully, I have not had to deal with it much. So I do find this mess rather confusing to deal with. The fact that you guys keep arguing with each other about things that should be quite simple only points to the underlying complexity of how Windows offers up it's ugly, hydra-like threading models. It makes me feel like it's not just my own stupidity. But I do really appreciate the help. 🙂

Hmmmm ... totally forgot about that. Never done it, actually. I'll look into it. Thanks!

This is running inside a VM and I cannot install Delphi in it, so the debugger cannot be used. If it could, I probably would have tracked this down. (Running the code on my dev machine isn't possible at the moment for reasons I have not fully grasped.) All I'm wanting to do here is capture log data without interfering with the operation of the logic, which seems to be what's happening here. Alternative approaches are worth considering. What I'll try next: (1) install MadExcept. (2) try to set up CodeSite Lite to catch the messages rather than the current approach. We might have another logging lib available, like EurekaLog. It was used on one project I worked on a while back, and it seemed kind of clunky. I think it has a separate app to read its log files. (Or am I thinking of another one?) Any thoughts along that line are welcome.

ok, let's review... this code was written around 2003 and has been changed very little. The company says it's "rock-solid". The guy who maintained it before me was on the job for 8 years or so, and I'm told he was meticulous. The app we've been running is dated early 2017 and it runs every day at 10 AM. None of this is a problem. However, it has a bug that's not related to any of this that I'm trying to fix. It started in early March and I stumbled onto it only because I started adding a bunch of lines that call StatusOut to report what's going on inside this code. It's basically behaving like a box containing Shröedinger's bug ... the more I try to zoom in to observe the bug, the more unstable the whole thing gets. 🙂 I used to do a lot of embedded real-time dev work, but have very little experience with multi-threading in Windows in general and Delphi in particular. The main form is very light-weight and only creates the thread and runs it. When the thread is finished, then the main form is closed. It's not getting shut down early, although I did manage to kill it once that left the thread running. That was weird to see. The code in the thread calls the StatusOut( ) method to send status info that's handled as can be seen above: it's copied into a TMemo and sent to a logger. What appears in the memo and the logger are identical. It's not sending records, allocating objects, or anything that needs to be disposed on the other side of the call. It's just sending strings. When I use the other method to send the message, it shows up as trash -- a bunch of Chinese or Japanese or some other text. I don't know why. The messages sent are mostly string constants with IntToStr() calls sometimes added at the end. I discovered there are other things happening that need to be addressed. Unfortunately, there's no way I've found to run this inside of the IDE to debug it, so I have to add debug traces to the code to see what's going on in the servers where it does run. Also, there's a server with Windows XP hosting this which is what motivated this whole thing, because the company needs to move it to over Windows 10, and that's where they started to see some weird stuff. It turns out that the weird stuff they noticed is what's supposed to happen; for whatever reason, it's NOT showing up in the Win XP machine, which is strange, but does not seem to be causing any problems. I don't care, I just need it running properly in Win 10. Yes, a lot of data is assigned to local string vars before it's being handed to StatusOut, and there are lots of Format statements generating data like: StatusOut( Format( '. . .', [a, b, c] ) ); where the variables are both local vars and members of an object. Since SendMessage is synchronous, the StatusOut calls are not returning until the other side has dispatched the data that was sent, right? I thought this process was fairly robust, but maybe I need to set up CodeSite instead. Any thoughts on this? During normal operation, the existing code fires off a Windows app as a child process. It runs for quite a while. It performs OCR on one-page PDFs. It's not interesting to watch, and all anybody does is open the VNC window to see that it's running and then they close it. I don't think anybody even looks at the log. They pretty much ignore it unless the client complains, which hasn't happened in ages from what I'm told. (I don't know why, because I discovered it has been broken so it's not attaching the PDFs to proper db records since around March 1st of this year and nobody knew.) I've replaced the Windows app with an OCR library that runs as a DLL, and it does the work in two seconds per page. A process that takes 8 hours with the app takes about 16 minutes with the DLL. The core part of this thing works as-is, except for one piece that's saving data into a zip file. It seems the more StatusOut calls I make to see what's going on, the more weird exceptions I get. Again, it's behaving more like a box containing Shröedinger's bug than a nice friendly Delphi app.

Changing to PostMessage didn't help either. The messages all showed up as some kind of Asian pictographs and other junk. The class being threaded is derived from TThread, if that helps with anything.

@Larry Hengen There are only 4 controls on the main form: a checkbox, two buttons, and a TMemo. The memo gets very little traffic from these StatusOut messages, at least it did until I started adding stuff. I don't know why the Application.ProcessMessages is there other than someone thought it was a Good Idea. I'll remove it and see if your theory is correct. UPDATE: Nope, it didn't help. Now I'll try changing SendMessage to PostMessage.

As I said, I can usually build a semi-functional prototype / mockup / whatever in the same time it takes a graphic artist to build a static wireframe model. I showed a prototype to some potential investors last year trying to raise some capital and I think they thought I was lying to them about how much was completed. So I showed the next ones screenshots, and they wanted to see that I was further along, more than just drawings. These guys are presented with wireframes and mockups all the time, and Delphi lets you build something that's functional, but it looks too complete for a prototype. It's hard to explain to people. I only wish there was a "Project Overview" tab on the IDE that shows the various forms and units in the project in a graphical fashion, rather than just the textual list in the Project Mgr.

off-by-one errors are the bane of most programmers' existence. I eventually learned to look for "... 0 to ... -1" or "... 1 to ... <x>" where "x" does not contain a '-1' The use of pred() in place of '-1' is inconsistent at best, and rarely used, so it much harder to detect quickly when it's not used appropriately. Wirth threw it into the language as a pedagogical tool. That was nice, but it's impractical. The 'for ... in' approach is far less likely to lead to such errors. The use of 'Low(x) to High(x)' is even less error-prone than '0 to ...-1' IMO.

I can't tell whether you're trying to solve a programming problem, or if you're just doing mental calisthenics trying to use different programming constructs in strange ways. I realize some people love using records and pointers to them in Pascal. But I spent 10+ years using c and c++ before getting into Delphi and I have always gotten confused about which side of the variable the ^ goes on and why you need to use it with a '.' because in c/c++ the '->' always goes on the right, and you never use any two of '.', '&', '*', and '->' next to each other. Maybe it's the same way some people have trouble grasping the difference between how ++i and i++ work in a for loop. Suffice it to say I bet you've used pointers in Delphi more in the past month than I've used them in the past 10 years. That class DOES show some effective uses of inheritance tho.

One of the biggest challenges people new to OOP run into is they don't really grasp that inheritance and composition are two equally useful tools, except they solve different problems. People tend to over-do the use of inheritance at first. I guess it just seems cool. Most of the time, all you need is, say, three classes instead of one, and then you compose objects from those classes into something else. As opposed to some contorted mess created from a strange hierarchy of inherited classes that you think might do the same job. I like to think of inheritance as just another form of Dependency Injection -- one that injects things into a class via ancestry rather than constructor, property, or method injection. You can inject behavior (via methods in the base class) as well as state (via common state variables in the base class). If you only need to inject patterns of behavior (like frameworks with no default behavior) but no state, then use Interfaces. Sometimes it can be rather fuzzy which way to go. I think the most common class I spend too much time deliberating over is the lowly TStringList. Sometimes I'll need something a little off the grid and add functions to a stringgrid. Then I'll put them all into their own class (encapsulation). Then I'll scratch my head and wonder if the stringlist should be a member of the class (composition) or if I should actually inherit from it (inheritance). It's not always obvious, and sometimes it's even only worth a coin-toss. Ditto when it comes to any kind of container -- do I inherit from a TList<something> or do I compose the TList inside of the class and reflect the methods I need and skip the rest? Either way, it's necessary to create methods to reflect the things I want to do that aren't already there. A common example is adding a new widget to a list of widgets. Inheritance lets you use an inherited Add method for free, while composition requires you to implement one yourself. But sooner or later you're going to think ... why not add a AddNewWidget( ... ) function that returns a TWidget to the container class so I don't have to create a new one first then add it separately. Oh, then I can throw in some parameters that I'll usually initialize it with, sort of like a factory pattern. That AddNewWidget function must be created yourself whether you use inheritance or composition to manage the list. And there are others as well. So it's not as simple as just looking at the built-in methods. (Yes, people get obsessed wtih not having to create reflection methods like Add, Delete, and a search function when they use composition rather than inheritance.) But over time, I'd have to say I don't spend time going around looking for ways to use inheritance. That's like carrying around an umbrella on a sunny day looking for rain. Rather, I wait until I run into a situation where I've got two or three classes that CLEARLY share some common behavior and/or states, and then I'll pull the common parts into a base class that I'll then inherit from. If you're trying to figure out how to accomplish some kind of inheritance, then I'd say you're not being driven by the nature of the problem, but mostly by your desire to fit a large square peg into a small round hole. The question is not, "How do you do this?" but "Why are you even trying?" The "how" part should really should be obvious, IMHO.

Just for fun, check out the use of INI files for this.

check out BigINI from this guy, the latest (v5) is most interesting. http://www.hinzen.de/delphi/index.html No diffs, but it extends basic INI file managment quite nicely. I've used it for years. (And added several new things.)

This does not smell right to me. You're using inheritance to model composition, IMO. Your base object is a file. You don't really need to make a class around just a file. If you want to operate on two files, you don't do that by inheritance -- that's composition. You'd want a function that takes two files and compares them and returns some kind of result. You don't need a class for this either. Another function could take one input file and the diff data and regenerate the second file. In a typical Linux command shell, you'd use: diff -e file1 file2 >diff_f1_f2 You also have directionality, so you could go the other way: diff -e file2 file 1, >diff_f2_f1 These output files are called "deltas" and one is a "forward delta" while the other is a "reverse delta". The "diff -e" command gives you an output file that consists of a series of edit commands (for 'ed') that you can feed into it with the first file to get at the second file. ed -f diff_f1_f2 file1 > file2_again I'm hard-pressed to think how I'd build classes around these commands. You could have a file mirroring scheme that uses a LocalStore and a RemoteStore. Each store would keep reverse deltas, but the LocalStore would generate forward deltas to send to the RemoteStore to reflect the latest changes to a file without having to send the entire file. This makes sense -- this scheme is maintaining state by keeping the RemoteStore an accurate reflection of the LocalStore while minimizing transmission bandwidth. If all you're trying to do is visualize things, that's a little different, but you could think of the output as commands to colorize the text rather than edit it. You'd go one direction then the other and colorize each file based on changes needed to get one side from the other. But you're still not maintaining any kind of state, since the files are static and everything can be derived again at any time from the same input files. The FILES themselves represent the "current state" of their relationship that you're looking to visualize. The 3-way compare simply uses three files and does three (or six) different comparisons. A class implements behavior and maintains state. There's no state that needs to be maintained here -- just some functions that take input files and generate output files. The same output is produced for the same input, every time. One could argue that the functions implement behavior -- yes, but they don't change the state of the underlying data. So it's just a container of convenience. This seems like a rather far reach to come up with a class design when it's of no obvious benefit.

I'm using an OCR library on some text and I'm getting a lot of "noise" characters. There's some demo code that looks like this: For C:=0 To OCRResultsHeader.NumItems-1 Do Begin S:=S+Chr(OCRResultItems[C].OCRChar); If OCRResultItems[C].OCRChar=13 Then S:=S+#10; AMsg(' OCRCha'#9 +IntToStr (OCRResultItems[C].OCRChar ), False); AMsg(' Confidence'#9+FloatToStr(OCRResultItems[C].Confidence), False); End; MemoResult.Lines.Add(S); OCRChar is the character at position C in the converted results (OCRResultItems), and Confidence is a Float in the range [0..1) -- ie, it's always <1.0. I want to get a sense of what the Confidence is for individual characters, so I was thinking of taking n := Integer(Confidence * 10) to get a number in the range [0..9]. Then I'd map that to colors, like: [light-red, med-red, dark-red, light-yellow, med-yellow, dark-yellow, light-green, med-green, dark-green, white] Then I'd display each letter in a RichEdit and set its background color ("highlight") to a red/yellow/green/white color based on 'n'. Unfortunately, I can't tell this library that in certain places I'm only looking for numbers. So for example, it often spits out things that look like a '1' (vertical lines) that aren't. Same for zeros ('0'). A lot of noise characters show up as vertical-bar | , periods, and underscores. I'm thinking that if they all show up with red backgrounds, then I could filter them out by their confidence values rather than text matching against similarly-shaped characters (homographs?) Here's my code that sets various boundaries on the width and number of lines, plus it splits lines up in the stringlist. I'm not using the Confidence value here yet. S := ''; for n := 0 To OCRResultsHeader.NumItems-1 do begin ch := OCRResultItems[n].OCRChar; conf := OCRResultItems[n].Confidence; if (ch = 13) then begin if (Length(S) > 4) then // nothing we want is <= 4 chars in length scanned_rslts_sl.AddObject( S ); if (scanned_rslts_sl.Count > 9) then // it's going to be in the first 9 lines... Break; S := ''; end; if (Length(S) > 20) then // we're looking for an 8-digit number, not a novella Continue; if (ch >= 20) then // ignore control chars (20 is a space char) S := S + Chr(ch); end; One thing I was thinking was to allocate an array of bytes the same length as each line and attach it to the stringlist via AddObject. (I'm not exactly sure how to allocate an array of bytes from the heap for each line to attach as the Objects[ ] value corresponding to each character, tho. Does it need to be an actual "object"? Or can it just be a pointer to an array of bytes?) Anyway, I'm curious how to add the array of byte values for 'n' for each character to the Objects array, and also if anybody has a more obvious or interesting approach.

Aside from the tags that cannot be replaced, and the text that you want displayed, I'm not sure what you can remove. It's not like javascript where you can change all of the variable names to 2- or 3-letter codes and remove most of the white space. What do you imagine can be compressed out?

I think the problem you're going to run into is in how the RTTI support changed massively in D2010. Legacy apps built before that aren't going to care. But most libraries since then have come to depend heavily on it for a variety of things, especially when it comes to offering services of many kinds. There's stuff that simply cannot be done in D7 or even D2007 that can be done with the new RTTI support in D2010 and fwd. A lot of these have to do with design-time support as well. So you can sometimes find libs that have run-time support back that far, but not much design-time support. A straightforward REST/JSON API shouldn't be terribly difficult to write, unless you want to be able to send objects over the wire and have them reconstituted at the other end. Short of that, you might find something that works for you.

I'm trying to load up a project that uses Konopka Components from a Tokyo project into Rio, and I can't find this library in Rio's GetIt. (Rio 10.3.3) I see people complaining that it was missing in the initial release, but ... is it still MIA?