aehimself

There are two connections between the client and the server at this state, both show up as ESTABLISHED. Damn, I really wanted to pass this on to the NW guys 🙂

Checking the TCP state with NetStat is a great idea, I don't know why I didn't think about this! Will do the check, thanks!

So I made a basic sketch of my updating mechanism, freely available for anyone to check: https://github.com/aehimself/AEFramework The only things you'll need are: - AE.Updater.Updater - AE.Misc.FileUtils - AE.Updater.UpdateFile - AE.Application.Settings - AE.Misc.ByteUtils At the moment it is using the System.Zip2 unit but can be reverted easily to Delphi's built in one by changing it to System.Zip in AE.Updater.Updater. It was built and tested on Delphi 11, relies heavily on generics and other built-in components. File versioning is strictly Windows-only... for the time being I found no better way to determine file data than in AE.Misc.FileUtils. That could use a refactor, but as it works for the time being I didn't bother. To test, have any type of web server ready and decide where you want to put your repository. Let's say our repository will be https://dev.lan/updates, locally available at D:\WWW_root\devlan\updates. I'll make the references accordingly. - Create a TAEUpdateFile instance and add a test product: updatefile := TAEUpdateFile.Create; Try var fname = ParamStr(0); updatefile.Product[FileProduct(fname)].URL := 'myproduct'; var pfile := updatefile.Product[FileProduct(fname)].ProjectFile[ExtractFileName(fname)]; pfile.LocalFileName := fname; var ver = FileVersion(fname).VersionNumber; var fver = pfile.Version[ver]; fver.ArchiveFileName := ChangeFileExt(ExtractFileName(fname), Format('_%s.zip', [FileVersionToString(ver)])); fver.Changelog := 'Improved some stuff' + sLineBreak + 'Broke lots of things I don''t yet know about'; fver.DeploymentDate := 1; // Use your favorite UNIX timestamping method, just don't leave it on 0. 0 means undeployed and will not be considered when checking for updates var ms := TMemoryStream.Create; Try updatefile.SaveToStream(ms); ms.SaveToFile('D:\WWW_root\devlan\updates\update.dat'); Finally ms.Free; End; Finally updatefile.Free; End; Deploying the actual update file is manual for the time being, just zip your .exe, rename it to "Project1_1.0.0.0.zip" (or whatever the original .EXE name and version number is) and copy it to D:\WWW_root\devlan\updates\myproduct. Basically right next to the update file there will be a bunch of folders (one for each product) and inside this folder there will be tons of .zip files, one for each version of each file. Later on this can be used to downgrade as long as the .zip is still available. Updating is a lot easier: Var upd: TAEUpdater; s, fname: String; ver: UInt64; Begin upd := TAEUpdater.Create(nil); Try upd.UpdateFileURL := 'https://dev.lan/updates/updates.dat'; upd.UpdateFileEtag := _etag; // string var on form to minimize web traffic upd.CheckForUpdates; _etag := upd.UpdateFileEtag; s := ''; For fname In upd.UpdateableFiles Do Begin s := s + fname + sLineBreak; For ver In upd.UpdateableFileVersions[fname] Do s := s + FileVersionToString(ver) + sLineBreak + upd.FileVersionChangelog[fname, ver] + sLineBreak + sLineBreak; upd.Update(fname); End; If Not s.IsEmpty Then ShowMessage(s); Finally FreeAndNil(upd); End; At the start of your application call TAEUpdater.Cleanup to remove the old version of files - if any. Todo: Error checking and handling... empty product url will probably result in 404 (unless https://dev.lan/updates//file.zip is a valid URL - didn't check). Files in subfolders aren't yet supported, all will be placed right next to the executable. Files without version information are not yet supported. Hash checking to be implemented, messages to be added, plus a basic demo app to manipulate the update file... in the long run I might replace generics and allow a custom way to download the files so instead of TNetHTTPClient ICS or Indy can be used according to the users taste. Yeah, this is only a skeleton for the time being but it seems to work. Any suggestion is greatly appreciated!

Wow, I didn't know about this! Without E-Tag caching: With E-Tag caching: Code is as easy as follows if someone is wondering: Var hr: IHTTPResponse; head: TNameValuePair; headers: TArray<TNameValuePair>; begin SetLength(headers, 1); headers[0].Name := 'If-None-Match'; headers[0].Value := ETAG; hr := NetHttpClient1.Get('https://localintra.net/test.zip', nil, headers); For head In hr.Headers Do Memo1.Lines.Add(head.Name + ' -> ' + head.Value); Memo1.Lines.Add(''); Memo1.Lines.Add(hr.StatusCode.ToString + ' ' + hr.StatusText + ', ' + hr.ContentLength.ToString + ' bytes received'); As a quick and dirty optimization I can even add it to my current update mechanism... Nice, thank you!

This idea popped up in my mind too - first to query only the project, server would reply with only the latest version of each file. If a new file or version is detected, the client could make a second request, sending the current version of each file in the reply. This way the second answer would only contain the changelog since said version - thus further reducing data to transfer and process. This could be one in one step too, actually, if the project query includes the known local file versions. I don't know why I want to minimize the amount of queries though as during the actual update - according to my new plan - one request will be performed for each file as they will be stored separately.

For the time being they are negligible. The update JSON which holds all version information is 60k, zipped to 20k; but it contains only 200-ish versions from two projects up until now. The archives containing the latest version of projects are ranging between 5 and 20MB, but these are only downloaded if a newer version was found in the JSON.

Data. Amount of requests can easily be controlled from the client (check for updates every application start / x days). I do, but it's so high it's not a concern now / for a couple of years for sure. The reason I want to do it like this is because it is the way how it should be done. If I'm doing everything from scratch I should think of the future too... have the least amount of data to be transferred / processed. Proxy is a really good catch, I did not even think about it! Can you please point me to a direction where I can learn more about how HTTP HEAD might return file info? I only found last modified date and content size which could be used to see if the static JSON changed; it won't tell me though if the project has new files / versions which I am interested in.

In some occasions the inaccuracy of Delphi's TTimer cased serious headaches so I did some digging and found an old post on how to turn a thread into a much more accurate timer. While I analyzed the code to find out how it works I made some comments and also took the liberty to change it here and there... and wrapped the whole thing in a TComponent which easily can be installed in the IDE and dropped on almost anything. The TThreadedTimer's OnTimer event runs in the VCL thread so any UI element can be manipulated from there. It is also a drop-in replacement for TTimer, meaning you change your DFM and PAS and it should work exactly the same way. My version delays enabling the thread so it won't spin freely until it has an event handler, properly resets the timing sequence if the Timer is set to Disabled and then Enabled again and also - in theory - the OnTimer event will no longer fire during destruction. Being a Windows-only guy it relies on WinApi, but I guess it can be made cross-platform by using Delphi's own TEvent class... as it yields zero benefits I didn't care to look into it. As the original idea wasn't mine the right thing to do is to release this version under the do-whatever-you-want license. Feel free to use, point out possible issues or modify it to fit your needs. God bless open source 🙂 uThreadedTimer.7z

No, as 1, TZipFile is clearly abandoned (ZIP64 was introduced over 20 years ago) 2, There is a workaround for most of these 3, I could not wait until Delphi 25 Llanfair­pwllgwyngyll­gogery­chwyrn­drobwll­llan­tysilio­gogo­goch when it will be officially fixed 🙂 Anyway. let's not hijack the topic.

- Inability to remove a file from the archive - Does not support ZIP64 and LZMA (which can be solved by Zip2) - Inability to read data from "corrupted" archives, like 7-zip can - If the ZIP file was created by C# (don't know which lib) TZipFile sometimes says that the archive is not corrupt, but there are no files in it. With 7-Zip I can see and extract the file without issues

Completely offtopic. I'm using an upped version of Delphi's TZipFile and always had enough of it's limitations so I went on and searched for VCLZip. Never seen a project this old before 😮

This is really interesting, since it's in VCL260.bpl...? Whenever I made these mistakes the AV always pointed to the executable (or .dll), rarely (or never) to a Delphi internal. In case it's true, how exactly this can happen?

I'm not saying it is the case at you but when my 24/7 application was closing unexpectedly it was always 1, Memory corruption 2, Improper thread object access / synchronization 3, In some cases memory / handle leak I found that if the code gets to a really messed up state no exceptions are raised when something happens (no more AVs), the program simply halts. My guess is somewhere in the stack... but I'm far away from this proficiency to be able to tell. I hope you have a test environment. Launch your application and set up performance monitors for memory and handle usages. I guess on a test machine only 1 serial port will be used, so give it 5x - 7x time, then extract your log and create a graph. If a value steadily increases - that CAN be your issue. For memory corruption, doublecheck your Move-s. I even wrote an own move procedure which in debug builds raises exceptions if I'm overflowing an area (I'm mostly using TBytes - that's how I know the limits). PChar reservations can cause this too - always reserve the memory for (Length(String) + 1) * SizeOf(Char). Thread things can be the hardest to find out, especially if they happen randomly. As performance is usually not crucial I tend to use a synchronized access to everything, that's why I have a generic TAtomicValue<T> class, which is handling the locking and unlocking for me. I'm using this instead of every simple value in the thread, plus inside every class which the thread allows external access to. Detecting memory or handle leaks is the worst. Build a test app and run each and every method 1.000.000 times. Whenever said value is increasing, something will be wrong there (also take into account that it CAN be inside a component, not necessarily your code). Or, simply purchase DeLeaker (or something similar). When I joined my current company it became a must-have; after I realized the memory management of our codebase was as messed up as the first applications I wrote. Took me more than a year to get rid of them (p.s.: I'm still wondering how noone recognized this flaw before...) "Happy" hunting. I wish you'll have your "ohgoddammit" moment soon.

Hello all, I was trying to get my head around this for a while but I simply can not find the solution. In my project I have a frame which I create runtime on a TTabSheet. On this frame I have 2 TTreeView components. 1 is on the frame itself, the second one is on an inner tabsheet: Now, if the TreeView on the TabSheet has ANY items created if the application has a VCL style active, those items will not be freed up. I put a breakpoint in TTreeNode.Destroy and I can confirm that those are ONLY being called for nodes in the other TreeView. I even put TreeView2.Items.Clear in the frame's destructor, but as Owner.HandleAllocated is false at that stage, deleting all nodes never happen. So I put a breakpoint in the frame destructor and TTreeNode.Destroy, and the thing I quickly realized is that the nodes of the first TreeView are being freed before the destructor, because of a window message (TVN_DELETEITEMA or TVN_DELETEITEMW). This is actually deleting the nodes in Vcl.ComCtrls.pas : 11910. But, this message is sent only to the TreeView on the frame, not to the one in the PageControl. Again, only when VCL styles are active. I will try to send this message manually in the destructor (or explicitly deleting all nodes...?) to see if it solves my issue, but it bugs me to hell that I don't know what is happening. If I'd know where this message is sent, I could investigate why it is sent to / received by one TreeView only. Anyone has any ideas? Edit: Manually trying to delete nodes will fail, as TreeView.Items.Count shows 0 at the destructor already. Edit-edit: I have an event on the frame itself which is being called BEFORE the .Free is called upon the owning tabseet. If I move TreeView2.Items.Clear in that handler, the memory leak disappears. I guess it's an other quirk of the VCL styles but I'd still like to know the reason...

Seems the editor adds an extra line break to the end of the translation 😞

Yes, exactly 🙂 Didn't see the button there. Thanks!

@Anders Melander Any chance for adding multi-line support in translations? I have a couple of button captions like Do something Shift + F9 If not needed, I wouldn't add the key combinations from code.

Hello, I was using MadExcept until now but it's limitations are growing on me slowly. First that it simply swallows TThread.FatalException which is an easy fix honestly, but still. But yesterday when I started to experiment with nested exceptions and realized that MadExcept simply doesn't support this I started to look for alternatives. I downloaded a trial of EurekaLog and while the settings look intimidating at a first glance, it seems to work as I expect. Before I'm reaching out to their official support I would like to know if anyone managed to achieve is a bare-minimum Eurekalog functionality: no bug reports, no custom forms; ONLY stack traces for exceptions? I'll continue to play around with the settings (Threads show no stack traces for now) but if someone knows what should I check and uncheck for the above I'd be able to experiment and decide if I'll switch a lot faster. Thank you!

Hello, I have a TObject descendant which has a message handler defined like: TProgressContent = Class strict private Procedure ProcessWindowMessage(Var Msg: TMessage); Message UM_WORKERENDED; End; This message gets sent when a worker thread finished so I extract some data and free it up. I started to get some strange results and some application freezes, and it quickly turned out that lots of other messages arrive in this handler. I have some 144's and 2's entering up here. The solution is easy - I just made an if Msg.Message = UM_WORKERENDED inside the handler and now all the issues are solved but isn't explicitly defining the message should already filter the rest out? Also, out of curiosity, what is a window message with the ID of 144? Couldn't find anything on Google so far. Cheers!

Well.. technically I am. I have one allocated with AllocateHwnd. I guess in this case all messages are sent there, even if it's filtered. Yes, I managed to find 2, but there was nothing on 144 though which I could find.

Issue is not Delphi Alexandria but that piece of ga... wonderful McAffee. I remember removing viruses by hand from multiple servers because the freshly updated McAfee failed to recognize a ~5 yo worm. We knew it worked because it always quarantined our patching tool. At work ESET, at home M$ Defender (AntiMalware, Security... idk how it's called nowadays) works just fine. The only reason we excluded Delphi executables is to speed up the building process on dev PCs.

All, I have a really strange situation here and I'm running out of ideas on what differences to check. Long story short: I have an application to access data in a database (Connection -> DataSet -> DataSource -> DBGrid, using Zeos and VCL DBGrid descendant for some custom features). In one specific codepage setting it seemingly starts to corrupt memory on ONE PC only (that I know of, that is). Details can be read here. Now, given the complexity the issue might be Oracle-related, PC environment-related, component-related, moon phase related or simply I messed something up in my code. My issue is that I'm running out of ideas to pinpoint the culprit. I'm not asking how to fix the issue (I know it's not possible without seeing the code), I'm asking for diagnostic steps; basically how would you start to debug this? What I know: - OS, Oracle client locale doesn't seem to affect the behavior - It's not thread-related, confirmed by moving all logic to VCL - Oracle client and server version seems to be irrelevant, the issue appeared with 12 and 19, client and server respectively - It is not a display issue, datasets actually contain garbage What I think: - I doubt the issue is in the component as the same binary acts differently on one PC only - Because of the same reason I would say that my code is good, too What I will do next: - Simple test app. Only a connection, query and a DBGrid - On a PC where it seems to work I'll use said codepage to see if just by some miracle corruption only happens at a later stage Thank you all for your input!

It seems I found and fixed the problem. For a quick recap: Using UTF16 character encoding I often experienced ORA-01406 fetched column value was truncated errors and when this went away by manually increasing the buffer size text data got malformed. As it turns out these two were independent errors and for the first my workaround is actually the solution. Text corruption was a bit trickier. There is a setting which controls how LOB data is cached and internally this chooses what kind of resultset is being created. The difference between the two of them was how the columns were defined: the one working properly checked if the codepage is set to Unicode and if yes it changed String types to UnicodeString. Moving the same block to the other resutset fixed the corruption.

I might be wrong here now as I believe this was on D7, but I think I met the same issue as @DavidJr. once. I lost my key to my own password database so I put together a simple brute-force program to try and open it with all possible combinations, using as many TThread descendants as physical CPU cores (which was 4 in my laptop back then). During the first test run it was immediately clear that something is off, as the total amount of CPU used was 25%; I couldn't max out the CPU. I seem to remember having some tries with BeginThread but I ended up running separate, single-threaded processes to utilize the rig's full protentional I managed to recover my passwords on (we had an unused 32 core Dell server which I could "borrow" for this task). P.s.: It seems yes, it was solved one day. Using the following test I can increase the total CPU usage count all the way up just by adding more instances to the list: program Project1; {$APPTYPE CONSOLE} {$R *.res} uses System.SysUtils, System.Classes, System.Generics.Collections; Type TMyThread = Class(TThread) protected Procedure Execute; Override; End; Var threads: TObjectList<TMyThread>; a: Integer; Procedure TMyThread.Execute; Var a, b: Integer; Begin b := 13; For a := Integer.MinValue To Integer.MaxValue Do b := b XOr a; a := b; End; begin threads := TObjectList<TMyThread>.Create(True); Try threads.Add(TMyThread.Create(False)); {...} For a := 0 To threads.Count - 1 Do threads[a].WaitFor; Finally FreeAndNil(threads); End; end. My laptop has only 1 CPU though, it's possible that the problem still exists, just on multiple CPUs?

@Attila Kovacs you are right. I extracted the data behind RawByteString, which was 195,129,108,108,97,110,100,195,179,32,108,97,107,99,195,173,109. This is indeed UTF8, and it just gave me an idea on where to look... @Dany Marmur The setting I'm talking about controls whether downloaded LOB data is cached or not. Inside the component this causes a different type of resultset created. On 3 PCs this setting was off (default), on one I turned it on. This is why one machine acted differently, as it was using a different type of resultset (the one which is faulty).