Lars Fosdal

At what point is the instance pointer reference for the string, interface or dyn.array changed? When are field values in instances not aligned (unless overrides have been added)? https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement

Is it a rights issue? Can the app lack access to the location where the db is supposed to be?

Naturally, it is a simplification and doesn't apply to every context, but we used to have lots of micro locking, and lots of lock failures. The problem was multiple threads competing over multiple objects over multiple fields from the same objects for read access. Locking per field behaved like a "zipper" between threads. Think Allocation / Article / Saldo / Location - multiple allocations of the same article at the same location. Object locking with wait/retry on contention solved that for us. You could argue that MRSW locking would be a better approach, but that is its own can of worms if you need to increase lock level. A more important rule, is do not lock unless you really need to lock. Analyze the way the code works. Straight assignments are atomic, unless they involve logic or reallocation. Will there REALLY be other threads trying to access the same data at the same time? Do the data change so frequently and are they accessed so frequently that it is a real problem?

Never do locking on a per field basis - unless you have one critical section per field. Otherwise, always lock the entire object. As Georgge says: The risk of deadlock is real. Been there, done that. if ipUser.TryLock then try ipUser.Name := 'New name' finally ipUser.Unlock; end else // do a sleep/retry?

Use the FireDAC Monitor to check the operations sent to the DB. Can it be type related? What field type is mYear, what type is cYear? Or - looking at the error message. Is the connection actually open? Is the connection or dataset used elsewhere?

Indeed I do. Size 12 on 32" 4K display, Windows 100% UI scaling. The uneven height of some of the lowercase letters on JetBrain was the first part of the problem. The second problem was that the font was to "heavy" i.e. the normal was too close to what I consider a bold font.

Actually, I wasn't thinking of OTL at all - but that would be worth looking into. I attached a generic scheduler though. I had to clean out some proprietary code, but not much. The idea is that the generic type T is an enumerated type like f.x. TJob = (jobUnknown, jobUpdateFiles, jobPublishPage, jobCleanupOldData, jobEtcEtcEtc); The actual task manager is a different story. It would take me quite some time to "unproprietaryize" that one - and that is really where the work is. GenericScheduler.pas

When you put it that way, I wonder why I had the notion that a remote TCP host's willful disconnect actually signaled the connected.

Write a simple scheduler - i,e, a collection of tasks and when / how often you want to run them. Oneshot immediately, oneshot at a specific time, repeating by interval, repeating at fixed points in time, etc. Create a task manager thread that checks the scheduler and queues tasks into a task execution queue. You can have a framework of rules such as tasks not being able to run in parallel, or tasks that should not be queued if already in the queue or being executed, etc. The task manager thread also start the tasks from the execution queue and launch them as separate threads at the appropriate time. You decide how many parallel tasks you can run at a time, and you could even have a task worker thread pool.

I still think I prefer Source Code Pro.

I've had the socket open doing writes and reads. I still keep the connection open as I intend to send more shortly. If the remote point then disconnects gracefully - should I not be notified?

The reason is that I seem to get it without having had a disconnect event. If I have had observed the disconnect, I obviously would not try to send data on a closed socket.

@Hans-Wolfgang - The official definition of classes can be found here: http://docwiki.embarcadero.com/RADStudio/Rio/en/Classes_and_Objects_(Delphi) If you look at the left side of the above mentioned page, there is also a link to switch to German. Or use this link: http://docwiki.embarcadero.com/RADStudio/Rio/de/Klassen_und_Objekte_(Delphi)

What about EIdConnClosedGracefully? That is a kind of an annoying one.

TRec<T> = record value: T; procedure Init; end; procedure TRec<T>.Init; begin Value := Default(T); end; TRec<Integer> = record value: Integer; // Default(T) = 0 end; TRec<TObject) = record value: TObject; // Default(T) = NIL end; // Assuming nullables would look like the C# equivalents TRec<Integer?> = record value: Integer?; // Default(T) = NULL end; TRec<TObject?) = record value: TObject?; // Default(T) = NULL end; For nullable object references - will we have a non-NULL that is nil? I guess the C# solution makes sense, although there is a difference between NULL (never assigned) and nil ( known non-reference assigned).

You know me. I am always in a state of bewilderment 🙂 Based on your updated sync requirements description, I'll stop envisioning a new generation of your system, and instead tell you about how I solved a similar need for synchronization. I did distributed stock, energy and weather market data databases for well over a decade. The solution was database agnostic and the central server generated change instructions - which technically were binary scripts for adding, changing or removing data. The change scripts were sequenced in chronological order by changes done at the central server, so that to do a refresh, the client requested an update from the last synch point, and then convert the list of change instructions to operations relevant to the local database. The change instruction format remained unchanged even though the underlying databases changed from flat files, to client side Paradox, to server side sql over a tcp connection. I still shudder at the memory of having terabytes of Paradox tables that I couldn't touch except through the change instructions 😛 Today, these data would have been retrieved through a REST interface.

I liked the good old RESTful Web Services by Leonard Richardson and Sam Ruby on O'Reilly Media - although it is a bit web-centric. It is indeed complex. Some times peek/poke is the right way, while at other times you do want structured "batch" updates. The design needs to be governed by the nature of the API usage, the amount of concurrent use, the complexity of the data.

This is going to get interesting if we get nullable generic types.

If you have thousands of documents related to a case - how many of those would be opened in the same sitting? If you are able to reach the server at any time, I would synchronize the catalog only and pull the documents on demand to an MRU cache You could even keep MFU statistics to pre-load the most frequently used docs at first connect. Personally, I would consider having it all server-side with a web UI. Then again - this is the kind of problem that Office 365 and SharePoint really excels at - including tried and tested security models from all angles.

C# is not completely alien for a Delphi dev, once you get past the begin/end abstinence and operator learning curve. It is also low cost to learn - since you get the tools more or less for free (unless you need the heavy hitter stuff).

I have fallen into the trap of underestimating the complexity of cross-database code before. It is no walk in the park. Each database has numerous functions / best practices that go beyond "standard" SQL. Consider MS SQL's FileStream vs Oracle SecureFile / LOBs. The first is documented for FireDAC, the second is not. Do you need to encrypt the traffic between the client and the server? FireDAC SQL Server driver supports encryption. No mention of it in the Oracle FireDAC doc, or the PostgreSQL FireDAC doc. Off-topic: Down the line, you may need to support Azure, AWS, Oracle Cloud, etc. Just curious: Why must the documents be downloaded? Due to usage requirements in locations without network access?

Given that various databases have significantly different ways of storing files and optmize queries, I would hide the access behind a REST service. I'd go so far as to have entirely separate services for the various databases I needed to support. That would make it possible to do compression in a generic way, not having to rely on the database capabilities. It also would allow the client to be "ignorant" of the database specifics - and allow you to do changes and optimizations server-side without having to change the client.

In a good database, the cost of complicated where statements usually is limited once you have tweaked your indexes. Not sure you can get away from those where statements if the client will be receiving NEW rows once every now and then. Not sure how good Postgre is with temp tables., but in MS SQL you could write your existing id array to a temp table, and do a join query. Probably more efficient than trying to match thousands of IDs in an IN statement.

@aehimself Just a thought. You may need to support "as" naming to handle duplicate field names in table joins or IsNull code. select aStrField, IsNull(aIntField, 0) as aIntField from aTable Without the as - the aIntField will not be named in the result set - at least not for MS SQL. Then there is the question of count, max, min, and so forth... Once you open for custom queries - you basically open a can of feature requests 😉

I still haven't found anything that beats Adobe Source Code Pro