bobD

On 3/25/2022 at 11:08 PM, Jeff Overcash said:

Database unavailable often happens because you have multiple InterBase installs and have the IB_PROTOCOL environment setting set either globally or within the IDE's local variables.  If that variable is set it must match the running server's port name.  If it doesn't you can not make local connections only loopback TCP connections.  You do not need IB_PROTOCOL set for anything it is just a quick way to identify the running instance so you can remove the environment variable without worry.  It is one of the first things I remove after installing a new RAD or new IB instance (I keep all mine on port 3050 just manually start the one I want to test against).

Sounds like a reasonable hypothesis, because this has been my main development machine for a few years now, and has seen multiple IB versions. Also the results are the same whether I try the sample app, or connecting to a local copy of my actual intended target db. I'll try to see what I can find this week after looking at cleaning up the installation.

Thanks, bobD

Thanks for response. I also looked at and followed along with the set-up demo at 

but no joy. At the end the project could not connect. Still trying to figure out what's wrong with the installation. May try uninstalling and reinstalling PAServer next.

I have similar outcome with the FireIBLite.dproj sample. If the local IBServer is running, I get a user name and password not defined error, If I check the Lite option and stop the local server, then I get database unavailable. Very frustrating...

bobD

On 3/22/2022 at 4:17 PM, corneliusdavid said:

 

It's better to free the memory outside of the class itself in a try-finally block:

 

Absolutely. The inclusion of Free in the instance method is essentially a side-effect of the method and constitutes a violation of the SRP. What if I want to call DoSomething twice?

Also, as a matter of consistency I personally always call the inherited constructor--even when descending from TObject directly. I have no guarantee that TObject.Create will always be an empty method.

Currently running IB 2020 server, using Delphi 11 Arch, via FireDac

Everything fine with  that, but want to add offline capability. Is there a simple embedded-IB sample project available anywhere. I need to know what dlls to use and include with application.

Intent is to have the application request a connection, then the connection unit

--return a server connection if on the local network, or

--if not return the local imbedded connection

Alternative might be to always open local, then handle network synch if on the network

Database is low volatility (read often, write seldom)

thanks,

bobD

22 hours ago, david_navigator said:

1. My UI, so I "know" when a user changes a date.
2. I need to know if any lists are the same as my list.
3. answered by 2 - I don't want to know anything if they're different.
4. No. Imagine each list represents a hotel room and each element of the list is the check in and check out time for each visitor over the next 6 months - I need to know if any two rooms have exactly the same booking schedules.

 

The dates change frequently, the comparison gets run infrequently, but the compare has to be very fast and the date changes can be slow.

"I need to know if any lists are the same as my list."

This implies an 'active record' where the question is really "does list <x> (my list) already exist in set of lists <y>"

"if any two rooms have exactly the same booking schedules."

This implies a much more complex question: "given set of lists <y>, are any two [or more] lists the same?" (basically, an iteration of the first question where x = 0 to number of lists -1.)

which is it?

Suppose a user changes a date-item for a room. One of four things happen:

--the list, previously distinct, is now a duplicate of another

--the list, previously a duplicate of another, is now distinct

--the list was and remains distinct

--the list was and remains a duplicate

Can we rule out any of these cases as impossible?

Does anything have to happen as a result of any of these cases? (For example, the change not being allowed)

Is there any persistence aspect to the question? What's being stored?

My initial design thought is to maintain a sorted list of list hashes, so that the question "does list <x> (my list) already exist in set of lists <y>" is a simple look-up on the list hash (which can itself be calculated and stored with the list whenever it changes). 

however, the statement "The dates change frequently, the comparison gets run infrequently" makes the whole idea of pre-calculation a case of swimming upstream. You don't generally make a frequent operation slower in order to speed up a relatively infrequent operation. So I agree with Joseph Mitzen's response that cautioned that some instrumentation is called for.

four questions:

1. Where do the datetimes come from and how/when do they change? For example, if you're comparing file datetimes, then maintaining a status when the data changes might be better than checking when you need to know. They might change a lot less often than you check for a difference.

2. What is it you really need to know: whether any particular pair is different? or whether any difference exists anywhere in the list?

3. Is that always simply a 'difference exists' boolean? Or would knowing [greater than | equal to | less than] also needed?

4. Is there a guarantee that for every element[x] in list one, the same element exists at position[x] in list two? How is that guarantee enforced or checked?

All of these affect your potential solution options. For the first example, if the list changes much less often than the need to check, then you might create a tracking object with two boolean values: FUpdated and FDifferent. Set FUpdated to true when any list element changes. When checking, if FUpdated is false, then simply return the FDifferent boolean, or if true recheck the list, store and return the comparison operation result, and reset FUpdated to false. This can save you thousands of list comparisons if the lists are much less volatile than the need to know whether they're the same. Answers to the other questions could modify the structure of this tracking object.

My sincere condolences to his family and all who knew him. Many of us continue to live by choice largely in the mental universe he architected.

bobD

13 hours ago, Fr0sT.Brutal said:

Solving similar issue in my map control,. I came to 3-level cache:

- Hot: Tbitmaps

- Medium: PNGImages

- Cold: TMemoryStreams

Capacity of all levels is limited by memory and GDI handles.

Comments in OSM.TileStorage.pas unit explain decisions I made

That encourages me that I'm on the right track with bitmaps. 

I'll report back with some further testing.

6 hours ago, mvanrijnen said:

The TJPGImage has a TBitmap in it which will take the memory i think. This is a simple calculation, width,height,bits/pixel or am i  mistaken?

mistaken 🙂

 

In what regard?

What I know so far is this: the lines

    img.Scale := jsEighth;
    img.Performance := jpBestSpeed;

speed up the load/display of the jpg file as advertised

Using the following code,

    bmp := TBitMap.Create(img.Width, img.Height);
    bmp.Assign(img);
    bmp.SaveToFile('C:\Users\bobd\Desktop\TestBMP.bmp');

and starting with a jpg file of 15.4 MB results in a bmp file of about 1.14 MB if Scale and Performance are so specified, or with the lines commented out 73MB (!). 

That suggests to me that there's no direct calculation of the JPGImage size from the bitmap it produces. You can see either a > 10x reduction, or a 4-5x expansion.

Interestingly though, the TJPGImage object saving itself to file after the scale and performance are reduced for load results in a file the same size as the original. So we can't assume that the TJPGImage so loaded is any smaller in memory that one loaded without the jsEighth reduction. That's interesting.

It appears to me that the most memory efficient cache (and thus the one with the largest practical capacity) for this app is going to be loading at jsEighth, then creating and caching the bitmap. Since the original image can be 6192 x 4192, I don't really need all that resolution anyway.

Unfortunately, this presumes something not in evidence: that the size of the TBitmap in memory has a direct relationship to its file size. As with the source TJPGImage, it doesn't have an accessible size-in-memory property.

59 minutes ago, David Heffernan said:

If you are going to cache them, cache the jpeg stream rather than the decoded image. Then decode the image on demand with LoadFromStream. 

I can do that. Is that suggested for performance or memory footprint?

It bugs me that I can't see what I'm doing. InstanceSize returns 100 bytes for the img, 24 bytes for the stream. Obviously neither really accounts for the data.

bobD

I'm working on a quick photo browsing app, using a TObjectDictionary<string, TJPEGImage> as a cache device. Images are requested for display using the path to the jpg file like this:

procedure TJpgStore.LoadPicture(aPicture: TPicture; const aSource: string);
var
  img : TJPEGImage;
begin
  if not FDict.TryGetValue(aSource, img) then
  begin
    img := TJPEGImage.Create;
    img.Scale := jsEighth;
    img.Performance := jpBestSpeed;
    img.LoadFromFile(aSource);
    FDict.Add(aSource, img);
  end;
  aPicture.Assign(img);
end;

This all seems to work fine, but two questions (since I'm not really a graphics guy):

1. Am  using Scale and Performance correctly? IOW, does it make any difference being here or after the LoadFromFile call?
2. How much memory does a loaded TJPGImage take up? ('m thinking about a least-recently-used cache size governor. The jpgs are commonly 15-25MB each, and a source directory might contain 1-5,000 picture, so caching them all doesn't seem realistic...)  

Those are specific questions, but glad to entertain other comments/suggestions/alternative approaches.

Thanks,

bobD

For whatever reason I've never used cached updates.

Fuller context is that I have an app on my local home network that uses an IB server. There are two main failure conditions: (1) the laptop's Wifi connection to the server is wonky, or (2) I'm off traveling and not on the home network at all.  I'm currently considering three different architectural approaches as I go from simple C/S design to briefcase (or more generally a 'network-optional' design).

1. Use IB's built-in change views--this would be preferred if I could find a decent tutorial for getting started. Major documentation shortfall here.

2. Use a server first approach (app tries to connect to IB server, failing that it opens local IB database). Obviously that requires I write a server<-->local reconciliation routine.

3. Use a local-first approach. That removes the need for solving the network slow-fail issue, because all server communication would be confined to the background reconciliation thread.

5 hours ago, Dany Marmur said:

When i had direct connections to DBs from applications, i would do the connection in a thread so the UI at least was responsive during a possible time-out trial.

The only solution that will cover this is a middle-tier. One could probably write a very small https server that only knows/cares about the DBs availability (sits on the same machine) and thus every now and then checks the DB. You could the make a simple request to that "database discovery service".

Thanks for suggestion.

I've actually isolated the problem at this point to the WiFi connection being flaky, since the app works fine on the wired-in network machines even as the laptop fails to connect, and stopping/starting the laptop wifi service always fixes the problem. The DB machine and DB are solid as a rock.

One of the things I'm thinking about doing as the app evolves is making it briefcase-enabled by running an local DB, and having the app run locally if the server isn't available. That creates an obvious synchronization requirement of course, and hence my interest in learning about change views. But that might be way more complicated than I actually need--the data at this point is pretty stable.

4 hours ago, Anders Melander said:

Why not just set the database connection timeout to the desired max. Connecting to the server with some other method fileshare/port 3050/whatever amounts to the same thing. The only reason they can get the result faster is that they have a smaller timeout defined.

I'll probably go with that for the moment. And I'll admit that 'seeming forever' is a very subjective complaint. The app normally opens with a fully populated dataset w/i a second or 2, so waiting is just waiting for inevitable exception handler.

bobD

19 minutes ago, Fr0sT.Brutal said:

All checks except the actual connect to the base you want are unreliable. Show splash screen if your app is useless without DB or connect in background thread otherwise.

Thanks Fr0st--

See my reply to Anders--Given the type of connection fails I get, I'm not actually looking for a 100% solution. Unreliable might be fine depending on why the test fails. For example, I wonder whether I can ask the router if the sever is available rather than even trying to even talk to it. The server itself doesn't publish for network browsing, nor does it even respond to pings--effectively it's stealth except for the database connection (on the standard 3050 port).

41 minutes ago, Anders Melander said:

Don't try to use the file system to verify the existence of the DB server. The server might not have the required ports open or the file sharing service available. Perfectly normal (and good sense) for a DB server.

Thanks Anders--

Home network, so the connection itself is 99.9+% reliable. The only time it can change is if I do it. What normally happens is that the laptop for whatever reason loses its wifi connection to the server. Turning wifi off and on always fixes things. And I simply don't see any failures on the wired-in desktops. So I'm perfectly fine with a file check not actually testing the connection validity, just whether the server is reachable.

Environment is current Delphi 10.4.2

FIreDac working against up-to-date 2021 Interbase

What's the best tutorial or book for developing a Delphi-Interbase change-view based app?

Looking for some simple procedure tips like

a. develop for server first, then add local when design stable?

b. develop for local first, then add network database when design stable?

c develop using change views from the start?

Also, does using change-views carry any db design implications? My normal practice in C/S development has been using a key-cache or Hi/Lo approach with integer keys (https://en.wikipedia.org/wiki/Hi/Lo_algorithm), but wondering if I need to switch to GUIDS...

Thanks for any help/tips/references

bobD

Environment is current Delphi 10.4.2

FIreDac working against up-to-date network IB server.

I have an app that opens the database on startup. However, while calling Connected := True is fine if it works, it can take seeming-forever to fail if the server can't be reached. Is there a faster call in FD/IB to make sure that a network server can be reached?

My standing practice has been to call FileExists(<db or marker file>) as a simple precheck on a connection, but that doesn't work in this case because the server does not publish its existence. FB can connect, but the server doesn't show up in standard network browsing. So if you type in a server location in FileExplorer it will request connection credentials, but FileExists simply fails. I haven't found an access-credentialed version of DirectoryExists or FileExists that I can use.

Thanks for any help/pointers,

bobD 

16 minutes ago, Henry Olive said:

Is it possible to solve this issue with a s.proc ?

Certainly. but the details of that would depend entirely on the database schema, including knowledge of the table structures, the view you're referring to, and any/all other relevant complications (such as db triggers and rights of the the user connection.)

One of the first questions that occurs to me is "what do you mean by an 'invoice'? My tendency is to regard invoices as simply statements (ie, point-in-time reports) of the financial status of an account. The 'real' underlying objects being reported on are the account and its billable orders. Credits applied to an account would normally be applied to each billable order amount in sequence from oldest to newest until the credit was exhausted, w/ any amount left over then becoming a positive balance (credit) on the account. A superseding invoice or refund might then be generated to reflect the new account status.  

bobD

16 hours ago, emailx45 said:

First, I would try alternatives to regularize the performance of your network,

https://www.embarcadero.com/interbase-licensing-options

In your RAD install, you have in (Samples folders)  ...\Object Pascal\Database = FireDAC and Interbase Demo for tests!

Don't know if I provided too much context or not enough.

I've got a pretty good idea what's slowing down the wifi, but not really interested in that, because (1) the unit router actually belongs to the apartment building. It's single band 2.4, but more than adequate for most purposes, and more importantly (2) this is mostly just an excuse to learn more about IB and FireDAC. I worked professionally against SQLServer for well over a decade, mostly dbGo, but also ODBC and (many years ago} BDE. But I never had the occasion to write a briefcase app: all strictly C/S. Since I'm retired now I have time to learn IB and FireDAC.

What I didn't see in the Rad Studio samples is a pair of clear examples of the difference between setting up a Delphi FireDAC app to run IB embedded vs IB desktop. Is that a matter of FireDAC connection settings? Specific files in the connection datamodule uses clause? Some specific FD physical layer support component? I can easily set up a local IB connection, but since my dev machines have a local IB server running, I can't say for sure I know whether it's embedded vs desktop. I want to get the connection options completely sussed before moving into all the other new (to me) IB/FD goodies

bobD

From the description of where you are ("click on button perform queries and display results"), this is an immense topic.

Probably the classic intro to object oriented design is Object-Oriented Software Construction by Bertrand Meyer

For a simpler approach (and with the benefit of being written for Delphi), try NIck Hodges' books on coding in Delphi. They're straightforward and very approachable.

Bit more detail in Hands-On Design Patterns with Delphi by Primoz Gabrijelcic

These  will get you started with OO thinking--the prerequisite to architectural layering.

In the mean time, how to refactor your current code to make it more maintainable? I'd suggest trying this:

New rule: no data access objects on forms. Those belong in data modules. Organize the modules into groups of logically related queries. That will start to separate out your code by 'topic of concern' so you can start grouping logically related code.

bobD

Context latest Delphi 10.4.1, Arch and Interbase 2020. Windows 10. FireDac.

Local home network, both cat-6 wired (desktops) and wifi (laptops)

The laptops are much slower on connecting--enough to be annoying--so I'm thinking about using primarily either ToGo or a local IB server. The data has a high-use but low volatility, so coordinating between the local and main servers will be relatively trivial.

Is there a good sample app for setting up an embedded or local IB delphi app? I assume I have to include different files. The one briefcase model I found used a server and local flat files.

Thanks for any help or links.

bobD

15 minutes ago, Mike Torrettinni said:

Is this true? The 'blue dots' are relevant after compile is done, so 1 unit or 1000s of units inproject, after compile IDE should know what's used and what not, right? Of course referring to local units, not packages or dlls...

The blue dots indicate that code was generated for that line, not that that code is known to be called. It's 'used' only in the sense that the compiler used it.

For a public method, knowing for sure that a public method is not used would entail inspecting the implementation code for all other units in the project; the class might appear in an implementation uses clause, so just reading all the interface sections wouldn't do it..

So Delphi would have to be rearchitected as a multi-pass compiler to optimize the code at that level, inspecting and cataloging the entire code base before compiling anything.

Said more simply:  while you can know what's used and what not, the compiler has no way to know that at the time it's compiling the unit.

I appreciate the fact that Delphi has solved the structural issues. Now, recommendations for a vendor that provides a high-dpi capable library of replacements for the classic (Delphi 7 era) icon files? I have an older app I'm updating, and need icons that will work for my menu items and speedbuttons.

Thanks,

bobD

On 9/9/2020 at 10:43 AM, Remy Lebeau said:

Just my 2-cents worth - I prefer to use something more like this:

What Remy's approach does is accomplish OO 'separation of concerns'--decomposing all the logic of 'Save' into those component parts which might be subject to independent change and testing. So while a bit more code initially, it's much more maintainable and reusable. The pattern is equally applicable whether the storage mechanism is a TList or a zillion record database--only the technical details of how find, inset, and update work need be addressed. The higher level application logic is stable because all the underlying technical details have been isolated out.

 bobD