Migration from BDE paradox to TFDtable or other options

[Javier Tarí 23]

Does someone know a TTable replacement that really works?

 

It should have about the same performance than paradox, with hundreds of tables, many with millions of registers

 

Changing from ttable-kind to TQuery-kind is not an option at this time: it will be done at a later stage

 

Currently the only DB I know capable of this is ElevateDB; but I'd like to know if there are more possibilities

 

It is an actively developed program, with hundreds of installations that started 20 years ago with D5 and Paradox, and today is with D10.4 and still paradox

 

It must be solved in order: first change from paradox to a server based database, SQL capable

 

Once that is finished, there are plans to slowly migrate from TTable-ish to TQuery-ish

 

Thanks

Edited October 3, 2020 by Javier Tarí
Correction

[Guest]

I know IBO (www.ibobjects.com) has "compatibility" classes for TTable and some conversion directives. AFAIK it was written as a "BDE replacement".

I'm sure some other DAC with a couple of years of history have similar stuff.

[Vandrovnik 215]

IBX components also have a tIBTable. I did this migration in one-step (Paradox -> tIBDataSet/tIBQuery).

[ConstantGardener 31]

...have a Look at NexusDb

[Fr0sT.Brutal 900]

Hmm, SQLite+Zeos?

[Javier Tarí 23]

On 10/3/2020 at 8:41 AM, ConstantGardener said:

...have a Look at NexusDb

Will try It; thank you

I discarded It due to a confusion regarding the engine distribution and AWE server

[Javier Tarí 23]

5 hours ago, Fr0sT.Brutal said:

Hmm, SQLite+Zeos?

Why?

[Fr0sT.Brutal 900]

2 minutes ago, Javier Tarí said:

Why?

SQLite - lightweight, widely used, open source

Zeos - mimics TTable

[Javier Tarí 23]

On 10/3/2020 at 8:40 AM, Vandrovnik said:

IBX components also have a tIBTable. I did this migration in one-step (Paradox -> tIBDataSet/tIBQuery).

When I tried it, many years ago, the implementation was awful: it loaded the whole table in memory

A joke instead of a tool

[Javier Tarí 23]

4 minutes ago, Fr0sT.Brutal said:

SQLite - lightweight, widely used, open source

Zeos - mimics TTable

No luck... Seems TZtable implementation is also a useless one, loading the whole table in memory:

https://zeoslib.sourceforge.io/viewtopic.php?t=11296

[Guest]

@Javier Tarí check out IBObjects again. Write an e-mail to the author (jason) and explain your situation. Things will not happen the same second but i am sure he will be of assistance. If you make contact Jason have alternative licensing scheemes (trustware) so you will not have to pay "up front".

[Fr0sT.Brutal 900]

49 minutes ago, Javier Tarí said:

When I tried it, many years ago, the implementation was awful: it loaded the whole table in memory

A joke instead of a tool

 

44 minutes ago, Javier Tarí said:

No luck... Seems TZtable implementation is also a useless one, loading the whole table in memory:

https://zeoslib.sourceforge.io/viewtopic.php?t=11296

You weren't asking for any additional requirements. TTable by definition stores all its fetched data (but it won't fetch ALL records in dataset at once unless you make it to). I'm not aware of any dataset implementation that is able to keep only a part of data besides the most primitive IBSQL that stores just the current record.

Edited October 5, 2020 by Fr0sT.Brutal

[Guest]

Meh...

Edited October 5, 2020 by Guest
Meh...

[Vandrovnik 215]

3 hours ago, Javier Tarí said:

When I tried it, many years ago, the implementation was awful: it loaded the whole table in memory

A joke instead of a tool

I do not use tIBTable at all, so I do not mind. I use mostly tIBDataSet, tIBQuery, tIBSql...

tIBDataSet has property UniDirectional, which disables storing visited records in memory.

Edited October 5, 2020 by Vandrovnik

[Dmitry Arefiev 106]

FireDAC TFDTable emulates BDE/Paradox TTable behavior using LDW mode. This is expensive, and disabling LDW mode will improve performance. More info:

http://docwiki.embarcadero.com/RADStudio/Sydney/en/Browsing_Tables_(FireDAC)
 

But anyway, you will never get 100% similar performance to BDE/Paradox TTable with big tables. Any TxxxTable will request DB server and will fetch the data. When data is fetched, then operations will be fast and similar to TTable.

[Javier Tarí 23]

5 hours ago, Fr0sT.Brutal said:

 

You weren't asking for any additional requirements. (Snip)

I asked this:

It should have about the same performance than paradox, with hundreds of tables, many with millions of registers

And the proposed solution does not deliver that at all

Thanks anyway

 

1 hour ago, Dmitry Arefiev said:

But anyway, you will never get 100% similar performance to BDE/Paradox TTable with big tables. Any TxxxTable will request DB server and will fetch the data. When data is fetched, then operations will be fast and similar to TTable.

That kind of emulation is, as I mentioned before, a joke

At least if you are in multiuser, hundreds of tables, and millions of registers

 

Reminder: I'm not asking for something impossible; as I mentioned in original post, ElevateDB delivers what I'm looking for

 

Will try NexusDB asap

 

[Fr0sT.Brutal 900]

13 hours ago, Javier Tarí said:

I asked this:

It should have about the same performance than paradox, with hundreds of tables, many with millions of registers

So you suppose everyone is aware of the Paradox performance? Seems it was already dead when I finished school

13 hours ago, Javier Tarí said:

At least if you are in multiuser, hundreds of tables, and millions of registers

Multiuser with hundreds of tables? On Paradox? You're such a joker!

[Javier Tarí 23]

1 hour ago, Fr0sT.Brutal said:

So you suppose everyone is aware of the Paradox performance?

My bad; you're right. I'm so used to it that forget it's so old that currently it is unknown territory for most people

 

1 hour ago, Fr0sT.Brutal said:

Multiuser with hundreds of tables? On Paradox? You're such a joker!

I understand that you don't have real experience with it, as it was already dead when you finished school

Yes, I have lots of installations with Paradox, and didn't had trouble till some W10 updates started breaking it all

I don't have experience with more than 50 concurrent users, so I can't talk past that point. With 50, it works (worked) fine

 

Not that the configuration for that to work was easy at all, but once you know how to do it, it's a matter of making your program self-check and self-configure it on the computer it's running, and that's it

[Fr0sT.Brutal 900]

3 hours ago, Javier Tarí said:

I don't have experience with more than 50 concurrent users, so I can't talk past that point. With 50, it works (worked) fine

Well then, probably you have low concurrent queries in fact or a truly brilliant application. Here at work we once had a complex based on Paradox and it was just a Pandora's box of paradoxical bugs. Ten years ago it was changed to Oracle and forgotten like a nightmare.

 

Anyway, the initial question is sort of incorrect. What exactly do you mean by performance? What structure your complex has? How many users, requests, reads/writes etc are expected? These are architectural questions. Implementation-specific questions are related to DB-classes. Usually if you reach the limits of TTable or experience very high memory consumption, it means you have wrong structure ("select * from DAMNBIGTABLE" + TTable + DBGrid with possibility to fetch billions of records). Client app doesn't need too much data.

 

Simple example of low memory consumption: one master table with only ID's and multiple detail tables linked by this ID to master. In every moment client app will have loaded only the list of ID's and details for just a single ID.

Edited October 6, 2020 by Fr0sT.Brutal

[Lars Fosdal 1793]

Back before 2006, I had literally terabytes of data in Paradox tables on client sites.

It initially used local tables on the client, the moved to shared tables on a file server, before I wrote a client/server middleware type of access method that got rid of a LOT of sharing/concurrency issues.

I was very happy to leave that in the back mirror.

 

If you want to keep the Paradox tables but change the access method - I'd question if it is worth the hassle.

If you are going to migrate, why not do it properly?

[Javier Tarí 23]

25 minutes ago, Lars Fosdal said:

If you want to keep the Paradox tables but change the access method - I'd question if it is worth the hassle.

If you are going to migrate, why not do it properly?

Nonono... I want both to first get rid of paradox, and get rid of TTable access method

But I can't just rewrite the whole application in a step

 

So 1st I would get rid of Paradox, keeping the access method, so the app still works

And then change the access method through the whole application, step by step.

 

[Javier Tarí 23]

2 hours ago, Fr0sT.Brutal said:

Usually if you reach the limits of TTable or experience very high memory consumption, it means you have wrong structure ("select * from DAMNBIGTABLE" + TTable + DBGrid with possibility to fetch billions of records). 

Remember is just paradox with TTable, so no select, no queries. You just open the table and it doesn't matter if it's 10 records or 10 millions. If you stick to locate, SetRange, it will always be light speed

When you need to crunch records, then it fall to it's knees and cryes

[Lars Fosdal 1793]

12 minutes ago, Javier Tarí said:

But I can't just rewrite the whole application in a step

In this case - you just may have to.  The mechanisms are so completely different from Paradox to SQL, especially for large tables.

You could in theory write a wrapper that can emulate locate and SetRange - but that would be a stumbling block.

[Guest]

I will release an OS "application-transport" framework. It's very RADddy.

Using something half-baked or not, in 30 years i have re-written (not upgraded) my core "products" at least 5 times.

These rewrites are a generation step. Each client must be manually converted / educated / involved in the process.

Each time, i had to bully myself into board meeting, pondering, even shouting about technical debt.

Writing up those contracts were never easy.

 

IMHO this is the only way to maintain a system over 30 years.

In that kind of timespan, you will encounter points in time were "upgrading/updating" simply does not suffice.

[Attila Kovacs 631]

there is also https://www.componentace.com/bde_replacement_database_delphi_absolute_database.htm