Best way to store data

[al17nichols 0]

I have some data that will only persist while the server is up and operational.  If the server is restarted daily the data is built over time as needed by the currently logged in users.  Currently I store the data using somewhat of a linked list.  This makes it harder to manage, ie. add, delete, or search for  messages for a single user, and is searched in one of 2 possible ways.  The question is which is the more efficient way to store this data: FDMemtable, or SQLLite.  I would say that at any point in time there might be 20,000 - 30,000 records during a given day.

[Fr0sT.Brutal 897]

The question is too broad currently. Answer depends on your operations. Linked lists are very good for adding/deleting but slower for search. Arrays are the opposite. Maybe you should consider hashmaps (dictionaries in Delphi). They simplify searching by key fields.

In general, I'd try to avoid 3rd party components if native classes could satisfy your needs.

Edited August 18, 2022 by Fr0sT.Brutal

[haentschman 92]

Hi...😎

Quote

by the currently logged in users

1. multiuser?

2. server?...other maschine?

3. persistent data? ...or daily?

4. program crash? ...which data?

[al17nichols 0]

20 hours ago, haentschman said:

Hi...😎

1. multiuser?

2. server?...other maschine?

3. persistent data? ...or daily?

4. program crash? ...which data?

Only accessed on the server by a threaded process.  Multiple connections via socket make be requesting data from the memory or sql lite store.  Want to use memory for quick access.  If the program crashes or the system is restarted the data needed will be rebuilt from a SQL database.

[Rollo62 500]

My guess from your rough description is that you want to store data locally

Quote

20,000 - 30,000 records during a given day

which might sum up day by day to a huge number.

In that case I think a real capable local DB, like Sqlite, should be preferred over an in-memory TFdMemTable.

Edited August 19, 2022 by Rollo62

[al17nichols 0]

On 8/19/2022 at 12:30 AM, Rollo62 said:

My guess from your rough description is that you want to store data locally

which might sum up day by day to a huge number.

In that case I think a real capable local DB, like Sqlite, should be preferred over an in-memory TFdMemTable.

everyday the memory data is flushed.  Only newly generated data is stored in the memory or db that I want to create.