No one can help

[programmerdelphi2k 235]

one tip:

• as your problem is just 1 specific record, then this record can be "trucated/corrupted" in your DB
• then, any try to read it... generate a internal error:  like "out of memory" is common in this case!

[limelect 47]

@programmerdelphi2k I sent you a private message

[PeterBelow 218]

5 hours ago, limelect said:

After so many years of successful Delphi, I almost give up

on my free program.

After out of memory  I made a new test program and this is what I came up with

1. NO PROBLEM with SQL.

 

 

Have you tried to build the program as a 64 bit executable? You also need the 64 bit version of MySQL of course.

[Anders Melander 1639]

Instead of all this guesswork why not simply use the debugger to determine where the out-of-memory occurs?

[Daniel 417]

Next time please choose a meaningful subject for your thread.

[limelect 47]

@Daniel all I meant by this thread is that my test program just

a DB grid and a few buttons the LOGIC is OK but on my full program

with the same logic, I have a memory problem.

I did not expect anyone to solve the problem as it is NOT a logic nor an SQL problem

which I thought at the beginning.

but a BIGGER Delphi problem as my main program is full of components 

forms and more.

I never saw such behavior with hunters of projects behind me.

This is a warning to the Delphi community about what to expect.

And finally, I hope I am wrong.

 

P.S I will keep looking into this problem

 

Edited December 11, 2022 by limelect

[aehimself 388]

Without seeing the code it's really hard to tell what is eating up the memory. I also have zero experience with FireDac, but...

 

I suspect you have a query, with a DBGrid connected and this query also downloads the BLOB. The issue is that the query will at least download all the blobs for all records which are visible in the grid, worst case it downloads all. This is already using up a huge amount of memory. When you assign the value to a string, you allocate even more memory, resulting in OOM.

What I'd try is to exclude the blob from the main query, add a second query with SELECT blobfield FROM MyTable WHERE ID = x. You can open this manually upon record change, load the data in the RichEdit and close it immediately. That should minimize the memory usage.

[limelect 47]

@aehimself I thought so too. One of my  tests was

to delete all components involved

 

Just one record !!!!

FDMemTable1.CloneCursor(ProjectsFDTable);<< Move record to memory table
//  FDMemTable1.RecordCount << check to see if data is inside

>>>>>> Close and free all databases and even connections!!!

ProjectsFDTable.Close;
FilesFDTable1.Close;
 FDConnection1.Close;
 ProjectsFDTable.Free;
FilesFDTable1.Free;
FDConnection1.Free;

 FDMemTable1.FieldByName('Description').AsString;<<< Then use the record

or this

MyMemStream:=  ProjectsFDTable.CreateBlobStream(ProjectsFDTable.FieldByName('Description'),bmRead);

crazy as it sounds memory problem.
 

"That should minimize the memory usage." It did not

 

Edited December 11, 2022 by limelect

[limelect 47]

Whoever wants to see which free program we are talking about

https://limelect.com/downloads/document-projects/

The main picture does not show how many pages there are.

Inserting has no problem

[limelect 47]

If someone wants to see Delphi in action goto

https://limelect.com/product-weighing-software-page/

A crystal growing machine 2 years in development with Delphi 4.

A weighing machine with a PC card and Delphi with D6 and 7

And many more.

[limelect 47]

Well as a last resort to find what's making the problem 

I developed many years ago what I call "scrounge the software"

which means taking parts of the program/or components out

until you reach what is your problem

Obviously, work only on a copy of your program. 

A lot of work but worth the self-teaching

Edited December 11, 2022 by limelect

[Hans J. Ellingsgaard 19]

3 hours ago, limelect said:

FDMemTable1.CloneCursor(ProjectsFDTable);<< 

Have you tried using a FDQuery with a where clause, where you select just one record, instead of a FDtable component?

[limelect 47]

@Hans J. Ellingsgaard yes by now I tried all combinations.

See above my last resort moving 1 record to memory freeing all

components and using in-memory just 1 record tough large but only 1.

I am left with no choice but to find which component/components

have an impact on the program

I am confident now that it is not only the database since I can insert, delete, change and do any manipulation on it.

except for reading one record under the search condition.

The search condition applies to all records with no problem only the use of 1 record.

 

P.S all other records can be read (and there are more than 700 of them)

with no problem.

 

At the beginning of the problem, I started with SQL only then I changed to 

combination of table and SQL.

Edited December 11, 2022 by limelect

[programmerdelphi2k 235]

as I said, your root-problem is your "record" in special... then:

• try just use your "SQL manager" to copy it for another "NEW" database!
• that way, your SQL file (target) will be cleaned of any "superfluous data", and the new record (if everything ok) can be more wiped!
• In case of problems with corrupted data, SQL-manager can try to recover some information!
•  
• But if, really, what you told me in the private-message is true, a record that is 1GBytes in size, then, certainly this is the real cause of the problem, because, when loading this 1GB, in a new "stream" , you could have 2GB or more as a result, because 1GB of the record + 1GB of the stream, and, who knows, +1GB of the visual component = 3GB, do you understand?

Edited December 11, 2022 by programmerdelphi2k

[limelect 47]

@programmerdelphi2k Sorry not to agree as since on my test program

with 1 grid and a few buttons, this same database is reading all records

even the large record.

I suspect as the full program has lots of components altogether

are the problem.

Somehow the combination does the problem.

 

P.S I send you the source for the test program and the database

see the 5'th record and the test read it

Edited December 11, 2022 by limelect

[programmerdelphi2k 235]

you send me a link with DB +1GBytes ... it's not possible download, I said to you!

[limelect 47]

Sorry I cannot help 

My test program works with the same logic

[programmerdelphi2k 235]

me, sorry about you! see you later!

[Fr0sT.Brutal 897]

First you were talking about 64M record. Then it appeared to be 1G. Where's the truth?

Anyway why not execute the app step by step and watch for memory consumption ? This will give more useful info than the whole this thread

Edited December 12, 2022 by Fr0sT.Brutal

[limelect 47]

@Fr0sT.Brutal number 4 in my first writing

4. leaving only one large record in the database, NO PROBLEM. Record size 61094574.

Can you elaborate on how to see memory consumption while debugging 

do you mean to add software to the source at some points

or there is an IDE tool for that which I never used?

D10.2.3

To make it clear your way might help but how?

I was trying to do what you suggested with vmmap.exe.

I tried a while ago but did not see anything special. all numbers were fixed.

Do you have a simpler tool?

 

P.S on my test program which has only a grid and button same

logic works which let me assume that components are involved

Edited December 12, 2022 by limelect

[Fr0sT.Brutal 897]

5 minutes ago, limelect said:

4. leaving only one large record in the database, NO PROBLEM. Record size 61094574.

I was referring to this

19 hours ago, programmerdelphi2k said:

a record that is 1GBytes in size

1G will surely cause issues. 64M shouldn't. Anyway without reproducible test project there's nothing more I can say.

BTW, FastMM throws OOM exception if a memory is corrupted.

6 minutes ago, limelect said:

Can you elaborate on how to see memory consumption while debugging 

Task manager

[limelect 47]

@Fr0sT.Brutal for some reason I did not use Task manager

As you can see the line is almost top of the screen

Any suggestion? Software wise

Edited December 12, 2022 by limelect

[skyzoframe[hun] 4]

I don't understand: why do you need so many big files, or blobs, in your database?

 

by the way.:

 

 

They are the same drawings in different file formats. If I need to save something in database, I'll use a *.emf file.

 

I only have one question: Can you reduce the file size somehow?

Edited December 12, 2022 by skyzoframe[hun]

[limelect 47]

@skyzoframe[hun] The only way to reduce Is to zip and then open.

I am using advrichedit which holds pictures and text obviously this

move to a database.

There are more options to decrease the size of the picture however

the whole program takes up a lot of space.

However, I am allowing to put a few pictures in the record which will bring back the

size problem.

Maybe the best is to decode as your suggestion to EMF file

However, advrichedit is streamed into the DATABASE so I have

To see how to split the data and use it.

 

Waaa it starts to be complicated

 

P.S using ADVrichedit is great as it allows printing in many formats

Edited December 12, 2022 by limelect

[limelect 47]

Right now from my point of view as a programmer, I catch the screen of a

program as a picture copy to clipboard saves to ADVrichedit so simple.