Jud

I've abandoned the old BlockRead/Write for tStream.

This goes back a few years, but when I got my four-core hyperthreaded i7, hyperthreading could get 1.5-1.6 more performance than a single core, in my tests on CPU-intensive tasks. Less than that on memory-intensive tasks.

You may not have installed the help file and the source code for the units.

That is what Delphi Basics says, and it appears to be right. OTOH, if in the Delphi IDE you do F1 for help on BlockWrite (or BlockRead), it says that it is an integer function, and the source code for the system unit shows it as an integer function.

Actually I did benchmarking. The first version of this program read a 30GB boolean array from a drive and used that. Naturally I did a single-thread before multithreading. I forgot the ratio, but 20 threads on a 12th generation i7 was something like 8-9x faster than the single-thread version. Now I'm working on replacing the 30GB boolean array with a much larger bit vector.

No benchmarking with the bit vector yet, but I've definitely encountered the memory bottleneck. In old tests with 8 threads (hyperthreaded 4-core i7), I would get about 5.5x over a single thread on CPU-intensive tasks. But on tests that were memory-intensive, I would get around 3x on 16 threads.

That looks like what I'm looking for!!! I'm going to try it shortly. Thanks!

If I do "Find Declaration" on BlockWrite, it goes to an integer function: function _BlockWrite(var F: TFileRec; Buffer: Pointer; RecCnt: Integer; RecsWritten: PInteger): Integer;

I doubt that it would be workable, since they are natually one dimentional, and the program just needs to see if a bit is 1 or 0, and it needs to do a large number of them.

Interesting idea, but I don't know how to deal with the GPU. This will be running 20 instances in parallel, but the accesses will be pretty much random. The target calls for doing at least 10 trillion accesses.

Performance is an issue. The plan is to do at least 10 trillion accesses to the bit vector - more if I can manage it.

Here is a short demo. With Delphi 11.3 I get a compiler error on the line with using BlockWrite as a function. The documentation says that BlockWrite is a function that returns an integer. procedure ErrorDemo; var CountNewData, WriteResult : integer; OutFile : file; NewData : array of int64; begin AssignFile( OutFile, 'c:\NewData.data'); rewrite( OutFile, 8); WriteResult := BlockWrite( OutFile, NewData[ 0], CountNewData, WriteResult); { compiler error - E2010 incompatible types - integer and procedure } BlockWrite( OutFile, NewData[ 0], CountNewData, WriteResult); // no error CloseFile( OutFile); end;

It is a compiler error - not a runtime error.

1. 32GB is nothing today. Every computer here that it might be running on has at last 128GB of RAM. 2. No, this is the best approach.

I estimate that about 5% of the bits will be 1s. They will be read-only in the actual run, so the data will be shared among 20 threads. I don't understand how storing just the 1s would work. That would result in a series of about 15,000,000 1s, but that would be useless.

Yes, I need more. I current;y need about 300 billion bits. I can write it myself, but I thought there might be one already available.

Recent models of Intel processors (12th and 13th generations) have Performance cores and Efficient cores. How does Delphi (11.3 in particular) handle this with the parallel for? Does it assign tasks only to performance cores or all cores?

Well, I need all of the power I have available. BUT - I realized a fallacy in my thinking and analysis. I assumed that if I ran 16 threads with the parallel for loop, they would all be on performance cores, and that if I ran 20 threads, the extra 4 would go to the efficient cores. But after more experimentation, parallel for seems to put the threads on any core. I thought of timing how long each thread took to run, and with, say 20 threads, there wasn't that much difference. And running 20 threads with parallel for (with 8 P=cores and 4 E-cores), the performance was 9-10% better than running 16 threads. So it is using the E-cores too. And running 100 or so threads, it pretty much evens out the difference between the E and P cores (because the ones on the P-cores finish sooner and get reassigned another thread.)

I don't know how it is handled, which is why I was asking.

I moved from an old computer to a new one, I get the following message about not being able to add columns or rows when I try to bring up a VCL project that worked on the old computer. I used the migration tool to copy my settings to the new computer. How can this be fixed? Also, I get a bunch of access errors after this. PS - also, this seems to happen with every VCL project but not with console apps.

Sorry for the delay, but that fixed it.

Thanks. After I had posted the message and two updates, I searched for the words in the error message and found that the problem had been answered: "That is known (reported issue). There is some problem with migrating or applying Welcome screen layout after migration. When you launch IDE click Edit Layout on Welcome Screen. Reset Welcome Screen to default layout and then adjust it again the way you like it. Next time you start IDE it should run normally."

I also get this error when trying to start a new VCL project. Can someone help?