How can I allocate memory without raising exceptions ?

[Marus 0]

I need a safe way to allocate memory for a string variable, without getting exceptions. I don't want to use "try-except" blocks and neither the function that do this, use them internaly. I need a speed optimised method. If there are errors it can return a nil pointer.

 

I checked the GetMem, AllocMem, GetMemory... and they all say that raise exceptions. Although I looked to the GetMemory code but I didn't see any code that raise exceptions.

[David Heffernan 2353]

What do you want to happen if memory allocation fails? Isn't process termination actually the best thing? 

 

When you say string, you can't expect to do this for built in string types since you don't control how they are allocated. 

Edited May 10, 2023 by David Heffernan

[Marus 0]

If memory allocation fails it can return a "nil" pointer... All Windows APIs returns a False boolean when they fail. Why memory allocation cannot work the same ?

[programmerdelphi2k 237]

because the API take care before send result boolean

be using TRY_EXCEPT (Delphi) or CATCH (C), for that you dont see it.

 

try_block,  dont costs almost nothing, and it was made for this.  it is up to you to determine what will be returned in case of an exception and what actions will be taken.

[David Heffernan 2353]

Because when memory allocation fails it's very hard to recover. 

 

Also, you don't control the allocation for strings. The rtl does that. You'd need to redesign you entire program to work with low memory scenarios. The entire rtl/vcl etc. is not designed to work in these scenarios. 

[David Schwartz 430]

9 hours ago, Marus said:

I need a safe way to allocate memory for a string variable, without getting exceptions. I don't want to use "try-except" blocks and neither the function that do this, use them internaly. I need a speed optimised method. If there are errors it can return a nil pointer.

 

I checked the GetMem, AllocMem, GetMemory... and they all say that raise exceptions. Although I looked to the GetMemory code but I didn't see any code that raise exceptions.

Simple -- build your own memory manager, allocate a big chunk of memory to it, then carve out pieces for your strings. Just be forewarned that as you allocate and free blocks of space, you're going to end up with something called "internal fragmentation" that will ultimately limit the maximum size of strings you can allocate. To fix that, you'll need to build what's called a "garbage collector" that runs periodically to move strings around and compress out the unused blocks in between the blocks that are still being used. But the more blocks that have been allocated and are still being used, then the fewer available free blocks you have. Your program will end up spending more and more time doing garbage collection.

 

And when you get to the point where there simply isn't enough space to allocate a new string ... what exactly do you propose to do? Just send back a False flag and pretend the request wasn't made?

[Kazantsev Alexey 24]

11 hours ago, Marus said:

I need a safe way to allocate memory for a string variable, without getting exceptions. I don't want to use "try-except" blocks and neither the function that do this, use them internaly. I need a speed optimised method. If there are errors it can return a nil pointer.

 

I checked the GetMem, AllocMem, GetMemory... and they all say that raise exceptions. Although I looked to the GetMemory code but I didn't see any code that raise exceptions. 

You can use SysGetMem, SysReallocMem, SysFreeMem.

[Dalija Prasnikar 1404]

11 hours ago, Marus said:

I need a safe way to allocate memory for a string variable, without getting exceptions. I don't want to use "try-except" blocks and neither the function that do this, use them internaly. I need a speed optimised method. If there are errors it can return a nil pointer.

What exactly are you trying to do? This sounds like XY problem. See https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem

 

Out of memory exception is recoverable if your application logic allows that at the moment it happens. For instance if there is enough free memory to do regular work, but there is not enough to load and process some large file user tries to open. This is why exception handlers are here for.

 

So if you can logically recover from failing to allocate large string, you can just do the same using exception handlers. There will be no memory corruption when string allocation fails that will have negative effect on your application. While exception handling does have some small impact on performance, I seriously doubt that this will be your main problem comparing to all you would have to do to get even close to functionality you want to achieve. Delphi applications are full of exception handling (both direct and indirect) on all levels and applications are still running fast. For instance, merely declaring a string variable will insert hidden exception handling code around that string.

[Marus 0]

Ok, I understand... Doing what I want will complicate things more that a `try - except` block. I don't want to build another memory manager just to allocate a string variable, so I think I will use `try New() / Dispose() except`. I built a complex function using mainly APIs that do not raise exceptions, and my code is free of "try"s. And adding one just for allocating a variable it ruins my art.  I read once in Delphi documentation that "try"s must be used only they are really neede, because of performance impact, and since then I try to avoid them.

 

Indeed, the X behind the Y problem is like this:

I want to send a message, from worker thread to the main thread queue, with a string in it. Because the message will be sent using PostMessage, which is asynchronous, I cannot send the string pointer (like in SendMessage), because this is not thread safe. So I need to allocate a string variable just for this message only, then assign the value and send the message with the string pointer. The main thread will receive the message, reads the string, and Dispose the pointer.

 

 

* By the way, Dalija, you are one of the most beautiful being I ever seen ! And that's not a compliment, it's a fact. I've known you for a long time from StackOverflow. You have helped me many times there too. I have wondered many times who is hiding behind that profile picture, and now after seeing the video on your website... it was wow ! You are a verry special person with something that shines from within... Sorry, I couldn't help myself. 

[David Heffernan 2353]

If memory can't be allocated then the message can't be sent and presumably the program stops working? 

 

I think your model of avoiding try is probably very flawed. You are about to call PostMessage which has a kernel mode transition. Your try is not the hot-spot in this code! 

[Stefan Glienke 2019]

54 minutes ago, Marus said:

just to allocate a string variable

If it's just a string variable you don't need to heap alloc anything - either use a const if you already know the exact content or use a preallocated buffer.

[Dalija Prasnikar 1404]

2 hours ago, Marus said:

I want to send a message, from worker thread to the main thread queue, with a string in it.

Possible solutions in your scenario depends on what kind of string are you sending. Is it a predefined constant, or it is provided by the code in a thread, do you already have it in a string variable and just need a copy for posting., how large are those strings, is the data in string critical or is it just some discardable notification, what is your logic if there is not enough memory...

 

Thanks for the compliments.

[Marus 0]

The string is a file path that is provided by the code (scanning directories, it's a backup app), so it cannot be verry long. Yes, I have it in a variable ready for posting. The data is not critical, it just show in the main form the file that is processed. If there is not enough memory or encounter other unrecoverable errors (because there are many reasons to fail) the thread stops and return the error.

 

The idea was for the worker thread not to wait for the message to be processed and to continue its work. That's why I wanted to use PostMessage. As otherwise, there would have been no problem with SendMessage...

[David Heffernan 2353]

This is the only place in your program where you use string variables? And you don't use any classes? 

[Fr0sT.Brutal 900]

Avoiding tries is weird style.

Avoiding tries because of some perf impact you've once read about somewhere is much weirder.

The really time critical parts should avoid tries indeed but that starts to play difference for millions ops/sec. My advice is to give up bothering about these nano things. If you really strive to achieve visually try-less code, just wrap AllocMem in try-except and hide the function somewhere deeply in utils unit. Anyway RTL has lots of tries inside

 

Edited May 11, 2023 by Fr0sT.Brutal

[programmerdelphi2k 237]

why not use a "pattern singleton" to read/write your message/or list-of-message-strings, then you managed in all app (same that GLOBAL is not necessary so good, but sometime it's used, since protected when writing) = CriticalSession or anyother

---> mySingleton.add( who-sends, text-msg...)  --> my Singleton.read(, who-sent, msg-index-or-any-other-way-if-exists-any-one)    ... who read, can exclude it from list

Edited May 11, 2023 by programmerdelphi2k

[DelphiUdIT 187]

This may be a bit OT, but if you want to avoid try .. and maybe even single windows api testing, you can use this feature (I've been using it for years):

type
  TFMain = class(TForm)
     .......
  protected
    procedure ErroriRunTimeNonGestiti(Sender: TObject; E: Exception);
     .......
end;

procedure TFMain.ErroriRunTimeNonGestiti(Sender: TObject; E: Exception);
begin
  .... do something;
end;

  // In the form create event you can insert:

  Application.OnException := ErroriRunTimeNonGestiti;
  //Handle critical system errors (caused by the application)
  //By default critical errors are handled directly by Windows
  SetErrorMode(SEM_FAILCRITICALERRORS OR SEM_NOGPFAULTERRORBOX);

 

Obviously this occurs if you don't use the try ... except.ì, i.e. for all the exceptions that your program generates and are not managed by it.

The fact that you assume that an allocation error is reported to you by the operating system is misleading: if the memory allocation fails (at least for a string allocation) it is likely that your program will not notice it because it probably "won't run", the operating system will probably have notified you on screen that there is a "lack of resources" and will "hang".

 

[Virgo 18]

Lets not forgot overcommit . On some platforms allocating memory from OS can succeed, but later writing that memory can fail, because no memory is available.

[Stefan Glienke 2019]

5 hours ago, Marus said:

If there is not enough memory or encounter other unrecoverable errors (because there are many reasons to fail) the thread stops and return the error.

So you are saying that your app is already using more than 2GB (or more, large address aware and all that) and might not have enough space for allocating a simple string to store a file path in... But at the same time you are talking about using a form and showing information there... and where do you think the VCL (I am assuming you are using the VCL and not creating your form by directly accessing the winapi) takes its memory from?

And worrying about the performance impact of try while scanning directories on the file system?

Sorry, but this all makes absolutely no sense.

Edited May 11, 2023 by Stefan Glienke

[David Heffernan 2353]

Yeah, just try setting a break point on the GetMem function implementation and see how many times it gets hit. 

 

I ask again. Does you program use any string variables? Does it instantiate any classes? What is you policy for these heap allocations? 

 

Why are you creating problems where there are none?