Releasing memory devoted to arrays

[Gary Mugford 12]

I have a small project in XE7, which is newish to me, where I have to get rid of an Access Violation error on exiting the application. From the error log, I know something is amiss vis-a-vis the arrays I'm trying to clean up.

 

I have one global constant static arrays of strings, two global static arrays of extended and two global static arrays of strings.

 

I have tried to use the setLength method for the cleaning up of the arrays, the finalize method, freeAndNil, free and setting the arrays to nil. My attempts sometimes survive compiling, sometimes don't. Researching the issue on the internet seems mostly to focus on dynamic array clearing. Not much about static arrays and not a solution that I could find. I've tried placing the clearing attempts in OnCloseQuery and disastrously in OnClose. At this point, I'm stumped.

 

Is there a definitive way to do this? GM

 

My creation code in the main form looks like this:

 

var
  ...
  ar1: Array [1..100] of string;
  ar1Total: Array [1..100] of extended;
  ar2: Array [1..100] of string;
  ar2Total: Array [1..100] of extended;

const
  TonerColors: TArray<string> = ['Cyan', 'Magenta', 'Yellow','Black'];

 

[Uwe Raabe 2057]

Static arrays are never freed. That's why they are called static. They live as long as their scope lives.

[Gary Mugford 12]

Uwe,

 

That had been my supposition in the past. But I get an Access Viol error on program exit and when I look at the stack that trips the error (admittedly a presumption since many errors are actually on the invisible next line, so to speak), I see that the program goes into the system for Halt(), FinalizeUnits, my main form's Finalization, _FinalizeArray and lastly _UStrArrayClr. It trips whether I do anything with the arrays or not. Looking at the assembler version of _UStrArrayClr, there's a MOV line that trips an exception, followed by a DEC and JL instruction and then the ominous warning that the next line is an unaccessible location.

 

Sorry to bring such a newbie question to the forum ... which I've found entertaining, be it at a higher level than I am at. Thanks for helping, GM

[Cristian Peța 103]

The code you posted can't give any exception. Try for yourself in a new project.

And us Uwe Raabe said: static arrays can't be freed manually, nor modify the length. Don't do this.

 

P.S. for string static arrays you can free memory allocated by strings (assign null string or modify length) but not the array itself.

Edited November 27, 2018 by Cristian Peța

[Gary Mugford 12]

Cristian,

 

  I was aware that the code did not represent the full code. Several pages worth. The AV occurs on program exit. From looking at the Error Log, I came to the conclusion my mistake was in not clearing out the memory assigned to arrays. I presented my declaration of the arrays in case I was doing that wrong. As for NOT DOING the things I tried, I ruefully am aware that what I was doing was not working.

 

  All that said, it appears something else, other than array finalization is creating the AV. More than possible. As a programmer, I'm self-taught, mostly through examples, it is very possible bordering on probable I've done something careless and that's the source of the AV, Error Log notwithstanding. In my original post, I did point out that the error is frequently the NEXT THING, rather than just what passed, in causing program faults. It does appear that is the case here and I've wasted everybody's time focusing in on the arrays.

 

  Thanks for moving me in that direction. GM

[Kryvich 165]

A demo program that demonstrates the bug would help the community to help you.

[Stefan Glienke 2002]

If _UStrArrayClr is giving you an AV I am putting all my money on passing something wrong somewhere and destroying it's reference counter causing a premature FreeMem or something similar.

Edited November 27, 2018 by Stefan Glienke

[ertank 27]

You might try to enable range checking and see if that helps.

 

It is possible somewhere in your code is writing a memory are that it shouldn't.

[David Schwartz 426]

My experience with the FastMM and the additional diagnostic tools it offers (you need to download the source and use it in your program) tells me that most of the time memory errors are like landmines planted way earlier, and when they blow up it can be quite random as to who or what triggers them.

 

Here's an analogy:

 

Using static arrays is like building a house. One simply does not call a friend to increase or decrease the size of a room with a quick refresh.

 

If something is blowing up in a room, it's because a previous tenant did something he should not have done and left a time-bomb -- not the current tenant.

 

You can stare at the current tenant all you want, you're not likely to find the error.

Something else is crapping over some memory unexpectedly. That's about all we know right now.

[David Heffernan 2345]

Heap corruption in code we can't see. Make a minimal reproduction. 

[Gary Mugford 12]

I appreciate each and every suggestion above. The problem with the program is that it's proprietary work and the sum total of the code is several pages. That said, I am looking very much at my use of the arrays. Not surprisingly, each of the two Var arrays are linked, i.e. one being a Dept code, the other the total for that Dept. BUT, I am not NEARLY using the full array. In fact, each run through of the program captures a count of the Depts and I use that to totalize up the Depts. BUT there are only a handful of Depts currently involved. I chose 100 for the limit to protect my backside against successful expansion of this project.

 

I am NOT initializing the arrays with a loop to '' and 0.00 respectively. Don't see a need. I never go PAST the DeptCount, so I'm not accessing unassigned memory. At least I'm PRETTY SURE that's the case. I AM force capturing the INDEX for the particular pairs. See this:

function TFrmMain.ReturnDeptIndex(ADept:string): integer;
var
 idx: integer;
begin
  Result := 0;
  for idx := 1 to DeptCount do begin
    if ADept = arDepts[idx]
      then begin
        Result := idx;
        break;
        end;
    end;
end;

I assumed there was a direct way to get the index in XE7 but couldn't find any. Thus the kludge above. It's pretty representative of my lack of elegance when it comes to programming. That said, I don't see anywhere where this can induce an error accessing memory it shouldn't. Slow, but successful. Besides, EVEN IF I DO NOT CLICK THE BUTTON that does the analysis, even to just starting the programming and immediately exiting, it throws out the AV.

 

At the top of the function, I include these lines:

var
  tmpIdx: integer;
  kounter: integer;
begin
  for kounter := 1 to DeptCount do arDeptsTotal[Kounter] := 0.00;
  ...

The function is later called via these lines:

tmpIdx := ReturnDeptIndex(TheDept);
arDeptsTotal[tmpIdx] := arDeptsTotal[tmpIdx] + LineVal;

Again, I'm missing what could be wrong with those lines.

 

I do create a fair number of SQL queries on the fly. There's a bunch of filterable fields, a date range, a results sort picker and a flag check box to include Zero Value lines. It's more "get it to work" code than incredibly featured code. But it runs on for a fair bit. Plus, I couldn't get my boss to give me permission to spend the time and effort to create the stripped down version that all of you are requesting (and reasonably at that).

 

At this very moment, the decision has been to start using the alpha, AV and all. It's an analysis tool rather than a data manipulator. So, the boss is happy to have something that gives him desired info. I'm still changing things around, literally looking at every procedure and function and seeing if I can re-write them and maybe expose the AV but tearing up the hidey-hole. But you can't be expected to do my work for me. So, I am respectfully going to ask that we call a halt to this thread since I'm wasting your time. I appreciate all who contributed. You've give me directions to go on debugging this. And I will explore each suggestion.

 

Thanks, GM

 

Edited November 28, 2018 by Gary Mugford
Forgot to include showing that I DO initialize the totals array elements

[Gary Mugford 12]

One more thing ... I left StackOverflow because help seemed to come at a very high price there. I've lurked since. My improvement as a programmer has stagnated because of that. MY FAULT. I should have been less sensitive, better at learning on my own. I found Praxis here and read through the threads, some of which were contentious. But I really appreciated the tone of those dialogs. Help seemed the raison d'etre for the site. And my request for help here has been answered with that same tone of voice. Asking to end the thread seems disrespectful towards the efforts already put forth. Please understand that that is not my purpose. I simply think the process moving forward would waste more of the collective time of the people trying to help. I do not wish to do that, nor insult anybody. THANK YOU for trying to help. I'm simply ill-equipped to aid in the helping of myself. GM

[David Heffernan 2345]

This problem could readily be solved if you let us help you learn how to debug it. 

[Kryvich 165]

First of all, you need to set your build configuration to Debug (Projects view | Build Configurations - double click Debug), and enable debugging options (menu Project | Options | Building | Delphi Compiler | Compiling):

• Optimization: False
• Stack Frames: True
• Debugging: all settings to True
• Runtime errors: all 3 settings to True

Perhaps after that this error will appear earlier than when you close the application.

 

Usually you do not need to initialize arrays - they are initialized with zeros automatically. How the arDeptsTotal array is declared? Is it 0-based or 1-based? 

 

Based on the code snippets you provided, the TList<TYourData> or TDictionary<string, TYourData> class might be more appropriate for you. But the good old array should works too.

Edited November 28, 2018 by Kryvich

[Gary Mugford 12]

Mr. Heffernan,

 

  Offers to help should always be welcome and they are here. Debugging this is obviously beyond my skill set at this point. I'm 'Changing things around' hoping to discover where my AV is ACTUALLY percolating, only to flare up at the end of the program. I'm doing it one change at a time, looking for the Eureka moment. I've started in places that look different to me from my decades of using D7. XE7 is a new world. Even the declaration of the one constant array was subject to researching doing so on the internet since I wanted to use variables as the end/high element in the var declarations. I was hoping for the variable arrays, for example, to be arDepts: Arrays [1..DeptCount] of string. It was there, as a side process that I found a different way of declaring the constant array. Interesting and easier to read. BUT, I wasn't going to be able to get the arDepts declared as tightly as I wanted. Ergo, make it 100 and that covers expansion for the rest of my life in the number of departments. It's currently TWO for this project. Eight if it goes company wide.

 

  Now, posting function by function here would resolve the curiosity AND the appreciated desire to help that folks here have. It would also be against the explicit instructions I got from my employer NOT to do that. Genericizing the material would suffice to some extent, but again, the Boss said no. He needs me to produce tools in a fast-changing environment that we are currently in. My time is HIS time to do with as he sees fit. Producing a 'simple' case would presume I know generally where things are tripping me up. It's the end of the program. Beyond that ... nothing? Creating a stripped out version of the program would require replacing 3rd party components that I can't publish (with GExperts), eliminating items that would specify my employer and do it all while MAINTAINING where the bug's born. I apologize, but I can't do that. Puttering around with MY spare time seems my best compromise because wasting YOUR time and the time of the others is disrespectful.

 

  I hope you understand. Thanks, GM

[Gary Mugford 12]

2 minutes ago, Kryvich said:

First of all, you need to set your build configuration to Debug (Projects view | Build Configurations - double click Debug), and enable debugging options (menu Project | Options | Building | Delphi Compiler | Compiling):

• Optimization: False
• Stack Frames: True
• Debugging: all settings to True
• Runtime errors: all 3 settings to True

Perhaps after that this error will appear earlier than when you close the application.

 

Usually you do not need to initialize arrays - they are initialized with zeros automatically. How the arDeptsTotal array is declared? Is it 0-based or 1-based? 

 

Based on the code snippets you provided, the TList<TYourData> class might be more appropriate for you. But the good old array should works too.

Kryvich,

 

  I did set the Optimization to false and turned on range-checking due to earlier contributions. I will further augment my debugging environment as you have suggest. Thank you.

 

  My arrays are one-based for convenience sake. Each array is set high at the count of those depts (or tasks in the other pairing). I have snooped around Lists a bit, going back to Orpheus days. The modern-day implementation seems much more to the point. But as you say, the good old array does get the work done. Identifying a LIST element's index through the string value would be nice, though.

 

  GM

[Kryvich 165]

If you decide to go with TDictionary<string, TYourData>, here is examples how to create, initialize, use and free it: http://docwiki.embarcadero.com/CodeExamples/Tokyo/en/Talk:Generics_Collections_TDictionary_(Delphi).

[David Heffernan 2345]

It is my experience that ignoring such defects in your code in the interests of saving time costs you more time in the medium term. 

 

It is my experience that putting off investing in learning debugging skills in the interests of saving time costs you more time in the medium term. 

 

I understand that your boss might not appreciate that. 

[Attila Kovacs 629]

I'm not sure what are you asking and how should we help blindfolded, but I would do these in first place:

 

move your declarations out to a separate unit if you can, if not, move it somewhere else inside the current unit and check if the same AV raises

(eventually declare dummy vars between the arrays)

turn on range and overflow checking (1..100 could be tricky)

check your class destructors if you have, they are processed at finalization

place a breakpoint at the "end." in your dpr and start stepping through and try to understand what happens

 

Edited November 28, 2018 by Attila Kovacs

[Gary Mugford 12]

Attila,

 

  As I've said, I'm hamstrung by employer instructions NOT to post code. As it is, I'm skating on thin ice. I appreciate your frustration. I mirror it.

 

To everybody,

 

  There are some new pieces of information. The Range-Check did, in fact, reveal an error that contributed to an AV UPON exit of the program without doing anything. Turns out, one of the minions had entered a Dept name WITH A RETURN attached ... obviously through copying and pasting ... and THAT was being read in the form creation and it was kicking out the AV on ending, sending me on a wild goose chase for the AV there. So, I fixed the code NOT to just over-ride bad departments and keep on going through a Try ... finally block that I never checked to see if it failed because it couldn't ... until it did. I now get an error message if another minion decides creativity can be fun. When I now run the program, I could exit without failure, having fixed the record with the bad dept in it. Happiness?

 

  No. I STILL had an AV if I ran the analysis BEFORE I SHUT DOWN. I looked through that button's code for places where things went south. I looked at the assignation of string grid cells looking at a FloatToStr conversion on the run. I commented it out. No fix. I commented out the whole string grid cell assignation code. No fix. I commented out each little part until I came to:

    VPP := 0;
    VPP := fieldByName('Parm1').AsFloat * fieldByName('ParmBV').AsFloat / 3;

  Comment out the second line and everything runs without AV. I changed EACH part to be 1 and EACH time I ran the program and ran the analysis, I got an AV.  VPP is a declared Extended variable. The range of values for Parm1 can be in the 0.00002 to 20 area. The second field runs 200-500. Each can go into five decimal values. And the last part is a static integer. The results are maxed out at about 30K. That's well within the scope of Extended, is it not? Even the temporary 100K which is borderline possible is within scope, isn't it? I DO convert the result to a string later JUST with the FloatToStr function. For sticking into a string grid.

 

  EVEN with the closing AV, the RESULTS of this line leads to correct results in the string grid. I've manually checked EVERY SINGLE cell.

 

  The more I know, the less I understand. And it doesn't help that I have to wait two minutes between compiles or I get the dreaded fatal error F2039 Count not create output file.

 

  So, I HAD two AVs although only one would show at Program exit. Now I'm down to one. And it's with basic math. That actually works.

 

  Dare I ask for ideas in a thread I've tried to shut down twice? Sure, why not?

 

  THanks to everybody for their patience and willingness to help. GM

[Kryvich 165]

@Gary Mugford The assignment "VPP := 0;" right before the second assignment you can delete. Actually the compiler must generate a hint: H2077 Value assigned to 'VPP' never used. Check the Messages view after compilation: there may be other hints and warnings.

You can ensure that the fields have proper values by inserting assertions right before the assignment:

  Assert((fieldByName('Parm1').AsFloat >= 0.00002)
    and (fieldByName('Parm1').AsFloat <= 20), 'Bad Parm1 value: '
    + fieldByName('Parm1').AsString);
  Assert((fieldByName('ParmBV').AsFloat >= 200)
    and (fieldByName('ParmBV').AsFloat <= 500), 'Bad ParmBV value: '
    + fieldByName('ParmBV').AsString);

I myself often use assertions at the beginning of a method to make sure that input parameters are correct. You can leave them in the code, because assertions usually do not affect performance of an application.

[Alexander Elagin 143]

What if those fields (TField instances) belong to some already destroyed component? Then the FieldByName references may point to some garbage. Or there are simply no fields with such names, then FieldByName returns nil and AsFloat produces an AV. Just guessing.

Edited November 29, 2018 by Alexander Elagin

[zinpub 0]

10 minutes ago, Alexander Elagin said:

What if those fields (TField instances) belong to some already destroyed component? Then the FieldByName references may point to some garbage. Or there are simply no fields with such names, then FieldByName returns nil and AsFloat produces an AV. Just guessing.

FieldByName raise Exception if there are no fields with such names. FindField return nil

[Gary Mugford 12]

I'd already ditched the assignment to a default value of zero. It's an old habit of mine to set some default for a variable before starting some formula work.

3 minutes ago, Kryvich said:

Assert((fieldByName('Parm1').AsFloat >= 0.00002) and (fieldByName('Parm1').AsFloat <= 20), 'Bad Parm1 value: ' + fieldByName('Parm1').AsString); Assert((fieldByName('ParmBV').AsFloat >= 200) and (fieldByName('ParmBV').AsFloat <= 500), 'Bad ParmBV value: ' + fieldByName('ParmBV').AsString);

So adding the assertions didn't change things, other than show no records failed the assertion. BUT adding explaining vars to turn the arrays with the lookup function to determine index DID. I've subsequently ran every filter, data range, data sort and flag for showing zero value records that there is. All sequentially within the same run. NO AV.

 

THANK YOU ALL. Your patience has been rewarded. I have my app, and a meeting with the boss in a half-hour for the next task.

[Gary Mugford 12]

Oh, and bonus, after using the explaining vars to ... slow down the app??? ... compiling didn't throw one of the F2039 errors during about 10 more compiles. Any changes I made before compiling were mostly a LOT less than 2 minutes apart. So, wins all round. Here's the code I used to either slow things down or actually cure the issue:

P1Value := fieldByName('Parm1').AsFloat;
P2Val := fieldByName('Parm2').AsFloat;
// Assert check to see if each value is actually within range ...
Assert((P1Value >= 0.00002) and (P1Value <= 20), 'Bad P1Value: ' + fieldByName('Parm1').AsString);
Assert((P2Val >= 200) and (P2Val <= 500), 'Bad P2Val: ' + fieldByName('Parm2').AsString);
VPP := P1Value * P2Val / 3;

When I look at it, it makes no sense to me that this fixes the AV. I admit, repeatedly calling the AsFloat function for the same field contents in a single line is not good. But the assertion NEVER triggered (as I expected) and changing the VPP assignment to use actual numbers rather than functions works, although I don't know why. It worked the original way to PRODUCE the data. But it triggered that exiting AV. I guess we call it a win for Assertions and refactoring, however small.

 

Thanks Kryvich and the rest of you. GM