appending to a dynamic array

[dummzeuch 1505]

Given an array

MyArr = array of TSomeRec

Appending to that array means:

1. Increase the length
2. Set the values for the new record

 

I have seen code like this:

SetLength(MyArr, Length(MyArr) + 1);
MyArr[High(MyArr)].SomeField := SomeValue;
// and repeated for each field in TSomeRec

This always triggers my bad smell sense because I would have coded it like this:

var
  Len: integer;
 
// [...]

Len := Length(MyArr);
SetLength(MyArr, Len + 1);
MyArr[Len].SomeField := SomeValue;
// and repeated for each field in TSomeRec

Am I just too pedantic?

(Please ignore for now that this might lead to memory fragmentation in both cases.)

[uligerhardt 18]

I use a "len" variable too. So I'd say you show exactly the right level of pedantry. 

[Uwe Raabe 2057]

In modern Delphi versions I try to combine record constructors and array concatenation to achieve that:

program Project481;

{$APPTYPE CONSOLE}

type
  TSomeRec = record
    SomeField: Integer;
  public
    constructor Create(ASomeField: Integer);
  end;

  TMyArr = TArray<TSomeRec>;

constructor TSomeRec.Create(ASomeField: Integer);
begin
  SomeField := ASomeField;
end;

procedure Main;
var
  myArr: TMyArr;
  myRec: TSomeRec;
begin
  myArr := [TSomeRec.Create(0), TSomeRec.Create(1)];
  myArr := myArr + [TSomeRec.Create(2)];
  Insert([TSomeRec.Create(3)], myArr, 1);
  for myRec in myArr do begin
    Writeln(myRec.SomeField);
  end;
  Readln;
end;

begin
  Main;
end.

 

[Der schöne Günther 316]

Why would you consider it bad?

• The arguments are evaluated before calling SetLength(..), so Length(..) returns the length before it was resized
• High(someArray) is a simple lookup, the size is known

[Uwe Raabe 2057]

3 hours ago, dummzeuch said:

Am I just too pedantic?

Talking about pedantic: The name len is probably misleading, as when it is used in MyArr[Len] it doesn't represent the actual length of the array any more. In contrast MyArr[High(MyArr)] is always correct.

 

Nevertheless, caching the index might be a good idea anyway: It eliminates some checks - and High(x) needs one additional DEC operation compared to Length(x).

[hsauro 40]

11 hours ago, Uwe Raabe said:

In modern Delphi versions I try to combine record constructors and array concatenation to achieve that:

program Project481;

{$APPTYPE CONSOLE}

type
  TSomeRec = record
    SomeField: Integer;
  public
    constructor Create(ASomeField: Integer);
  end;

  TMyArr = TArray<TSomeRec>;

constructor TSomeRec.Create(ASomeField: Integer);
begin
  SomeField := ASomeField;
end;

procedure Main;
var
  myArr: TMyArr;
  myRec: TSomeRec;
begin
  myArr := [TSomeRec.Create(0), TSomeRec.Create(1)];
  myArr := myArr + [TSomeRec.Create(2)];
  Insert([TSomeRec.Create(3)], myArr, 1);
  for myRec in myArr do begin
    Writeln(myRec.SomeField);
  end;
  Readln;
end;

begin
  Main;
end.

 

 

Using record constructors like this is very convenient but I wonder what the speed penalties are like?

[Uwe Raabe 2057]

3 hours ago, hsauro said:

Using record constructors like this is very convenient but I wonder what the speed penalties are like?

Actually, I don't know and, to be honest, I don't care much. Usually there are other places slowing down a program and the gains from rethinking an algorithm or an architecture are probably better targets.

[Rollo62 536]

Low/High shoukd work well, at least they are considered to take out the 0-based / 1-based questions in Strings.
Personally I prefer to use Length too, because I like to have something where I know what to expect, instead of "hidden magic".

What I always wanted to know is: Are there any performance differences between High/Low and Length solutions ?

[Uwe Raabe 2057]

46 minutes ago, Rollo62 said:

Are there any performance differences between High/Low and Length solutions ?

High(x) is implemented as Dec(Length(x)), while Low(x) can already be evaluated by the compiler. High and Length have some execution overhead to check for a nil array, so caching the value might be worth it.

[Rollo62 536]

@Uwe Raabe thanks for the info. So I can stay with Length, and hope that EMBT doesn't change arrays to 1-based, one sunny day

[Stefan Glienke 2002]

What's also interesting about Length and High is that if you are just writing code that directly compiles into a binary they are being inlined by the compiler whereas if you are using them in units that are pre-compiled as part of a package (if you are 3rd party component vendor for example) they are not and always cause a call to System._DynArrayHigh and System._DynArrayLength. Funny enough this is not the case for the source shipped with Delphi because their DCUs are not generated from their containing packages.

[Rollo62 536]

Sorry, its just a little off-topic, but I think fits well to this discussion ...

As always in life: by chance I just got the case that I wanted to change the 1. char in a string.

I stumbled into that this 0-based method didn't work, since .Chars is read only

LStr.Chars[0] := LStr.Chars[0].ToLower;

So I have to fall back to the High/Low solution here

LStr[Low(LStr)] := LStr[Low(LStr)].ToLower;

Is this right, that TStringHelper for 0-based strings doesn't offer any solution to replace single characters, beside the Low/High solution ?

This is a little disappointing, because I was changing new code to 0-based StringHelper but now it seems that there is a solution missing
for single-char handling.

The TStringHelper function which I expected to solve this, TStringHelper.Replace(), does seems to have a different purpose.

So I think I still have to rely on Low/High for a while, instead of moving to a clear, common 0-based handling for all cases.

 

 

 

 

Edited January 28, 2019 by Rollo62

[Der schöne Günther 316]

The TStringHelper was designed for immutable strings. If I'm not mistaken, Strings on platforms such as iOS are immutable whereas they are still "copy on write" on Windows. All the TStringHelper methods return a new string instance.

[Stefan Glienke 2002]

Turn ZBS off for that code and use 1.

[Rollo62 536]

@Der schöne Günther

Yes immutable strings are fine, if I want to use them.
But it seems that then the only possible to re-build the whole string, if I only want to change one character.
 

@Stefan Glienke

ZBS = OFF would be a hard choice, which I want to avoid, it would be not necessary just for my simple, single case now.
The Low/High solution works fine,
but I'm aware now that removing all 1-based strings causes probably some unexpected performance issues.

[PeterBelow 238]

Via TStringHelper you have to first convert the string into an array of char, modify that, and then convert the array into a new string.

 

var
  LChars: TArray<char>;
  LStr: string;
begin
  LStr := StringOfChar('A', 20);
  LChars := LStr.ToCharArray;
  LChars[Low(LChars)+1] := LChars[Low(LChars)+1].ToLower;
  LStr := String.Create(LChars);

 

[hsauro 40]

Slightly off topic I know but what what is the rational for having immutable strings in languages such a Java, Python etc?

 

It seems immutability is related to efficiency and security. I didn’t find the arguments that compelling however. It seems it’s either to stop the programmer from getting into trouble or it makes like easier for the complier writer.

Edited January 28, 2019 by hsauro

[PeterBelow 238]

27 minutes ago, hsauro said:

Slightly off topic I know but what what is the rational for having immutable strings in languages such a Java, Python etc?

 

It seems immutability is related to efficiency and security. I didn’t find the arguments that compelling however. It seems it’s either to stop the programmer from getting into trouble or it makes like easier for the complier writer.

It makes automatic memory management via garbage collection easier to implement and more efficient as far as I know.

[Stefan Glienke 2002]

https://stackoverflow.com/questions/22397861/why-is-string-immutable-in-java

https://stackoverflow.com/questions/2365272/why-net-string-is-immutable

[Rollo62 536]

@PeterBelow

That looks alittle to much copying for simple task,  from my point of view.

Ok, its possible, but i See no advantage beside thats low is still there.

The current string is Not immutable in my case, but a subresult from a former function call,

where i want to modify the 1. Char in a second step.. 

 

[Микола Петрівський 10]

19 hours ago, Stefan Glienke said:

https://stackoverflow.com/questions/22397861/why-is-string-immutable-in-java

https://stackoverflow.com/questions/2365272/why-net-string-is-immutable

Second link counts COW as immutable. While in Delphi community we usually treat it as attribute of mutable strings.

[Stefan Glienke 2002]

5 minutes ago, Микола Петрівський said:

Second link counts COW as immutable. While in Delphi community we usually treat it as attribute of mutable strings.

Technically in Delphi strings are only mutable if they have a ref-count of 1, otherwise a copy happens.

And while it might be convenient to the average developer if you actually look under the hood how much instructions and implicit heap allocations happen if you are not paying attention and naively work with strings you will be scared.

[Rudy Velthuis 91]

On 1/24/2019 at 2:41 PM, dummzeuch said:

Given an array

MyArr = array of TSomeRec

Appending to that array means:

1. Increase the length
2. Set the values for the new record

 

I have seen code like this:

SetLength(MyArr, Length(MyArr) + 1);
MyArr[High(MyArr)].SomeField := SomeValue;
// and repeated for each field in TSomeRec

This always triggers my bad smell sense because I would have coded it like this:

var
  Len: integer;
 
// [...]

Len := Length(MyArr);
SetLength(MyArr, Len + 1);
MyArr[Len].SomeField := SomeValue;
// and repeated for each field in TSomeRec

Am I just too pedantic?

(Please ignore for now that this might lead to memory fragmentation in both cases.)

It is never a good idea to increment the size by one. See this blog post: Extending arrays

 

But otherwise: yes, you are being too pedantic.  If this is in a tight loop, you might think of profiling which is faster, otherwise, it doesn't really matter.

 

FWIW, I would do it like this (after Delphi XE7, IIRC):

NewRecord.SomeField := SomeValue;
// etc...
MyArray := MyArray + [NewRecord];

No need for SetLength, Len or High

Edited February 1, 2019 by Rudy Velthuis

[Remy Lebeau 1393]

On 1/24/2019 at 5:41 AM, dummzeuch said:

I would have coded it like this:

var
  Len: integer;
 
// [...]

Len := Length(MyArr);
SetLength(MyArr, Len + 1);
MyArr[Len].SomeField := SomeValue;
// and repeated for each field in TSomeRec

 

I would probably take it a step further, to avoid repeatedly indexing into the array:

var
  Idx: integer;
  Rec: ^TSomeRec;

// [...]

Idx := CountInMyArr;
SetLength(MyArr, Idx + SomeDelta);
Rec := @MyArr[Idx];
Rec.SomeField := SomeValue;
// and repeated for each field in TSomeRec
Inc(CountInMyArr);

 

Edited February 7, 2019 by Remy Lebeau

[Rudy Velthuis 91]

On 1/25/2019 at 2:45 PM, Uwe Raabe said:

High(x) is implemented as Dec(Length(x)), while Low(x) can already be evaluated by the compiler. High and Length have some execution overhead to check for a nil array, so caching the value might be worth it.

Actually, High(x) is not always implemented as Pred(Length(x)). In routines with open array parameters, Length(x) is actually implemented as Succ(High(x)), because the High value is passed, not the Length value.

 

But you are right about checking for nil and caching the value.