Dynamic Arrays, TBytes and Concat performance question

[Rollo62 616]

Hi there,

I am unsure about the Concat behaviour for dynamic arrays, for modern Delphi only.
There is an old SO post from Remy, but this is 14 Years old and might not be the best choice any more, I assume.
https://stackoverflow.com/questions/7520068/best-way-to-combine-multiple-tbytes-arrays
 

var
  arr1, arr2, merged: TBytes;
begin
  ...
  SetLength(merged, Length(arr1) + Length(arr2));
  if arr1 <> nil then Move(arr1[0], merged[0], Length(arr1));
  if arr2 <> nil then Move(arr2[0], merged[Length(arr1)], Length(arr2));
end;

With the Concat for modern Delphi, this should be reduced to this line probably

https://docwiki.embarcadero.com/Libraries/Athens/en/System.Concat

A := Concat([1,2,3],[4,5,6]); //A will become [1,2,3,4,5,6]

 

The question is, if Concat does this already in an perfect, optimized way, or if this might still be better:

function AppendTo( var ADst : TBytes; const AAppend : TBytes ) : Integer;
var 
    LSrcLen, LAddLen : Integer;
begin
  LSrcLen := Length(ADst);
  LAddLen := Length(AAppend);
  
  if LAddLen = 0 then Exit(LSrcLen);

  SetLength( ADst, LSrcLen + LAddLen );     // SetLength ensure that content is unchanged, may or may not copy, I assume

  System.Move( AAppend[0], ADst[LSrcLen], LAddLen );  // Copy only the appended part, in best case no copy happend before.
  
Result := Length(ADst);
end;

According to the SetLength
https://docwiki.embarcadero.com/Libraries/Florence/en/System.SetLength

Quote

For a dynamic array variable, SetLength reallocates the array referenced by S to the given length.
Existing elements in the array are preserved and newly allocated space is set to 0 or nil. For multidimensional dynamic arrays, SetLength may take more than one-length parameter (up to the number of array dimensions). Each parameter specifies the number of elements along a particular dimension.

Following a call to SetLength, S is guaranteed to reference a unique string or array -- that is, a string or array with a reference count of one.
If there is not enough memory available to reallocate the variable, SetLength raises an EOutOfMemory exception.

! There is also TBytesStream, for multiple, small appends to an array, but this is NOT my topic here, I want to handle two already existing dynamic arrays.
Sorry, the documentation is not 100% clear to me and I also cannot find much about this in the web.

Shall I consider a version with "Move" version above, or is the "Concat" version alone the right one-and-only way to go nowadays?

 

Edited Saturday at 09:22 AM by Rollo62

[pyscripter 858]

I would just use:

 

type

  A, B, C : TArray<something>;

 

C := A + B

 

The above results in calling System._DynArrayCat3 which works roughly as @Remy Lebeau suggestion at StackOverflow and has a clear and concise  syntax.

 

In recent versions of Delphi, there is also TArray<T>.Concat.

[David Heffernan 2494]

It's difficult to comment on performance related questions without knowledge of real use cases. Presumably you've measured the various options with realistic data on a range of hardware? What did that tell you? 

[Rollo62 616]

2 hours ago, pyscripter said:

The above results in calling System._DynArrayCat3 which works roughly as @Remy Lebeau suggestion at StackOverflow 

Oh, I thought I could see that call under System, your right, the debugger moves into it.
It lands under Intel MacOS at _DynArrayCat for me, not _DynArrayCat3, perhaps thats Windows specific and not compatible under all Platforms.
Yes, this shows similar, but more pointer-oriented way.

So it seems: Concat behaves New-Allocation + copy 1 (old part) + copy 2 (new part).
Ok, I will use Concat then, no need for me to over-optimize here towards specific TBytes arrays.