Returning a dynamic array from DLL

[Tim Chilvers 0]

Hi,

 

I am in the process of writting a plugin system for my application and would like to return a dynamic array of interfaces from the plugin DLLs.

 

The function that is returning the array is below

function TTestWizard.GetCategories(var ArrayCount: Integer): Pointer;
var
  LArray: TArray<IProjectItemCategory>;
  LManager: IProjectItemCategoryServices;
  LCategory: IProjectItemCategory;
begin
  SetLength(LArray, 0);
    
  LManager := Services as IProjectItemCategoryServices;

  if LManager <> nil then
  begin
    LCategory := LManager.FindCategory('New');
    
    LArray[0] := LCategory;
    ArrayCount := Length(LArray);
    
    Result := LArray[0];
  end;
end;

However when I'm trying to run a comparison of the returned interface fields I am getting an AV:

 

To method that does the work, calls the function, loops through the array and adds the resulting data to a TList<>

procedure TProjectItemDialog.TreeViewFocusChanged(Sender: TBaseVirtualTree;
  Node: PVirtualNode; Column: TColumnIndex);

  function IsGalleryCategory(const ACategory: IProjectItemCategory;
    const AIdString: string): Boolean;
  begin
    Result := (ACategory <> nil) and (ACategory.IdString = AIdString);
  end;

  function IsInGalleryCategories(const ACategories: TArray<IProjectItemCategory>;
    const AIdString: string): Boolean;
  var
    LArrayCount: Integer;
  begin
    Result := False;

    if ACategories <> nil then
    begin
      for LArrayCount := Low(ACategories) to High(ACategories) do
      begin
        Result := IsGalleryCategory(ACategories[LArrayCount], AIdString);

        if Result then
        begin
          Exit;
        end;
      end;
    end;
  end;

var
  LData: PCategoryNodeData;
  LServices: IWizardServices;
  LCount: Integer;
  LCategoryCount: Integer;
  LWizard: IProjectItemWizard;
  LCategory: IProjectItemCategory;
  LCategories: TArray<IProjectItemCategory>;
  LPtrArrayData: Pointer;
begin
  WizardControlList.ItemCount := 0;
  try
    FWizardList.Clear;

    LData := GetNodeData(Node);

    if LData <> nil then
    begin
      LServices := Services as IWizardServices;

      if LServices <> nil then
      begin
        for LCount := 0 to LServices.WizardCount - 1 do
        begin
          if Supports(LServices.Wizard[LCount], IProjectItemWizard, LWizard) then
          begin
            LCategory := LWizard.Category;
//            LCategories := LWizard.Categories;

            LCategoryCount := 0;
            LWizard.GetCategories(LPtrArrayData, LCategoryCount);
            SetLength(LCategories, LCategoryCount);

            Move(LPtrArrayData^, LCategories[0], LCategoryCount);

            if IsGalleryCategory(LCategory, LData.Id) or IsInGalleryCategories(LCategories, LData.Id) then
            begin
              FWizardList.Add(LWizard);
            end;
          end;
        end;
      end;
    end;
  finally
    WizardControlList.ItemCount := FWizardList.Count;
  end;
end;

The AV is happening in the function IsGalleryCategory when it is being called by IsInGalleryCategories.

 

If I change to using another function within the DLL to return a TArray<IProjectItemCategory> then everything works correctly. The reason for trying to resurn an array like this is so I can allow people to develop plugins for my application without having to rely on them using Delphi

 

Tim

Edited April 1, 2021 by Tim Chilvers

[David Heffernan 2345]

Thus can't work. You need both sides of the interface to use the same memory allocator and the same dynamic array implementation. Which ties you to specific delphi versions. That forces your plugins writers to use Delphi. Amongst other requirements. 

[Remy Lebeau 1393]

Using interfaces is fine, as long as compilers agree to a common ABI for them.  Which typically means you need to use COM interfaces if you want non-Delphi/C++Builder compilers to interact with your code.

 

The main reason your GetCategories() function crashed was two-fold:

 

- it did not allocate any memory for its local TArray, thus when it tried to save an interface to the array's 1st element, it was going out of bounds.

 

- it was returning a raw pointer to the interface stored in the local array's 1st element, not the array itself. But either way, that array and all of its elements were getting finalized when the function exited, thus you were returning a dangling pointer to invalid memory.

 

To persist the array behind the function exit, you need to allocate it dynamically, and you need to increment the refcounts of the individual interfaces in the array.  Using COM interfaces, you can let (Ole)Variant handle all of that for you, eg:

type
  IProjectItemCategory = interface(IUnknown)
    ['{840DD036-8F0F-4B0F-97D0-AB76CCC2157B}']
    function GetIdString: WideString;
    ...
    property IdString: WideString read GetIdString;
    ...
  end;

  IProjectItemWizard = interface(IUnknown)
    ['{6C65F413-F2E2-4554-8828-1FF10613855B}']
    function GetCategory: IProjectItemCategory; safecall;
    function GetCategories: OleVariant; safecall;
    ...
    property Category: IProjectItemCategory read GetCategory;
    property Categories: OleVariant read GetCategories;
    ...
  end;

...

function TTestWizard.GetCategory: IProjectItemCategory; safecall;
begin
  Result := ...;
end;

function TTestWizard.GetCategories: OleVariant; safecall;
var
  LManager: IProjectItemCategoryServices;
  LCategory: IProjectItemCategory;
  LArray: Variant;
begin
  LManager := Services as IProjectItemCategoryServices;
  if LManager <> nil then
  begin
    LCategory := LManager.FindCategory('New');
    if LCategory <> nil then
    begin
      LArray := VarArrayCreate([0..0], varUnknown);
      LArray[0] := IUnknown(LCategory);
      // alternatively:
      // LArray := VarArrayOf([IUnknown(LCategory)]);
    end;
  end;
  Result := LArray;
end;

procedure TProjectItemDialog.TreeViewFocusChanged(Sender: TBaseVirtualTree;
  Node: PVirtualNode; Column: TColumnIndex);

  function IsGalleryCategory(const ACategory: IProjectItemCategory;
    const AIdString: string): Boolean;
  begin
    Result := (ACategory <> nil) and (ACategory.IdString = AIdString);
  end;

  function IsInGalleryCategories(const ACategories: OleVariant;
    const AIdString: string): Boolean;
  var
    I: Integer;
  begin
    Result := False;
    if VarIsArray(ACategories) then
    begin
      for I := VarArrayLowBound(ACategories, 1) to VarArrayHighBound(ACategories, 1) do
      begin
        Result := IsGalleryCategory(IUnknown(ACategories[I]) as IProjectItemCategory, AIdString);
        if Result then Exit;
      end;
    end;
  end;

var
  LData: PCategoryNodeData;
  LServices: IWizardServices;
  LCount, I: Integer;
  LCategoryCount: Integer;
  LWizard: IProjectItemWizard;
  LCategory: IProjectItemCategory;
  LCategories: OleVariant;
begin
  WizardControlList.ItemCount := 0;
  try
    FWizardList.Clear;

    LData := GetNodeData(Node);
    if LData <> nil then
    begin
      LServices := Services as IWizardServices;
      if LServices <> nil then
      begin
        for LCount := 0 to LServices.WizardCount - 1 do
        begin
          if Supports(LServices.Wizard[LCount], IProjectItemWizard, LWizard) then
          begin
            LCategory := LWizard.Category;
            LCategories := LWizard.GetCategories;
            if IsGalleryCategory(LCategory, LData.Id) or IsInGalleryCategories(LCategories, LData.Id) then
            begin
              FWizardList.Add(LWizard);
            end;
          end;
        end;
      end;
    end;
  finally
    WizardControlList.ItemCount := FWizardList.Count;
  end;
end;

 

Edited April 2, 2021 by Remy Lebeau

[Fr0sT.Brutal 900]

General approach is to write results to a memory allocated by caller. To handle beforehand unknown number of elements, there are options:

1 - limit a maximal number of elements and always allocate memory for this number

2 - caller allocates memory and passes number of elements he had allocated space for, function returns how much elements it had copied (if ResultCnt <= AllocatedCnt) or how much elements were skipped (if ResultCnt > AllocatedCnt)

3 - caller calls function with NULL memory, function returns how much elements it could copy, caller allocates memory and calls function again

 

P.S. Learn WinAPI, it has pretty good approaches to interacting with DLLs. Everything I know on this subject, I took from WinAPI practice 🙂

Edited April 2, 2021 by Fr0sT.Brutal

[Tim Chilvers 0]

8 hours ago, Remy Lebeau said:

Using interfaces is fine, as long as compilers agree to a common ABI for them.  Which typically means you need to use COM interfaces if you want non-Delphi/C++Builder compilers to interact with your code.

 

The main reason your GetCategories() function crashed was two-fold:

 

- it did not allocate any memory for its local TArray, thus when it tried to save an interface to the array's 1st element, it was going out of bounds.

 

- it was returning a raw pointer to the local array's 1st element, not the array itself. But either way, that array and all of its elements were getting finalized when the function exited, thus you were returning a dangling pointer to invalid memory.

Hi Remy,

 

Thanks for the help, this works perfectly both as an "internel" plugin and also via a DLL.

 

I had to make a slight change to the VarArrayCreate call as it wanted the parameters to be: VarArrayCreate([0, 1], varUnknown). This maybe different as I am using Delphi 10.4.2

[Remy Lebeau 1393]

6 hours ago, Tim Chilvers said:

I had to make a slight change to the VarArrayCreate call as it wanted the parameters to be: VarArrayCreate([0, 1], varUnknown). This maybe different as I am using Delphi 10.4.2

The 1st parameter of VarArrayCreate() does not take an index and a count, as you are suggesting.  It takes lower/upper bounds (indexes) in pairs, one for each dimension of the array.  So, specifying [0, 1] will create a 1-dimensional array with indexes from 0 to 1 (thus 2 elements).  Whereas [0, 0] will create a 1-dimensional array with indexes from 0 to 0 (thus 1 element).  An empty array would be [0, -1], indexes from 0 to -1 (thus 0 elements).

 

Alternatively, if you know all of the array values up front, you can use VarArrayOf() instead of VarArrayCreate().  I have updated my earlier example to show that.

Edited April 2, 2021 by Remy Lebeau