Upgraded to D12 and can't call the GPU for AI inference on Android

[tomye 1]

I am using the inference library provided by Tensorflow Lite, and calling it in Delphi to implement AI inference

 

Tensorflow Lite can support GPU inference well on mobile phones, greatly improving the inference speed
 

CPU inference speed:

 

 

GPU inference speed:

 

 

 

I call the libtensorflowlite_gpu_jni.so of Tensorflow Lite

  LibHandle := System.SysUtils.LoadLibrary(PWideChar('libtensorflowlite_gpu_jni.so')); 

  InterpreterOptionsAddDelegate := GetProcAddress(LibHandle, 'TfLiteInterpreterOptionsAddDelegate');

 

And add a GPU delegate during program initialization

  InterpreterOptionsAddDelegate(InterpreterOptions, GpuDelegate);

  Interpreter := InterpreterCreate(Model, InterpreterOptions);

 

The problem occurs here, before I used D11, the Interpreter can return an object pointer

 

But after upgrading to D12, it will return a nil, unable to call the GPU, I also tried D11.2 and D11.3

 

It's the same, all return a nil, only D11 can return an object pointer, and smoothly call the GPU on the phone

 

I thought it might be caused by the upgrade of the Java SDK or NDK version, but after importing the Java SDK and NDK library used by D11, the problem still cannot be solved

 

I am unfamiliar with Java, and in Delphi's SDK Manager, there seem to be only 3 SDK configurations, namely sdk, ndk, and jdk

 

I don't know what these 3 SDKs represent, and I vaguely feel that the versions of these 3 libraries are not correct, causing my problem

 

But I also tried to replace all 3 SDKs with the ones used by D11, and it still doesn't work
 

During testing, I found a detail, I use the following code to call Toast

 

unit Android.Toast;

interface

{$IFDEF ANDROID}

uses
  FMX.Platform.Android,
  Androidapi.Helpers,
  Androidapi.JNIBridge,
  Androidapi.JNI.JavaTypes,
  Androidapi.JNI.GraphicsContentViewText;
{$ENDIF}
{$IFDEF ANDROID}

type
  TToastLength = (LongToast, ShortToast);

  JToast = interface;

  JToastClass = interface(JObjectClass)
    ['{69E2D233-B9D3-4F3E-B882-474C8E1D50E9}']
    { Property methods }
    function _GetLENGTH_LONG: Integer; cdecl;
    function _GetLENGTH_SHORT: Integer; cdecl;
    { Methods }
    function init(context: JContext): JToast; cdecl; overload;
    function makeText(context: JContext; text: JCharSequence; duration: Integer)
      : JToast; cdecl;
    { Properties }
    property LENGTH_LONG: Integer read _GetLENGTH_LONG;
    property LENGTH_SHORT: Integer read _GetLENGTH_SHORT;
  end;

  [JavaSignature('android/widget/Toast')]
  JToast = interface(JObject)
    ['{FD81CC32-BFBC-4838-8893-9DD01DE47B00}']
    { Methods }
    procedure cancel; cdecl;
    function getDuration: Integer; cdecl;
    function getGravity: Integer; cdecl;
    function getHorizontalMargin: Single; cdecl;
    function getVerticalMargin: Single; cdecl;
    function getView: JView; cdecl;
    function getXOffset: Integer; cdecl;
    function getYOffset: Integer; cdecl;
    procedure setDuration(value: Integer); cdecl;
    procedure setGravity(gravity, xOffset, yOffset: Integer); cdecl;
    procedure setMargin(horizontalMargin, verticalMargin: Single); cdecl;
    procedure setText(s: JCharSequence); cdecl;
    procedure setView(view: JView); cdecl;
    procedure show; cdecl;
  end;

  TJToast = class(TJavaGenericImport<JToastClass, JToast>)
  end;
var
  PToast:JToast;

procedure Toast(const Msg: string; duration: TToastLength = ShortToast);

{$ENDIF}

implementation

{$IFDEF ANDROID}

uses
  FMX.Helpers.Android;


procedure Toast(const Msg: string; duration: TToastLength);
var
  ToastLength: Integer;
begin
  if duration = ShortToast then
    ToastLength := TJToast.JavaClass.LENGTH_SHORT
  else
    ToastLength := TJToast.JavaClass.LENGTH_LONG;
  CallInUiThread(
    procedure
    begin
      //TJToast.JavaClass.makeText(SharedActivityContext, StrToJCharSequence(Msg),
        //ToastLength).show
      if not Assigned(PToast) then
        PToast:=TJToast.JavaClass.makeText(SharedActivityContext, StrToJCharSequence(Msg),
        ToastLength)
      else
      begin
        PToast.setDuration(ToastLength);
        PToast.setText(StrToJCharSequence(Msg));
      end;
      PToast.show;
    end);
end;
{$ENDIF}

end.

 

Using this Toast in D11, there is no icon displayed in the Toast prompt, but GPU can be called normally

 

 

 

However, in D11.2, D11.3, and D12, when calling, an icon will automatically appear in the Toast prompt, but the GPU cannot be called
 

 

 

From this phenomenon, it is inferred that the SDK version with the icon should be a higher version, that is, the higher version of the SDK cannot call the GPU

 

So, how can I make D12 use a lower version of the SDK now (i believe the no icon version should be worked)?

(I have tried specifying a lower version of the SDK in the SDK Manager, but the Toast still displays an icon and the GPU cannot be called)
 

 

 

[Dave Nottage 557]

43 minutes ago, tomye said:

I call the libtensorflowlite_gpu_jni.so of Tensorflow Lite

Your example is a bit light on details - how are all these defined?

 

InterpreterOptionsAddDelegate

InterpreterOptions

GpuDelegate

InterpreterCreate
Model

 

Regardless, have you used a logcat viewer to look for related messages (error, or otherwise)?

[tomye 1]

2 hours ago, Dave Nottage said:

Your example is a bit light on details - how are all these defined?

 

InterpreterOptionsAddDelegate

InterpreterOptions

GpuDelegate

InterpreterCreate
Model

 

Regardless, have you used a logcat viewer to look for related messages (error, or otherwise)?

 

i can't get detail error message, beacuase it calls .so library file

 

 

only return a nil object

 

but it doesn't happen on D11

 

 

 

 

[Cristian Peța 103]

1 hour ago, tomye said:

i can't get detail error message, beacuase it calls .so library file

Dave asked if you looked for messages into the log, not the error in debugging.

[tomye 1]

1 hour ago, Cristian Peța said:

Dave asked if you looked for messages into the log, not the error in debugging.

 

there are no any useful message 😞

even doesn't raise an error, just return a nil object

 

 

Edited May 10 by tomye

[Cristian Peța 103]

logcat is about Android log:

https://developer.android.com/tools/logcat

[Dave Nottage 557]

6 hours ago, tomye said:

i can't get detail error message, beacuase it calls .so library file

There's still insufficient detail in your code example (and screenshots are the worst form of example code).

3 hours ago, tomye said:

there are no any useful message 😞

As per my initial reply, I said to use "a logcat viewer to look for related messages" (from the Android device), not messages in the IDE. 

[tomye 1]

i have known the reason.

 

the isssue comes from the java sdk version, the high version (D11.2 to D12 used) doesn't support the current TFLite GPU delegate

 

i copy the D11 PAClient to D12 and got an error message:

 

[PAClient Error] Error: E7688 java.lang.UnsupportedClassVersionError: com/embarcadero/dexer/Main has been compiled

by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 52.0

 

At the time of Delphi release, it was bundled with the java sdk version

 

the only way is i have to upgrade the TFLite GPU delegate ( libtensorflowlite_gpu_jni. so ) to high version, maybe it can be supported. 

 

thank you for your all help

[Dave Nottage 557]

5 hours ago, tomye said:

the only way is i have to upgrade the TFLite GPU delegate ( libtensorflowlite_gpu_jni. so ) to high version, maybe it can be supported. 

I sincerely doubt that is the only way, however since you refuse to give a more complete example (not in a screenshot, and something that is sufficient to reproduce the issue), it's difficult to find an answer.

 

Good luck

[tomye 1]

On 5/11/2024 at 4:05 AM, Dave Nottage said:

I sincerely doubt that is the only way, however since you refuse to give a more complete example (not in a screenshot, and something that is sufficient to reproduce the issue), it's difficult to find an answer.

 

Good luck

finally, i solved the problem

 

there is a file which is named AndroidManifest.template.xml

 

i modified this code

...

    <uses-sdk android:minSdkVersion="%minSdkVersion%" android:targetSdkVersion="%targetSdkVersion%" />

...

to 

...

    <uses-sdk android:minSdkVersion="%minSdkVersion%" android:targetSdkVersion="30" />

...

 

re-build and run , it does work now on D12

[Dave Nottage 557]

2 hours ago, tomye said:

i modified this code

It appears TF is accessing resources that have been restricted in Android 12 and above. Do you plan on deploying the app to Play Store?