Transferring TBitmap to Python and back

BennieC · March 25, 2022

Hi, we are doing some image processing and will use OpenCV for it. Doing this from Python is very useful as it disconnects us from the OpenCV libraries which we can now directly use. However, most processing is done with bitmaps. I have used the Demo29 and that works but it has the following limitations from our perspective.

It requires image in a TGraphic form which we don't get from a frame grabber and I find no documentation to describe either the structure of or how to convert a bitmap to it.

Internally we can convert the Byte stream with cv.imdecode into a opencv image and can convert our result image back to PILLOW

Returning the image however does not work unless the image is converted back to a PILLOW format from where demo29 code can display it.

I think part of my problem is that I don't have enough information regarding the PILLOW format or the TGraphic format. Also I don't find information about functions such as PyEval_CallObjectWithKeywords and ExtractPythonObjectFrom as the statement in demo29 returns a nil if I return any other format than PILLOW from Python.

Any assistance will be appreciated and if anyone can point me to support documentation describing how to use bitmaps in Python and also how to understand the many Python methods. I ahve been mostly a Delphi programmer.

An example of how we do it is as follows but this seems unnecessary and also requires input in TGraphic form, not TBitmap.

def LoadPicData(data):
# Convert data (image converted into Python Bytes) into a ocv image
stream = BytesIO(data)
ocv_img = cv.imdecode(np.asarray(bytearray(stream.read()), dtype=np.uint8), cv.IMREAD_COLOR)
pil_src = Image.open(stream)
# cv.imshow('ocv', ocv_img)
ocv_img = cv.cvtColor(ocv_img, cv.COLOR_BGR2GRAY)
# cv.imshow('gray', ocv_img)

# Convert this image to pil image
color_converted = cv.cvtColor(ocv_img, cv.COLOR_BGR2RGB)
pil_img = Image.fromarray(color_converted) # convert ocv to PIL image
pil_img.format = pil_src.format
# ImageShow.show(pil_img, 'PIL')
return pil_img

Some assistance will be much appreciated.

shineworld · March 26, 2022

IMO, you have two solutions:

a) Use the new, but experimental, Delphi-OpenCV-Class (opencv 4.5.5) for Delphi 10.4 & 11.0 package:
https://github.com/Laex/Delphi-OpenCV-Class

Laex is doing a very very good job to import opencv 4.5.5 C++ based DLLs to Delphi.
I don't know how much of opencv is already ported but I've cloned the project only yesterday and tried a basic Canny function:

Works very very fastly, with native Delphi (in the snapshot I've got frames from a USB camera and applied canny filter).
During run-in IDE environment there are some delays (also in Release) because uses debug versions of DLLs,
but when you close the IDE and start the program from Explorer BOOM is a rocket.
I will move, only in the next future, my Python delphivcl application to be a native Delphi set of classes.
This ONLY to fully integrate into Forms of native VCL program.

b) To back processed image to Delphi just pass the opencv NumPy array content which can be o monodimensional array

for gray images or a tridimensional array for images with RGB.
Pass also Width and Height info.
Then you have only to use a Bitmap, and related canvas HDC Handle with the same width and height and use

DIB windows functions to recreate bitmap image from a byte array.

This is an extract of yesterday's experiments for upon Image.
TMat is the Laex generics implementation of OpenCV mat so array of bytes of image:

function MatDraw(DC: HDC; Image: TMat; const Rect: TRect; const Stretch: Boolean = True): Boolean;
type
  pCOLORREF         = ^COLORREF;
  pBITMAPINFOHEADER = ^BITMAPINFOHEADER;
var
  // isrgb: Boolean;
  IsGray: Boolean;
  buf: array [1 .. SizeOf(BITMAPINFOHEADER) + SizeOf(RGBQUAD) * 256] of byte;
  dibhdr: pBITMAPINFOHEADER;
  _dibhdr: TBitmapInfo ABSOLUTE buf;
  _rgb: pCOLORREF;
  i: Integer;
  iResult: Integer;
begin
  if Image.empty then
    Exit(False);

  // isrgb := ('R' = upcase(img^.colorModel[0])) and ('G' = upcase(img^.colorModel[1])) and ('B' = upcase(img^.colorModel[2]));
  // isgray := 'G' = upcase(img^.colorModel[0]);
  IsGray := Image.channels = 1;
  // if (not isgray) and (not isrgb) then
  // Exit(false);
  // if (1 = img^.nChannels) and (not isgray) then
  // Exit(false);

  dibhdr := pBITMAPINFOHEADER(@buf);
  _rgb := pCOLORREF(Integer(dibhdr) + SizeOf(BITMAPINFOHEADER));

  if (IsGray) then
    for i := 0 to 255 do
      _rgb[i] := rgb(i, i, i);

  dibhdr^.biSize := SizeOf(BITMAPINFOHEADER);
  dibhdr^.biWidth := Image.cols;
  // Check origin for display
  // if img^.Origin = 0 then
  dibhdr^.biHeight := -Image.rows;
  // else
  // dibhdr^.biHeight := img^.Height;

  dibhdr^.biPlanes := 1;
  dibhdr^.biBitCount := 8 * Image.channels;
  dibhdr^.biCompression := BI_RGB;
  dibhdr^.biSizeImage := 0; // img^.imageSize;
  dibhdr^.biXPelsPerMeter := 0;
  dibhdr^.biYPelsPerMeter := 0;
  dibhdr^.biClrUsed := 0;
  dibhdr^.biClrImportant := 0;

  if Stretch then
  begin
    SetStretchBltMode(DC, COLORONCOLOR);
    SetMapMode(DC, MM_TEXT);
    // Stretch the image to fit the rectangle
    iResult := StretchDIBits(DC, Rect.Left, Rect.Top, Rect.Width, Rect.Height, 0, 0, Image.cols, Image.rows, Image.Data, _dibhdr, DIB_RGB_COLORS, SRCCOPY);
    Result := (iResult > 0); // and (iResult <> GDI_ERROR);
  end
  else
  begin
    // Draw without scaling
    iResult := SetDIBitsToDevice(DC, Rect.Left, Rect.Top, Image.cols, Image.rows, 0, 0, 0, Image.rows, Image.Data, _dibhdr, DIB_RGB_COLORS);
    Result := (iResult > 0); // and (iResult <> GDI_ERROR);
  end;
end;

Code went from Laex library sources so try to read them.

BennieC · March 28, 2022

Thank you for this info - undoubtedly the direct Delphi/OpenCV approach would be best (We thought we lost these at OpenCV V2) and we will try and use that. It would be great if Embarcadero would also support it so that it could stay up to date.

BennieC · March 29, 2022

I have installed the Delphi-OpenCV class but have difficulty in running the samples. After a lot of running around to find the various C++ redistributables I finally got stuck with vcruntime140_1d.dll which was nowhere to be found. I did find some download (Whether it is legit I don't know) but all samples including the video capture fail with 'The application was unable to start correctly'. Any idea on how to trace this problem.

shineworld · March 29, 2022

In the dropbox link, I can't attach files bigger than 4.8Mb here,

you can find a very simple Delphi Sydney 10.4.1 project where

the OpenCV get frames from a USB camera (VideoCapture device 0)

and with the trackbar, you can apply Canny filter thresholds 1 & 2.

The required DLLs are in Win64\Debug & Release folders BUT

can also be placed one time for always in Windows OS folders.

Just my first test with Delphi-OpenCV.

ONLY 64 bit applications are supported by Delphi-OpenCV class.

https://www.dropbox.com/s/iqzlnfwttj20pnr/cnc_vision_2.7z?dl=0

PS: I don't know if there are other MS dependencies because I've Microsoft Visual Studio installed
and some DLL could be already available for MSVC packages.

During run-in IDE these are modules loaded:

Module Load: cnc_vision_2. No Debug Info. Base Address: $0000000000400000. Process cnc_vision_2.exe (14340)
Module Load: ntdll.dll. No Debug Info. Base Address: $00007FFD371B0000. Process cnc_vision_2.exe (14340)
Module Load: KERNEL32.dll. No Debug Info. Base Address: $00007FFD36870000. Process cnc_vision_2.exe (14340)
Module Load: KERNELBASE.dll. No Debug Info. Base Address: $00007FFD34BA0000. Process cnc_vision_2.exe (14340)
Module Load: WINSPOOL.DRV. No Debug Info. Base Address: $00007FFD2AF90000. Process cnc_vision_2.exe (14340)
Module Load: SHELL32.dll. No Debug Info. Base Address: $00007FFD36930000. Process cnc_vision_2.exe (14340)
Module Load: WINMM.dll. No Debug Info. Base Address: $00007FFD2B2A0000. Process cnc_vision_2.exe (14340)
Module Load: msvcp_win.dll. No Debug Info. Base Address: $00007FFD34FC0000. Process cnc_vision_2.exe (14340)
Module Load: COMCTL32.dll. No Debug Info. Base Address: $00007FFD26F50000. Process cnc_vision_2.exe (14340)
Module Load: msvcrt.dll. No Debug Info. Base Address: $00007FFD35B40000. Process cnc_vision_2.exe (14340)
Module Load: ucrtbase.dll. No Debug Info. Base Address: $00007FFD348D0000. Process cnc_vision_2.exe (14340)
Module Load: msvcrt.dll. No Debug Info. Base Address: $0000000000D50000. Process cnc_vision_2.exe (14340)
Module Unload: msvcrt.dll. Process cnc_vision_2.exe (14340)
Module Load: msvcrt.dll. No Debug Info. Base Address: $0000000000DF0000. Process cnc_vision_2.exe (14340)
Module Unload: msvcrt.dll. Process cnc_vision_2.exe (14340)
Module Load: USER32.dll. No Debug Info. Base Address: $00007FFD35BE0000. Process cnc_vision_2.exe (14340)
Module Load: GDI32.dll. No Debug Info. Base Address: $00007FFD370E0000. Process cnc_vision_2.exe (14340)
Module Load: win32u.dll. No Debug Info. Base Address: $00007FFD34F20000. Process cnc_vision_2.exe (14340)
Module Load: win32u.dll. No Debug Info. Base Address: $00000000001A0000. Process cnc_vision_2.exe (14340)
Module Unload: win32u.dll. Process cnc_vision_2.exe (14340)
Module Load: OLEAUT32.dll. No Debug Info. Base Address: $00007FFD35390000. Process cnc_vision_2.exe (14340)
Module Load: gdi32full.dll. No Debug Info. Base Address: $00007FFD34A00000. Process cnc_vision_2.exe (14340)
Module Load: VERSION.dll. No Debug Info. Base Address: $00007FFD2DB50000. Process cnc_vision_2.exe (14340)
Module Load: combase.dll. No Debug Info. Base Address: $00007FFD36460000. Process cnc_vision_2.exe (14340)
Module Load: RPCRT4.dll. No Debug Info. Base Address: $00007FFD35540000. Process cnc_vision_2.exe (14340)
Module Load: ADVAPI32.dll. No Debug Info. Base Address: $00007FFD36140000. Process cnc_vision_2.exe (14340)
Module Load: SECHOST.dll. No Debug Info. Base Address: $00007FFD36290000. Process cnc_vision_2.exe (14340)
Module Load: ole32.dll. No Debug Info. Base Address: $00007FFD36330000. Process cnc_vision_2.exe (14340)
Module Load: NETAPI32.dll. No Debug Info. Base Address: $00007FFD21B00000. Process cnc_vision_2.exe (14340)
Module Load: netutils.dll. No Debug Info. Base Address: $00007FFD33E40000. Process cnc_vision_2.exe (14340)
Module Load: IMM32.dll. No Debug Info. Base Address: $00007FFD356A0000. Process cnc_vision_2.exe (14340)
Module Load: MSCTF.dll. No Debug Info. Base Address: $00007FFD35210000. Process cnc_vision_2.exe (14340)
Module Load: UxTheme.dll. No Debug Info. Base Address: $00007FFD32280000. Process cnc_vision_2.exe (14340)
Module Load: AppCore.dll. No Debug Info. Base Address: $00007FFD32800000. Process cnc_vision_2.exe (14340)
Module Load: bcryptPrimitives.dll. No Debug Info. Base Address: $00007FFD34B10000. Process cnc_vision_2.exe (14340)
Module Load: WTSAPI32.dll. No Debug Info. Base Address: $00007FFD2F7C0000. Process cnc_vision_2.exe (14340)
Module Load: WINSTA.dll. No Debug Info. Base Address: $00007FFD33640000. Process cnc_vision_2.exe (14340)
Module Load: opencv_world455.dll. No Debug Info. Base Address: $00007FFCE7660000. Process cnc_vision_2.exe (14340)
Module Load: WS2_32.dll. No Debug Info. Base Address: $00007FFD35D80000. Process cnc_vision_2.exe (14340)
Module Load: COMDLG32.dll. No Debug Info. Base Address: $00007FFD35460000. Process cnc_vision_2.exe (14340)
Module Load: SHCORE.dll. No Debug Info. Base Address: $00007FFD367C0000. Process cnc_vision_2.exe (14340)
Module Load: SHLWAPI.dll. No Debug Info. Base Address: $00007FFD35330000. Process cnc_vision_2.exe (14340)
Module Load: VCRUNTIME140.dll. No Debug Info. Base Address: $00007FFD21D40000. Process cnc_vision_2.exe (14340)
Module Load: CONCRT140.dll. No Debug Info. Base Address: $00007FFD231C0000. Process cnc_vision_2.exe (14340)
Module Load: MSVCP140.dll. No Debug Info. Base Address: $00007FFD14770000. Process cnc_vision_2.exe (14340)
Module Load: VCRUNTIME140_1.dll. No Debug Info. Base Address: $0000000000830000. Process cnc_vision_2.exe (14340)
Module Unload: VCRUNTIME140_1.dll. Process cnc_vision_2.exe (14340)
Module Load: VCRUNTIME140_1.dll. No Debug Info. Base Address: $00007FFD2F5D0000. Process cnc_vision_2.exe (14340)
Module Load: opencv_videoio_msmf455_64.dll. No Debug Info. Base Address: $00007FFD22EB0000. Process cnc_vision_2.exe (14340)
Module Load: MF.dll. No Debug Info. Base Address: $00007FFD20830000. Process cnc_vision_2.exe (14340)
Module Load: MFReadWrite.dll. No Debug Info. Base Address: $00007FFD1F890000. Process cnc_vision_2.exe (14340)
Module Load: dxgi.dll. No Debug Info. Base Address: $00007FFD33210000. Process cnc_vision_2.exe (14340)
Module Load: MFPlat.DLL. No Debug Info. Base Address: $00007FFD060C0000. Process cnc_vision_2.exe (14340)
Module Load: CFGMGR32.dll. No Debug Info. Base Address: $00007FFD351C0000. Process cnc_vision_2.exe (14340)
Module Load: d3d11.dll. No Debug Info. Base Address: $00007FFD300B0000. Process cnc_vision_2.exe (14340)
Module Load: MFCORE.dll. No Debug Info. Base Address: $00007FFCE71D0000. Process cnc_vision_2.exe (14340)
Module Load: CRYPT32.dll. No Debug Info. Base Address: $00007FFD35060000. Process cnc_vision_2.exe (14340)
Module Load: bcrypt.dll. No Debug Info. Base Address: $00007FFD349D0000. Process cnc_vision_2.exe (14340)
Module Load: POWRPROF.dll. No Debug Info. Base Address: $00007FFD33EA0000. Process cnc_vision_2.exe (14340)
Module Load: ksuser.dll. No Debug Info. Base Address: $00007FFD32120000. Process cnc_vision_2.exe (14340)
Module Load: CRYPTBASE.dll. No Debug Info. Base Address: $00007FFD342A0000. Process cnc_vision_2.exe (14340)
Module Load: RTWorkQ.DLL. No Debug Info. Base Address: $00007FFD1D060000. Process cnc_vision_2.exe (14340)
Module Load: UMPDC.dll. No Debug Info. Base Address: $00007FFD33D10000. Process cnc_vision_2.exe (14340)
Module Load: CLBCatQ.DLL. No Debug Info. Base Address: $00007FFD36010000. Process cnc_vision_2.exe (14340)
Module Load: DEVENUM.DLL. No Debug Info. Base Address: $00007FFD252A0000. Process cnc_vision_2.exe (14340)
Module Load: SETUPAPI.dll. No Debug Info. Base Address: $00007FFD356D0000. Process cnc_vision_2.exe (14340)
Module Load: NTMARTA.dll. No Debug Info. Base Address: $00007FFD335A0000. Process cnc_vision_2.exe (14340)
Module Load: DEVOBJ.dll. No Debug Info. Base Address: $00007FFD34670000. Process cnc_vision_2.exe (14340)
Module Load: WINTRUST.dll. No Debug Info. Base Address: $00007FFD34F50000. Process cnc_vision_2.exe (14340)
Module Load: MSASN1.dll. No Debug Info. Base Address: $00007FFD344B0000. Process cnc_vision_2.exe (14340)
Module Load: msdmo.dll. No Debug Info. Base Address: $00007FFD2C5B0000. Process cnc_vision_2.exe (14340)
Module Load: QCap.dll. No Debug Info. Base Address: $00007FFD16770000. Process cnc_vision_2.exe (14340)
Module Load: QUARTZ.dll. No Debug Info. Base Address: $00007FFD145A0000. Process cnc_vision_2.exe (14340)
Module Load: Windows.Storage.dll. No Debug Info. Base Address: $00007FFD32A00000. Process cnc_vision_2.exe (14340)
Module Load: Wldp.dll. No Debug Info. Base Address: $00007FFD34330000. Process cnc_vision_2.exe (14340)
Module Load: Source.dll. No Debug Info. Base Address: $00007FFD166B0000. Process cnc_vision_2.exe (14340)
Module Load: ATL.DLL. No Debug Info. Base Address: $00007FFD24C60000. Process cnc_vision_2.exe (14340)
Module Load: MFSENSORGROUP.dll. No Debug Info. Base Address: $00007FFD15110000. Process cnc_vision_2.exe (14340)
Module Load: ksproxy.ax. No Debug Info. Base Address: $00007FFD14550000. Process cnc_vision_2.exe (14340)
Module Load: d3d9.dll. No Debug Info. Base Address: $00007FFD2CF10000. Process cnc_vision_2.exe (14340)
Module Load: dwmapi.dll. No Debug Info. Base Address: $00007FFD32510000. Process cnc_vision_2.exe (14340)
Module Load: policymanager.dll. No Debug Info. Base Address: $00007FFD2F4A0000. Process cnc_vision_2.exe (14340)
Module Load: msvcp110_win.dll. No Debug Info. Base Address: $00007FFD33A30000. Process cnc_vision_2.exe (14340)
Module Load: vidcap.dll. No Debug Info. Base Address: $00007FFD24C50000. Process cnc_vision_2.exe (14340)
Module Load: kswdmcap.dll. No Debug Info. Base Address: $00007FFD23190000. Process cnc_vision_2.exe (14340)
Module Load: MFC42.dll. No Debug Info. Base Address: $00007FFCF43E0000. Process cnc_vision_2.exe (14340)
Module Load: QEdit.dll. No Debug Info. Base Address: $00007FFD144A0000. Process cnc_vision_2.exe (14340)
Module Load: gdiplus.dll. No Debug Info. Base Address: $00007FFD2B040000. Process cnc_vision_2.exe (14340)
Module Load: MSVFW32.dll. No Debug Info. Base Address: $00007FFD1F210000. Process cnc_vision_2.exe (14340)
Module Load: DDRAW.dll. No Debug Info. Base Address: $00007FFCFC5D0000. Process cnc_vision_2.exe (14340)
Module Load: DCIMAN32.dll. No Debug Info. Base Address: $00007FFD238F0000. Process cnc_vision_2.exe (14340)
Module Load: nvldumdx.dll. No Debug Info. Base Address: $00007FFD2ADB0000. Process cnc_vision_2.exe (14340)
Module Load: imagehlp.dll. No Debug Info. Base Address: $00007FFD35670000. Process cnc_vision_2.exe (14340)
Module Load: CRYPTSP.dll. No Debug Info. Base Address: $00007FFD34280000. Process cnc_vision_2.exe (14340)
Module Load: RSAENH.dll. No Debug Info. Base Address: $00007FFD339A0000. Process cnc_vision_2.exe (14340)
Module Load: NVD3DUMX.dll. No Debug Info. Base Address: $00007FFCC5A50000. Process cnc_vision_2.exe (14340)
Module Load: nvspcap64.dll. No Debug Info. Base Address: $00007FFD09D00000. Process cnc_vision_2.exe (14340)
Module Load: profapi.dll. No Debug Info. Base Address: $00007FFD34810000. Process cnc_vision_2.exe (14340)
Module Load: NvCameraWhitelisting64.dll. No Debug Info. Base Address: $00007FFD14410000. Process cnc_vision_2.exe (14340)
Module Unload: NvCameraWhitelisting64.dll. Process cnc_vision_2.exe (14340)
Module Load: dxcore.dll. No Debug Info. Base Address: $00007FFD2B680000. Process cnc_vision_2.exe (14340)
Module Unload: NVD3DUMX.dll. Process cnc_vision_2.exe (14340)
Module Unload: nvldumdx.dll. Process cnc_vision_2.exe (14340)
Module Load: nvldumdx.dll. No Debug Info. Base Address: $00007FFD2ADB0000. Process cnc_vision_2.exe (14340)
Module Load: NVD3DUMX.dll. No Debug Info. Base Address: $00007FFCC5A50000. Process cnc_vision_2.exe (14340)
Module Load: NvCameraWhitelisting64.dll. No Debug Info. Base Address: $00007FFD14410000. Process cnc_vision_2.exe (14340)
Module Unload: NvCameraWhitelisting64.dll. Process cnc_vision_2.exe (14340)
Module Unload: NVD3DUMX.dll. Process cnc_vision_2.exe (14340)
Module Unload: nvldumdx.dll. Process cnc_vision_2.exe (14340)
Module Load: nvldumdx.dll. No Debug Info. Base Address: $00007FFD2ADB0000. Process cnc_vision_2.exe (14340)
Module Load: NVD3DUMX.dll. No Debug Info. Base Address: $00007FFCC5A50000. Process cnc_vision_2.exe (14340)
Module Load: NvCameraWhitelisting64.dll. No Debug Info. Base Address: $00007FFD14410000. Process cnc_vision_2.exe (14340)
Module Unload: NvCameraWhitelisting64.dll. Process cnc_vision_2.exe (14340)
Module Unload: NVD3DUMX.dll. Process cnc_vision_2.exe (14340)
Module Unload: nvldumdx.dll. Process cnc_vision_2.exe (14340)
Module Load: WINMMBASE.dll. No Debug Info. Base Address: $00007FFD22CE0000. Process cnc_vision_2.exe (14340)
Module Load: MSYUV.dll. No Debug Info. Base Address: $00007FFD238E0000. Process cnc_vision_2.exe (14340)
Module Unload: MSYUV.dll. Process cnc_vision_2.exe (14340)
Module Load: TextInputFramework.dll. No Debug Info. Base Address: $00007FFD2B430000. Process cnc_vision_2.exe (14340)
Module Load: CoreUIComponents.dll. No Debug Info. Base Address: $00007FFD31640000. Process cnc_vision_2.exe (14340)
Module Load: CoreMessaging.dll. No Debug Info. Base Address: $00007FFD319A0000. Process cnc_vision_2.exe (14340)
Module Load: WinTypes.dll. No Debug Info. Base Address: $0000000007A20000. Process cnc_vision_2.exe (14340)
Module Load: WinTypes.dll. No Debug Info. Base Address: $0000000007B80000. Process cnc_vision_2.exe (14340)
Module Unload: WinTypes.dll. Process cnc_vision_2.exe (14340)
Module Unload: WinTypes.dll. Process cnc_vision_2.exe (14340)
Module Load: WinTypes.dll. No Debug Info. Base Address: $00007FFD30ED0000. Process cnc_vision_2.exe (14340)
Module Load: OLEACC.dll. No Debug Info. Base Address: $00007FFD2AF20000. Process cnc_vision_2.exe (14340)

Edited March 29, 2022 by shineworld

BennieC · April 6, 2022

Thanks Shine, managed to get your sample to work. Now for my own stuff

BennieC · April 6, 2022

Just a quick question - where is the TVideoCapture defined - I can find no reference but the code works well!

shineworld · April 7, 2022

TVideoCapture is managed as a record, not a class, with class members so do not requires a TVideoCapture.Create

It is defined in videoio.inc.

Never tried if this structure permits more than one instance.

I've checked the demos and copied/pasted the minimal code to try it.

My actual implementation is yet on Python + delphivcl + opencv.
It's a very big project and I'm yet in the writting phase.

When finished I will try to recreate my vision python framework directly in Delphi code + DelphiPythonVCL.

shineworld · April 12, 2022

To fastly send an image in NumPy array to a delphivcl.Image component, without use Pillow and DIB way, you can create a new python extension module with Delphi

where you wrap a update_image_from_bytes(image: Image(), data: bytes, width: int, height: int, channels: int) -> bool

{ generic functions wrappers }

function CncVisionUpdateImageFromBytesArray_Wrapper(pself, args: PPyObject): PPyObject; cdecl;
var
  Image: TImage;
  Width: Integer;
  Height: Integer;
  Channels: Integer;
  Bytes: TByteArray;
  BytesPyObj: PPyObject;
  ImagePyObj: PPyObject;

  function PyBytesAsBytes(Obj: PPyObject): TByteArray;
  var
    Size: NativeInt;
    Buffer: PAnsiChar;
  begin
    Result := nil;
    with GetPythonEngine do
    begin
      if not PyBytes_Check(Obj) then Exit;
      PyBytes_AsStringAndSize(Obj, Buffer, Size);
      if Size = 0 then Exit;
      SetLength(Result, Size);
      CopyMemory(Result, Buffer, Size);
    end;
  end;

begin
  with GetPythonEngine do
  begin
    try
      if PyArg_ParseTuple(args, 'OOiii:update_image_from_bytes', @ImagePyObj, @BytesPyObj, @Width, @Height, @Channels) <> 0 then
      begin
        if ImagePyObj.ob_type.tp_name <> 'Image' then AbortFast;
        Image := TPyDelphiImage(TPyObject(PAnsiChar(ImagePyObj) + SizeOf(PyObject))).DelphiObject;
        if Image = nil then AbortFast;

        if BytesPyObj.ob_type.tp_name <> 'bytes' then AbortFast;
        Bytes := PyBytesAsBytes(BytesPyObj);
        if Bytes = nil then AbortFast;
        if Length(Bytes) <> (Width * Height * Channels) then AbortFast;

        if not CncVisionUpdateImageFromBytesArray(Image, Bytes, Width, Height, Channels) then AbortFast;
        Result := ReturnTrue;
      end
      else
        Result := ReturnFalse;
    except
      Result := ReturnFalse;
    end;
  end;
end;

...
.
...

procedure TPyExtensionManager.WrapperInitializationEvent(Sender: TObject);
begin
  FWrapper.RegisterFunction
  (
    PAnsiChar('update_image_from_bytes'),
    CncVisionUpdateImageFromBytesArray_Wrapper,
    PAnsiChar('update delphivcl Image object from bytes array with width, height & channels')
  );
end;

Then the pascal byte array to Image is:

function CncVisionUpdateImageFromBytesArray(Image: TImage; Bytes: osCncVisionTypes.TByteArray; Width, Height, Channels: Integer): Boolean;
type
  TRGBBitmapInfoHeader = record
    Header: TBitmapInfoHeader;
    ColorTable: array[0..255] of TRGBQuad;
  end;
  PBitmapInfoHeader = ^TBitmapInfoHeader;
var
  I: Integer;
  Bitmap: TBitmap;
  IsGray: Boolean;
  Buffer: TRGBBitmapInfoHeader;
  BmpInfoHeader: PBitmapInfoHeader;
var
  BmpInfoBuffer: TBitmapInfo absolute Buffer;
begin
  try
    if Length(Bytes) = 0 then
      Exit(False);
    if Length(Bytes) <> (Width * Height * Channels) then
      Exit(False);
    if not Channels in [1, 3] then
      Exit(False);

    if Image.Picture.Bitmap = nil then
      Image.Picture.Bitmap := TBitmap.Create;
    Bitmap := Image.Picture.Bitmap;
    Bitmap.Width := Width;
    Bitmap.Height := Height;

    BmpInfoHeader := PBitmapInfoHeader(@Buffer);
    BmpInfoHeader^.biSize          := SizeOf(TBitmapInfoHeader);
    BmpInfoHeader^.biWidth         := Width;
    BmpInfoHeader^.biHeight        := -Height;
    BmpInfoHeader^.biPlanes        := 1;
    BmpInfoHeader^.biBitCount      := 8 * Channels;
    BmpInfoHeader^.biCompression   := BI_RGB;
    BmpInfoHeader^.biSizeImage     := 0;
    BmpInfoHeader^.biXPelsPerMeter := 0;
    BmpInfoHeader^.biYPelsPerMeter := 0;
    BmpInfoHeader^.biClrUsed       := 0;
    BmpInfoHeader^.biClrImportant  := 0;

    // if Bytes array is for monochrome image (channels = 1) normalizes bitmap color table
    // https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapinfoheader
    //
    // The BITMAPINFOHEADER structure may be followed by an array of palette entries or color masks.
    // The rules depend on the value of biCompression.
    //
    // - If biCompression equals BI_RGB and the bitmap uses 8 bpp or less, the bitmap has a color table immediately
    //   following the BITMAPINFOHEADER structure. The color table consists of an array of RGBQUAD values. The size
    //   of the array is given by the biClrUsed member. If biClrUsed is zero, the array contains the maximum number
    //   of colors for the given bitdepth; that is, 2^biBitCount colors.
    if Channels = 1 then
    begin
      for I := 0 to 255 do
      begin
        Buffer.ColorTable[I].rgbBlue := I;
        Buffer.ColorTable[I].rgbGreen := I;
        Buffer.ColorTable[I].rgbRed := I;
      end;
    end;

    LockWindowUpdate(Bitmap.Canvas.Handle);
    try
      Result := SetDIBitsToDevice
      (
        Bitmap.Canvas.Handle, // handle to device context
        0,                    // x-coordinate of upper-left corner of
        0,                    // y-coordinate of upper-left corner of
        Width,                // source rectangle width
        Height,               // source rectangle height
        0,                    // x-coordinate of Lower-left corner of
        0,                    // y-coordinate of Lower-left corner of
        0,                    // first scan line in array
        Height,               // number of scan lines
        Bytes,                // address of array with DIB bits
        BmpInfoBuffer,        // address of structure with bitmap info
        DIB_RGB_COLORS        // RGB or palette indexes
      ) > 0;
    finally
      LockWindowUpdate(0)
    end;
  except
    Result := False;
  end;
end;

CncVisionUpdateImageFromBytesArray permits RGB or Monochrome Bytes array.

In python the code to update image is simple:

import cnc_vision_ext as ext # this is only my delphi extension module for python where extra features for delphivcl are placed

# for monochrome frames to CameraImage (delphivcl.Image object)
        frame = get_frame_image_from_somewhere()
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        ext.update_image_from_bytes(
            self.CameraImage,
            frame_inp.tobytes(),
            frame_inp.shape[1],
            frame_inp.shape[0],
            3 if len(frame_inp.shape) == 3 else 1)
        self.CameraImage.Invalidate()

# for color frames to CameraImage (delphivcl.Image object)
        frame = get_frame_image_from_somewhere()
        ext.update_image_from_bytes(
            self.CameraImage,
            frame_inp.tobytes(),
            frame_inp.shape[1],
            frame_inp.shape[0],
            3 if len(frame_inp.shape) == 3 else 1)
        self.CameraImage.Invalidate()

Edited April 12, 2022 by shineworld

shineworld · April 12, 2022

Delphi Extension way vs Pillow using time.time_perf:

        # delphi module extension way
        t1 = time.perf_counter()
        ext.update_image_from_bytes(
            self.CameraImage,
            frame_inp.tobytes(),
            frame_inp.shape[1],
            frame_inp.shape[0],
            3 if len(frame_inp.shape) == 3 else 1)
        self.CameraImage.Repaint()
        t2 = time.perf_counter()

        # pillow way
        t3 = time.perf_counter()
        rgb_im: PIL_Image.Image = PIL_Image.fromarray(np.uint8(frame_inp[...,[2,1,0]].copy())).convert('RGB')
        self.CameraImage.Picture.Bitmap.SetSize(rgb_im.width, rgb_im.height)
        dib = PIL_ImageWin.Dib(rgb_im)
        dib.expose(self.CameraImage.Picture.Bitmap.Canvas.Handle)
        self.CameraImage.Repaint()
        t4 = time.perf_counter()

        # timings report
        self.Caption = 'EXT = {:7.3f} ms - PILLOW {:7.3f}'.format(t2 - t1, t4 - t3)

Delphi way take half time in this case 🙂

SwiftExpat · April 12, 2022

Thank you for taking the time to write all of that up and identify each part. It is always nice to have a real world sample to reference

shineworld · April 12, 2022

I've left only the upon described code in my extension so you can evaluate the code

to check if meets your needs.

cnc_vision_video_ext.7z

tomye · April 21, 2022

On 3/25/2022 at 9:31 PM, BennieC said:

Hi, we are doing some image processing and will use OpenCV for it. Doing this from Python is very useful as it disconnects us from the OpenCV libraries which we can now directly use. However, most processing is done with bitmaps. I have used the Demo29 and that works but it has the following limitations from our perspective.

It requires image in a TGraphic form which we don't get from a frame grabber and I find no documentation to describe either the structure of or how to convert a bitmap to it.

Internally we can convert the Byte stream with cv.imdecode into a opencv image and can convert our result image back to PILLOW

Returning the image however does not work unless the image is converted back to a PILLOW format from where demo29 code can display it.

I think part of my problem is that I don't have enough information regarding the PILLOW format or the TGraphic format. Also I don't find information about functions such as PyEval_CallObjectWithKeywords and ExtractPythonObjectFrom as the statement in demo29 returns a nil if I return any other format than PILLOW from Python.

Any assistance will be appreciated and if anyone can point me to support documentation describing how to use bitmaps in Python and also how to understand the many Python methods. I ahve been mostly a Delphi programmer.

An example of how we do it is as follows but this seems unnecessary and also requires input in TGraphic form, not TBitmap.

def LoadPicData(data):
# Convert data (image converted into Python Bytes) into a ocv image
stream = BytesIO(data)
ocv_img = cv.imdecode(np.asarray(bytearray(stream.read()), dtype=np.uint8), cv.IMREAD_COLOR)
pil_src = Image.open(stream)
# cv.imshow('ocv', ocv_img)
ocv_img = cv.cvtColor(ocv_img, cv.COLOR_BGR2GRAY)
# cv.imshow('gray', ocv_img)

# Convert this image to pil image
color_converted = cv.cvtColor(ocv_img, cv.COLOR_BGR2RGB)
pil_img = Image.fromarray(color_converted) # convert ocv to PIL image
pil_img.format = pil_src.format
# ImageShow.show(pil_img, 'PIL')
return pil_img

Some assistance will be much appreciated.

don't know much about your project, but i have same problem with you,

my thinking is DELPHI just send an image to Python, after the running, Python sending back not an Image just an array data

if it works well then it will resolve a big problem of mine, because i am doing a machine vision, using neural network, as we

know the DELPHI is so weak on this kind of field, it's not DELPHI's problem as Python is too hot now, everything are around of Python

actually, i feel the Python grammar is not good for me, it's too unprecise, i can't fit it.

however, i met a strange exception, i am testing it on DEMO29 too, but got an error after the fist pressing of the button

why i said it was strange because when i click the "Execute" button again, everything was OK

do you have this problem?

shineworld · April 21, 2022

The way to manage FPU exceptions in DELPHI is not the same on Python so you have to add this in your Delphi unit:

initialization
  MaskFPUExceptions(True);

PS: Actually I've moved from PIL to SKIA in image management.
Skia offers more features than Pillow.

import skia
		
...
.
...

# frame_inp is a numpy containing RGB image

# convert numpy image from RGB to RGBA (add AlphaBlend layer)
skia_frame = cv2.cvtColor(frame_inp, cv2.COLOR_RGB2RGBA)
        
# create a skia image from numpy RGBA array
skia_image = skia.Image.fromarray(skia_frame)

# sample of image resize with skia
image_ratio = 1.0
image_width = skia_image.width()
image_height = skia_image.height()
camera_image_width = self.CameraImage.Width
camera_image_height = self.CameraImage.Height

r_w = image_width / camera_image_width
r_h = image_height /  camera_image_height
if r_w > 1.0 or r_h > 1.0:
	image_ratio = max(r_w, r_h)
    image_width = int(image_width / image_ratio)
    image_height = int(image_height / image_ratio)
    skia_image = skia_image.resize(image_width, image_height, skia.FilterQuality.kLow_FilterQuality)
    skia_frame = skia_image.toarray()

# sample of antialiasing drawings on skia
paint = skia.Paint(AntiAlias=True, Color=skia.ColorRED)
canvas = skia.Canvas(skia_frame)
canvas.drawLine(0, int(image_height / 2), image_width, int(image_height / 2), paint)
canvas.drawLine(int(image_width / 2), 0, int(image_width / 2), image_height, paint)

# back skia image to numpy array RGB
skia_frame = cv2.cvtColor(skia_frame, cv2.COLOR_RGBA2RGB)

# draw of numpy array RGB to delphi Image
ext.update_image_from_bytes(
	self.CameraImage,
	skia_frame.tobytes(),
	skia_frame.shape[1],
	skia_frame.shape[0],
	3 if len(skia_frame.shape) == 3 else 1
)
self.CameraImage.Repaint()

tomye · April 21, 2022

34 minutes ago, shineworld said:

The way to manage FPU exceptions in DELPHI is not the same on Python so you have to add this in your Delphi unit:


initialization
  MaskFPUExceptions(True);

PS: Actually I've moved from PIL to SKIA in image management.
Skia offers more features than Pillow.


import skia
		
...
.
...

# frame_inp is a numpy containing RGB image

# convert numpy image from RGB to RGBA (add AlphaBlend layer)
skia_frame = cv2.cvtColor(frame_inp, cv2.COLOR_RGB2RGBA)
        
# create a skia image from numpy RGBA array
skia_image = skia.Image.fromarray(skia_frame)

# sample of image resize with skia
image_ratio = 1.0
image_width = skia_image.width()
image_height = skia_image.height()
camera_image_width = self.CameraImage.Width
camera_image_height = self.CameraImage.Height

r_w = image_width / camera_image_width
r_h = image_height /  camera_image_height
if r_w > 1.0 or r_h > 1.0:
	image_ratio = max(r_w, r_h)
    image_width = int(image_width / image_ratio)
    image_height = int(image_height / image_ratio)
    skia_image = skia_image.resize(image_width, image_height, skia.FilterQuality.kLow_FilterQuality)
    skia_frame = skia_image.toarray()

# sample of antialiasing drawings on skia
paint = skia.Paint(AntiAlias=True, Color=skia.ColorRED)
canvas = skia.Canvas(skia_frame)
canvas.drawLine(0, int(image_height / 2), image_width, int(image_height / 2), paint)
canvas.drawLine(int(image_width / 2), 0, int(image_width / 2), image_height, paint)

# back skia image to numpy array RGB
skia_frame = cv2.cvtColor(skia_frame, cv2.COLOR_RGBA2RGB)

# draw of numpy array RGB to delphi Image
ext.update_image_from_bytes(
	self.CameraImage,
	skia_frame.tobytes(),
	skia_frame.shape[1],
	skia_frame.shape[0],
	3 if len(skia_frame.shape) == 3 else 1
)
self.CameraImage.Repaint()

it does work, thank you guy

BTW, my idea for my project is , i use FFMpeg engine on DELPHI to open any formatted video media , includes every frames of these video

and send the frame (it's an image actually) to PYTHON, the python codes will run the neural network algorithm and calculate the result

and then these results will be sent back to DELPHI (results are arrays data ) , then i can draw the result on bitmap and show it to users.

in the passed of months, i even don't know the P4D, so i spent a lot of time to do other ways , all failed until find this forum.

seems just spending a couple of times it might be successful, but i don't know how to return arrays to DELPHI at now, do you have any samples?

i defined these arrays in Python like this:

image.png.c52ea0e089d2d94ce4bb6119853b5b98.png

does the P4D support it sending to DELPHI ?

best regards

Sign In

Transferring TBitmap to Python and back

Recommended Posts

BennieC 0

Share this post

Link to post

shineworld 68

Share this post

Link to post

BennieC 0

Share this post

Link to post

BennieC 0

Share this post

Link to post

shineworld 68

Share this post

Link to post

BennieC 0

Share this post

Link to post

BennieC 0

Share this post

Link to post

shineworld 68

Share this post

Link to post

shineworld 68

Share this post

Link to post

shineworld 68

Share this post

Link to post

SwiftExpat 65

Share this post

Link to post

shineworld 68

Share this post

Link to post

tomye 1

Share this post

Link to post

shineworld 68

Share this post

Link to post

tomye 1

Share this post

Link to post

Create an account or sign in to comment

Create an account

Sign in

Browse

Activity