Jump to content

SwiftExpat

Members
 View Profile  See their activity

  • Content Count

    222

  • Joined

  • Last visited

  • Days Won

    9

Posts posted by SwiftExpat

  1. Posted

    My guesses are:

    1.  TCP connection went into timewait on the client.  Netstat is the only tool that I am aware will show you this, so you need to wrap it in loop and capture the output.
    2. Router killed it to conserve memory.

    The registry reference you are looking for is here (older version, but you can adjust):  https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc757512(v=ws.10)

     

    It is a mix of timed wait and keep alive that you try to tune.  I have only had mixed success trying to keep long db connections open, the keepalives are often killed at the firewall / router level.  A lot of modern network equipment tracks those connections and kills them to save router memory.

     

    I help it helps you to get some insight.

    • Like 1

  4. Posted

    8 hours ago, tomye said:

    it should be OK,

    When you use python embedded, you have to be aware of the OS system path.  Assume that nothing is on the OS path and you will be ok.

     

    I choose not to modify the path as part of my app install, and python is in a subdirectory to my exe.  This allows me to delete the python directory when I need to update.  This forces me to load everything relying only on site to find the module.

     

    8 hours ago, tomye said:

    i just copy c:\programdata\anaconda3\lib\site-packages\cv2 ; c:\programdata\anaconda3\lib\site-packages\onnxruntime to {my project} \Libs\ 

    I install pip in the embedded distribution and let pip install the packages and dependencies. 

    • Like 1

  8. Posted

    This might take a lot of code to accomplish, but this would be my goal before going multiprocess.  Keep in mind thread construction has a cost and IPC will always have a high cost.

     

    The code works fine in my usage, the thread hosts the engine and stays in a loop. No reason to use TPythonThread, you are not creating a thread in Python.

     

    In your example you have the thread performing several tasks, split them. The blocking point is Python, so the goal is to keep a steady stream of work going to it.  This is the limit you design around.

     

    The thread that is executing the python should only be watching a work queue, TThreadedQueueCS. Handle all the synchronization in Delphi.

      

    procedure TThreadProcess.Execute;
var
  v: Variant;
  itm: TItem;
begin
  inherited;
  do while not terminated
  begin
  itm :=  QSessionAnalyze.PopItem;
  v := MainModule.ProcessImage(ImageToPyBytes(BMP));  
  QSessionDraw.PushItem(itm)
  end;
end;

    Could all of this other could happen in another thread or at the time it is rendered. 

    procedure TThreadDraw.Execute;
var
  v: Variant;
begin
  inherited;
  DT.Init(true);
  DT.FillObjsData(v,0,0);
  DT.DrawObjects(BMP.Canvas,1280);
  DT.DrawSpeedInfo(BMP.Canvas,'NA',1280);
  FitDrawEx(Img,BMP);
  Img.Invalidate;
end;

     

    11 hours ago, tomye said:

    procedure TThreadDetect.Execute;
    var
      v: Variant;
    begin
      inherited;
      DT.sTime:=GetTickCount;
      DT.Init(true);

     

      v := MainModule.ProcessImage(ImageToPyBytes(BMP));   // dead in here , no any response

     

      DT.FillObjsData(v,0,0);
      DT.DrawObjects(BMP.Canvas,1280);

      DT.eTime:=GetTickCount;
      DT.Elapsed:=DT.eTime-DT.sTime;

      //画帧率和速度信息
      DT.DrawSpeedInfo(BMP.Canvas,'NA',1280);

      FitDrawEx(Img,BMP);
      Img.Invalidate;

    end;

     

  9. Posted

    It looks to me like your thread is doing processing and display, can you seperate it?

     

    Have you considered running PYEngine on a thread instead of main? Create a work queue on the thread and only have it execute the python steps necessary, then send the instructions to draw the result back to Delphi for display.

     

    My thread looks like this and I use DEB for messaging between main and the python thread. 

    type
  TSERTCPythonEngine = class(TThread)
  private
    PE: TPythonEngine;
    PythonIO: TPythonInputOutput;
    PythonResultVar: TPythonDelphiVar;
    PyMod: TPythonModule;
    QSessionAnalyze: TThreadedQueueCS<IEventSessionAnalyze>;
    function FilePathPython(AFilePath: string): string;
    procedure PythonIOReceiveData(Sender: TObject; var Data: AnsiString);
    procedure PythonIOReceiveUniData(Sender: TObject; var Data: string);
    procedure PythonIOSendData(Sender: TObject; const Data: AnsiString);
    procedure PythonIOSendUniData(Sender: TObject; const Data: string);
    procedure StartEngine;
    procedure SetInitScript;
    function InstallVerify: boolean;
    procedure InstallPython(AUpdateLocation: string);
    procedure InstallRttkPackage(AUpdateLocation: string);
    procedure InstallDependDLLs;
    procedure InstallDLLCopy(ADllName: string);
    procedure ProcessSessionAnalyze;
    procedure PySessionAnalyze(ASessionAnalyze: IEventSessionAnalyze);
    procedure ProcessSshHostReload;
    procedure PyModExecHostConfigFound(Sender: TObject; PSelf, Args: PPyObject; var Result: PPyObject);
    procedure PyModExecHostConfigUpdated(Sender: TObject; PSelf, Args: PPyObject; var Result: PPyObject);
  protected
    procedure Execute; override;
  public
    class function PythonDll: string;
    class function PythonDir: string;
    class function PyPkgDir: string;
    class function PythonExists: boolean;
    constructor Create;
    destructor Destroy; override;
    [Subscribe(TThreadMode.Background)]
    procedure OnEventSessionAnalyze(AEvent: IEventSessionAnalyze);
  end;
  
  
  
  
  procedure TSERTCPythonEngine.Execute;
begin
  inherited;
  NameThreadForDebugging('THPythonEngine');
  try
    try
      CoInitialize(nil);
      if InstallVerify then
        if PythonExists then
        begin
          StartEngine;
          while not Terminated do
          begin
            ProcessSessionAnalyze;
            sleep(10);
          end;
        end
        else
          Logger.Critical('Python does not exist')
      else
        Logger.Critical('Python install verifiction failed!');
    finally
      CoUninitialize();
    end;
  except
    on E: Exception do
      Logger.Critical('Python Exectue failed with ' + E.Message);
  end;

end;



procedure TSERTCPythonEngine.ProcessSessionAnalyze;
var
  lEsa: IEventSessionAnalyze;
begin
  if (not QSessionAnalyze.ShutDown) and ((QSessionAnalyze.TotalItemsPushed - QSessionAnalyze.TotalItemsPopped) > 0) then
  begin
    lEsa := QSessionAnalyze.PopItem;
    if TFile.Exists(lEsa.SessionFileName) then
      PySessionAnalyze(lEsa);
    GlobalEventBus.Post(lEsa, 'Processed');
  end;
end;

     

  10. Posted

    Two items that might help you get a focused answer:

    1. Collection based, virtual or using a TDataset?
    2. What performance (insert, sort, scroll, filter) are you concerns?

    TMS has 2 for FMX, so make sure you evaluate the FNC one not the legacy FMX one. It is very clear on the page which one to look to.

  15. Posted

    With some hesitation, as this creates its own set of challenges, I will open this for discussion. ( I doubt I am the first to suggest / think this )

    Since we are developers here, we could write an expert that logs the data. Putting aside the privacy concern, though it is valid and will have to be addressed,

    What data would need to be collected? What analysis would be usefull after collecting the data?

     

     

  16. Posted

    Ian,

    On the multiple versions, it might help to collect daily usage vs occasional.

    One situation would be  I use  delphi 11  95% of the day on active projects.

    Another is I occasionally support old apps on 10.4, 10.3, 10.2

     

    I would almost suggest to force them to specify 1 daily version.

     

     

  17. Posted

    5 minutes ago, aehimself said:

    I don't know why I want to minimize the amount of queries though as during the actual update

    The tradeoff is between retries for failed / interrupted transfers. Probably better to add a robust retry mechanism, failures will happen even with a good connection.

  18. Posted · Edited by SwiftExpat
    Benchmark comment

    14 minutes ago, aehimself said:

    it won't tell me though if the project has new files / versions which I am interested in. 

    I use a delta file to give me that information, so it is always a 2 step approach. It helps drive some of my update logic, you would have to judge the savings for your case.

     

    As a bench mark, because you are already doing most of what this would do for you,

    Commerically a new product from TMS, I have not tried it, only read the docs:
    https://www.tmssoftware.com/site/tmsfncapptools.asp

  19. Posted

    One other item you should analyze is what size JSON is actually transferred by the server.  Most web servers can zip that file before transfer, so size on disk does not necessarily equal size transferred.

    Would you be able to post some sizes so people can comment with a little more perspective?

  20. Posted

    19 minutes ago, aehimself said:

    minimize network traffic.

    What is the concern here, number of requests per hour or amount of data transferred? Size might dictate solution, but basic server load balancing would be an option.  Do you have a bandwidth limit at the hosting provider?

     

    23 minutes ago, aehimself said:

    Protocol. Should I really stick to HTTP, or is there a better / create my own?

    HTTPS for easy proxy support. Also if you use a HEAD request you can get the file info from the server with 0 bytes transferred, supports your minimize traffic requirement.

  21. Posted

    Fellow Delphi Developers
    RunTime ToolKit Marshal version 2022.4.24 is available.


    Marshal Updates:

    1. Object inspector is now Object Plus . Demo Video here for this learning exercise
    2. Inspectors for TDataset and TFDConnections are in this demo:

    Follow along with this playlist to get an overview of inspection at runtime.


    Follow this link for price info
    For Documentation take a look at the Wiki
     
    Thank you for looking and I welcome feedback
    SwiftExpat
    https://swiftexpat.com

    • Like 1

  22. Posted

    2 hours ago, Uwe Raabe said:

    It seems that these plugins don't have a high enough priority to get the necessary resources attached

    Agreed. But they could also survey the features used and trim the stack of items that need to work.
    My top annoyance with the built in bookmarks is that I overwrite them when I use the key strokes.  And yes I am tempted to write my own stack history to track this and shuffle accordingly, but I think the implementation in the Parnassus plugin was good so I hold out.

  24. Posted

    The result of BeamSquare is a tuple string 'Quadrate Square', 1125 . You are not setting a value, 1125 to a variable. so when you read BeamSquare.Value it is evaluating a( tuple * 55) not (1125*55).

     

    I expect if you set a breakpoint on S in QuadrateBeam_Value and then inspect BeamSquare.value it will show you more detail.

     

    Could be wrong, but it is my best guess from the screen shots.

×