Jump to content

mikerabat

Members
 View Profile  See their activity

  • Content Count

    36

  • Joined

  • Last visited

  • Days Won

    1

Posts posted by mikerabat

  1. Posted · Edited by mikerabat

    I'm not sure about exactly this library but some predecessors indeed were able to sort pointer list whereas the "key" is a float/int 32/64 . The key can be specified as an "offset" to the

    starting address -> this allows to sort pointers to records/classes....

    at least that is the way they do it in "Fast quicksort Implementation using AVX Instructions"

     

    The definitions in the library for AVX512 are actually using arrays of templates so I guess that means that you can sort records right not only Double/float... ?

     

    On 11/3/2023 at 1:23 PM, DJSox said:

    In statistics and/or data analysis to calculate the median you have to sort the data, at least I always have. When plotting data, say in an xy plot, sometimes it's better to sort the data by x first. So, in technical fields it's not that uncommon.

    This problem is solved by an algorithm that allows to extract the kth largest element (not only the median) -> basically it does the partitioning of quicksort but does not sort hte "unnecessary" half. That in fact allows to pick the median in O(n) (roughly) time....

     

    And if you want to go fancy you can do a rolling median over an array of doubles. The algorithm can be found here:
    https://github.com/mikerabat/mrmath/blob/master/RollingMedMean.pas

     

     

  4. Posted

    Jus FYI: That is intentional - the left side shall show the original assembler call the right side the DB translation

    -> my tool, that derrives the DB statements from the assembler code does a line by line comparison.

     

  7. Posted

    Sorry... the tab key is broken or at least behaves completly different than in Delphi 2010 (or afaik Delphi 10.4) which is ... anoying.

    e.g.:

    Consider the following code

     Result := Format('%s;%s;%s', [
                               DateToStr( fDay, Fmt),
                               TimeToStr( fFrom, Fmt ),
                               TimeToStr( fTo, Fmt )  ]);

     

    when you place the cursor just behind the '[' bracket and then press enter the cursor is placed just below the R of Result... so far so good.

    But... In previous Delphi versions - when tab is invoked - the cursor jumps to the ":" then to "F" of format. Now the tab just moves the cursor

    by 2 (acording to the setting) which is actually anyoing.

     

    Another case: Consider the following code (we like to have the then else block that way in case the statement is only one line aka without begin end):

    if fLunchLen > 0.01
    then
           Result := Result + Format(';%s;%.1fh', [TimeToStr(fLunch, Fmt), fLunchLen], Fmt)
    else
          Result := Result + ';;'; // no free lunch... it bloody friday!

     

    When you place the cursor behind then, press enter -> press tab in previous delphi versions the cursor was placed just right below the then... now it only jumps by 2.

     

    any comments on how to get that old behaviour back is highly appreciated...

     

    kind regards

      Mike

  9. Posted · Edited by mikerabat

    You could do something like CBOR encoding does. Basically there they encode values based on the NEEDED space.

    If integer then encode it in an int8 - to int16. You can also check if the encoding would fit into a 16bit float (half a single)

     

    In addition it seems that a differential approach for each block could be feasable ... does it change much??

    And of course at the end just compress the stream 😉

  11. Posted

    Hey! We actually use the following code to get the system serial - maybe that can help:
      

    function GetSystemSerial : string;
var FSWbemLocator : OLEVariant;
  FWMIService   : OLEVariant;
  FWbemObjectSet: OLEVariant;
  FWbemObject   : OLEVariant;
  oEnum         : IEnumvariant;
  iValue        : LongWord;
begin
     CoInitialize(nil);
     try
        result:='';
        FSWbemLocator := CreateOleObject('WbemScripting.SWbemLocator');

        // ATT: we had a case, where on a Win2016 server, the system service failed to
        //      execute this command. Changing the service to a local user solved the problem.

        // There has been 2 cases where the WMI service was corrupted or was not running.
        // -> to check: enter WmiMgmt.msc in the console and right click on the local wmi - control -> Properties
        //    if an error occurs the repository seems to be corrupt
        // one time it helped to reset and rebuild the repository:
        // https://techcommunity.microsoft.com/t5/ask-the-performance-team/wmi-rebuilding-the-wmi-repository/ba-p/373846
        try
           FWMIService   := FSWbemLocator.ConnectServer('localhost', 'root\CIMV2', '', '');
        except
              on e : Exception do
              begin
                   OutputDebugString(PChar('WMI service not properly running: '+e.Message+#13#10+
                     'https://techcommunity.microsoft.com/t5/ask-the-performance-team/wmi-rebuilding-the-wmi-repository/ba-p/373846'));
                   raise;
              end;
        end;

        // todo: if we still support winxp this wmi class is not supported?!?
        FWbemObjectSet:= FWMIService.ExecQuery('SELECT UUID FROM Win32_ComputerSystemProduct','WQL',$00000020 {wbemFlagForwardOnly});
        oEnum         := IUnknown(FWbemObjectSet._NewEnum) as IEnumVariant;
        if oEnum.Next(1, FWbemObject, iValue) = 0 then
           result:=String(FWbemObject.UUID);
     finally
       CoUninitialize;
     end;
end;

     

    There is also another place where you could get some unique id:
      

    // from http://www.nextofwindows.com/the-best-way-to-uniquely-identify-a-windows-machine
     with OpenHKLMReadOnly('SOFTWARE\Microsoft\Cryptography', True) do
     begin
          AktMachineGuid:=ReadString('', 'MachineGuid', '');
          Log('Machine guid: ' + AktMachineGuid);
          Free;
     end;

     

    hope that helps!

     

    kind regards

      Mike

    • Like 2

  12. Posted

    I'm puzzled... Since Delphi 11 the tab key does not work as it has been the case in all

    the previous versions... I tried to setup the editor options the same way as my Delphi 2010 (or 10.4)

    installation but the tab key behaves differntly....

     

    What I actually want is that when I press tab the cursor shall be placed after the last non whitespace from the previous line

    e.g.
    if you have

    var i : integer;

    begin

    |<- cursor here

    now when pressing tab the cursor shall jump the character after the n

     

    Also in Delphi 2010 it was standard if:

     

     if abc then

     begin

             while i < 10 do

             begin

    |<- e.g.: cursor here

             end;

     

     

    now the first tab jumps just right below the b of begin, a second tab to the end of begin

     

    That was standard for us for up until Delphi 10 ... but now in Delphi 11 I cannot get this working.

    it either works or it jumps the number of characters that has been defined in the TabStop field.

     

    anyone has a clue what there coudl be the problem?

     

    A also have DDevextensions and gexperts installed....

     

    kind regards

       Mike

     

     

  14. Posted

    Thanks Remy!!

     

    Right... the call to BeginThread actually suceeds - the callstack clearly indicates that it's the ResumeThread call that fails.

    So... the only thing I found online was that there is a right  " THREAD_SUSPEND_RESUME " which may have

    some influence here... But I have absolutely no idea what could change that.

    Note that the code actually works on around 8k of installations but there are only a few (2 for now) that seem to

    make trouble in that regard. The thing is that if there are way fewer files to be loaded (e.g. 6-8) it works most of the time

    (aka not always)... Also the system worked for 2 years now without any problems... so I'm quite stunned that now something pops up.

     

    So... Is there any 3rd party or any configurable restrictions on Windows that disallow something like releasing many threads at once?

     

    kind regards

      Mike

     

  15. Posted

    Hi all!

     

    In our system we load a bunch of files on startup in parallel so for each of these files we craete a thread

    that does that in the background.

     

    basically this is what how it is done:

    TEVentChannel is the data object that holds the data

    -> on the data object creation a thread (TEvtLoadThread = class(TThread)) is created which does the loading

    -> of course it is ensured that data access and loading is mutal exclusive.

    The snippset of code that is involved is:

      

    constructor TEventChannel.TEvtLoadThread.Create(parent: TEventChannel);
begin
     fParent := parent;

     inherited Create(False);
end;

procedure TEventChannel.TEvtLoadThread.Execute;
begin
     NameThreadForDebugging('Load '+AnsiString(fParent.ChanName));
     // ###########################################
     // #### Just start loading the data
     try
        fParent.Reload;
     except
           on E : EChannelException do
           begin
                fParent.fLoadFailed := False;
                fParent.fDataLoaded := lsBadRecId;
           end;
     else
         // damaged file??
         fParent.fDataLoaded := lsAll;
         fParent.Clear;

         fParent.fLoadFailed := True;
     end;
end;

     

    My problem here is that on a system on which that code ran for over 2 years without problems now happens to fail with the error

    "Cannot call Start on a running or suspended thread"

    This error is issued in the TThread.Afterconstruction call and it seems that our system fails to create that thread...

     

    Looking at the TThread code it is most likely that the Thread could not be created so what on earth could be could interfere here?

     

    We tried to disable FSecure (or at least they claim no security system is monitoring our program) and Windows Defender without any change.

    Is there anything known out there that could limit an external program in such way?

     

    kind regards

      Mike

     

     

     

  16. Posted

    Short answer ... yes - if you got simultaneously more connections from clients than you have requests handles in Delphi then an

    exception will be raised... Actually I find that property very stupid at all... e.g. apache creates on startup those 50 threads beforehand

    waiting for incoming connections. On each request a WebModule object will either be taken from the cache or created and put into the

    cache so this number should be either growing to the demand or be set to the number of the maximum active connections

    the server is allowed to serve.

  17. Posted

    40 minutes ago, JoEmbedded said:

    Hello Mike, thank you very much for Your offer. But I think I wait (and hope Embarcadero will migrate to 64-Bit, I started in 1988 with Turbo-C, so no hurry...).

    In the meantime I'll continue my work on the Progressive-Web-APP (still a lot 2do...) Best regards, Jo

    Hey Jo!

     

    Actually there is already a 64bit compiler so.... I don't know what you are waiting for 😉
    anyway.... the probelm is not the compiler but rather the api used aka WinRT and the fixes are

    put in there in Delhi11 . So either update to this version or check the fixes.

    I can remember that I also put them on stackoverflow: https://stackoverflow.com/questions/61890445/delphi-bluetooth-le-heart-rate-profile-access

     

  18. Posted

    Actually the fix I posted to Embarcadero was exactly that and a few things more. Basically they used

    an outdated API and needed pairing.

    The code there should fix that. Let me know if you can find it - if not let me know. Maybe I can send you the

    fixed Windows BlueTooth LE stuff directly (I guess I may not post stuff here 😉 )

     

    kind regards

      MIke

  20. Posted

    Strange enough on a second installation (aka our test site) the OnWork property works as expected... uploading around 1MByte per second

    and not waiting at all at the end. It looks like there is some kind of proxy in between that first receives the full message - relays that to the

    actual server - and waits until something comes back... I have though never heard of something like this and they couldn't clarify that either...

    Is there actually some kind of hardware around that could reproduce such a behaviour?

     

  21. Posted · Edited by mikerabat
    Added code to clarify

    Dear all!

     

    Our Software supports new uploading large files (100MB + ) to a server. To get a nice feedback we wanted

    to have some kind of progress telling the user how far we are already with the upload.

    For the upload we use the TidHTTP component + an openssl io handler.

     

    To get the progress working we used the OnWork event that is published by the component.

    While we were using the component for testing in our internal network (which is quite fast) the

    progress seemed ok but when we tested that on a slow network (around 2MBit) we saw that the first 99% were

    very fast (20MB per second +) and then the last bytes "hang" until the full file was transmitted. Basically the

    progress bar showed a progress of 99% for around 4minutes until the file was finally uploaded.

     

    What am I missing here? How can I determine how much data is transmitted to the server?

     

     

    The code involved is:
      

    function CreateLibertyHTTP( method : TIdSSLVersion = sslvTLSv1_2; proxySet : ISuperObject = nil ) : TIdHTTP;
var sslIOHandler : TIdSSLIOHandlerSocketOpenSSL;
begin
     Result := TIdHTTP.Create(nil);
     sslIOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(Result);
     sslIOHandler.SSLOptions.Method := method;
     
     Result.IOHandler := sslIOHandler;

     if (proxySet <> nil) and (proxySet.S['Server'] <> '') then
     begin
          Result.ProxyParams.ProxyServer := proxySet.S['Server'];
          if proxySet.I['Port'] <> 0 then
             Result.ProxyParams.ProxyPort := proxySet.I['Port'];
          if proxySet.s['Username'] <> '' then
          begin
               Result.ProxyParams.ProxyUsername := proxySet.s['Username'];
               Result.ProxyParams.ProxyPassword := proxySet.s['Pwd'];
          end;
          Result.ProxyParams.BasicAuthentication := proxySet.B['BasicAuthentication'];
     end;
end;
    
httpReq := CreateLibertyHTTPReq( some params here );
DL.LLog(dlInfo, 'Liberty upload');
                         libertyParams.Clear;
                         libertyParams.Add('SID=' + SessionId);
                         libertyParams.Add('blockearly=1');
                         libertyParams.Add('csrfToken=' + csrfToken);

                         // eg 'http://192.168.0.193:81/tom24x/uploadFile?SID=45784095&blockearly=1'
                         s := httpReq.Post(LibServer+'tom24x/uploadFile', libertyParams, IndyTextEncoding_UTF8);
                         if Pos('error', LowerCase(s)) > 0 then
                            raise Exception.Create('Error initializing upload');

                         uplFileResp := SO(s);

                         fUploadFileSize:=FileSizeByName(DawFileName);
                         //s:='?SID='+SessionId+'&blockearly=1&overwrite=1';  // overwrite recording in case it's already uploaded
                         Stream := TStringStream.Create('');
                         Params := TIdMultipartFormDataStream.Create;
                         httpReq.OnWork:=LibertyUploadProgress;
                         try
                            Params.AddFile('pdf', DawFileName, 'application/octet-stream');
                            // New in 2.9.3: The Uploadfile issues a token which can be used as session id (one time thing)
                            // so the param does no harm at all.
                            s := LibServer+'rec/UploadRecording.exe?SID=' + uplFileResp.S['session'] + '&blockearly=1&overwrite=1';
                            // e.g. 'http://192.168.0.193:81/rec/UploadRecording.exe?SID=45784095&blockearly=1&overwrite=1'
                            try
                               httpReq.Post(s, Params, Stream);
                            except
                                  on E: Exception do
                                     raise Exception.Create('Sending DAW file failed: ' + E.Message);
                            end;
                            DL.LLog(dlInfo, 'Liberty says: '+Stream.DataString);

      

    procedure TfrmDBBrowser.LibertyUploadProgress(ASender: TObject; AWorkMode: TWorkMode;
  AWorkCount: Int64);
function BytesToStr( numBytes : Int64 ) : string;
begin
     if numBytes > 1000000 then
     begin
          Result := Format('%.2f MB', [numBytes/(1000*1000)]);
     end
     else if numBytes > 10000 then
     begin
          Result := Format('%d kB', [numBytes div 1000]);
     end
     else
         Result := intToStr( numBytes ) + ' B';
end;
begin
     if fSxProgress.Cancelled
     then
         Abort
     else
     begin
          // note: the upload file size may be a bit higher than the actual file size due to header overhead
          fSxProgress.lblTopLabel.Caption := fLibertyUplLabel + ' ' + BytesToStr( Min(fUploadFileSize, AWorkCount ) ) + '/' + BytesToStr( fUploadFileSize );
          fSxProgress.lblTopLabel.Update;
          fSxProgress.SetProgress(-1, MulDiv(AWorkCount, 100, fUploadFileSize));
     end;
end;

     

    kind regards

       Mike

     

     

  22. Posted

    17 hours ago, DelphiUdIT said:

    The WIN32 apis for BlueTooth are system based and are present in:

     

     rtl/net/System.Bluetooth

     rtl/net/System.Bluetooth.Components

    That is only the one half of the story. This is the base - in the next abstraction layer Emba decided to go to the different platforms -

    and for windows and Bluetooth LE the last file that actually implements stuff is System.Win.BluetoothWinRT.pas

     

     

    17 hours ago, DelphiUdIT said:

    Some difference with use of BLE is that with Windows the devices must be paired almost one time to be used with BLE. 

    That is also not entierly true. The imlementation from Embarcadero lacks this ability yes - I managed though to extend the WinRT implementation such that this

    is possible too.

    See the attached file on:
    https://quality.embarcadero.com/browse/RSP-21711

     

    hope that helps 😉

    • Like 1

  23. Posted

    1 hour ago, DelphiUdIT said:
    I don't know the WinRT API and have never developed a UWP application.
    I would have created an external timer (a normal TTimer) and called the CancelDiscovery function without "touching" the BLE timer.
     
    In normal WIN32 api I do this and I have no problems whatsoever.

    And how do you do that in win32? I wasn't able to find any BT LE classes or functions besides the WinRT stuff that is covered in Delphi.

    Please note that my code here patches the actual TBluetoothLE component that ships with Delphi 10.4.2!

    And yes I use this code in conjunction with a standard VCL app 😉

  24. Posted

    I ran into the same problem and managed to fix that with an external timer that calls CancelDiscovery. It works but leaves a massive

    memory/thread leak since every time a new discovery is started the timer variable gets overwritten and not freed leaving the pending thread and

    it's resources....

     

    The problem actually is:

    the problem is within the Delphi implementation of the Thread that triggers the CancelDiscovery function.

    In Windows there is a Thread (TWinRTBluetoothLEAdapter.TDiscoverThreadTimer in System.Win.BluetoothWinRT.pas) that

    actually just waits for a specific amount of time and calls the given timer procedure. The problem here is that the

    Timer procedure cleans up the thread (calls .Free) which waits for the end of the Thread -> deadlock.

     

    To circumvent that problem I needed to adjust System.Win.BluetoothWinRT do the following:

     

    Augment the class

    TWinRTBluetoothLEAdapter

    by

     

    private
        const WM_BTLE_CANCELDISCOVERY = $400 + 666; // WM_USER

    ...

    private

      fHDL : HWND;
      procedure BTLETimerHandler( var msg : TMessage );

    protected

      procedure ThrDoCancelDiscovery;

     

    ..

    implementation

     

    procedure TWinRTBluetoothLEAdapter.BTLETimerHandler(var msg: TMessage);
    begin
         if msg.Msg = WM_BTLE_CANCELDISCOVERY then
            DoCancelDiscovery;
    end;

     

    procedure TWinRTBluetoothLEAdapter.ThrDoCancelDiscovery;
    begin
         // just post otherwise the thread hangs!
         PostMessage( fHDL, WM_BTLE_CANCELDISCOVERY, 0, 0);
    end;

     

    constructor TWinRTBluetoothLEAdapter.Create(const AManager: TBluetoothLEManager; const ARadioInfo: TBluetoothRadioInfo);

    begin

      inherited Create(AManager);
      fHDL := AllocateHWnd(BTLETimerHandler);

    ...

    end;

     

    destructor TWinRTBluetoothLEAdapter.Destroy;
    begin
      DeallocateHWnd(fHDL);

    ...

    end;

     

    function TWinRTBluetoothLEAdapter.DoStartDiscovery(Timeout: Cardinal; const AFilterUUIDList: TBluetoothUUIDsList;
      const ABluetoothLEScanFilterList: TBluetoothLEScanFilterList): Boolean;

    begin

    ...

     

    // new code

    if Assigned(fTimerThread) then
      begin
        fTimerThread.Cancel;
        fTimerThread.Free;
      end;

    // changed from DoCancelDiscovery to ThrDoCancelDiscovery

      FTimerThread := TDiscoverThreadTimer.Create(Self, ThrDoCancelDiscovery, Timeout);

    ...

    end;

     

    I guess one could also use TThread.Queue instead of the "complicated" window allocation in the timer and that should do the trick too!

  25. Posted

    On 6/25/2021 at 9:06 PM, Remy Lebeau said:

    The main problem with TEncoding.GetBytes() is that its 'public' overloads all require the output to be a TBytes array, which you would then have to copy into your record afterwards, eg: 

    
var
  patData : TPatData;
  SurName : String;
  Bytes: TBytes;
begin
  FillChar(@patData, sizeof(patData), 0);
  SurName := ...;
  Bytes := TEncoding.UTF8.GetBytes(SurName);
  Move(PBytes(Bytes)^, patData.SurName[0], Math.Min(SizeOf(patData.SurName), Length(Bytes)));
  ...
end;

    The one overload of GetBytes() that would actually let you output directly into your record without using TBytes is declared as 'strict protected', which means you can't use it without involving some hacks, eg: 

    
type
  TEncodingHelper = class(TEncoding)
  public
    function GetBytes(const S: string; Bytes: PBytes; ByteCount: Integer): Integer;
  end;

function TEncodingHelper.GetBytes(const S: string; Bytes: PBytes; ByteCount: Integer): Integer;
begin
  Result := GetBytes(PChar(S), Length(S), Bytes, ByteCount);
end;

var
  patData : TPatData;
  SurName : String;
begin
  FillChar(@patData, sizeof(patData), 0);
  SurName := ...;
  TEncodingHelper(TEncoding.UTF8).GetBytes(S, PByte(@patData.SurName[0]), SizeOf(patData.SurName));
  ...
end;

    Or: 

    
type
  TEncodingHelper = class helper for TEncoding
  public
    function GetBytes(const S: string; Bytes: PByte; ByteCount: Integer): Integer;
  end;

function TEncodingHelper.GetBytes(const S: string; Bytes: PByte; ByteCount: Integer): Integer;
begin
  Result := Self.GetBytes(PChar(S), Length(S), Bytes, ByteCount);
end;

var
  patData : TPatData;
  SurName : String;
begin
  FillChar(@patData, sizeof(patData), 0);
  SurName := ...;
  TEncoding.UTF8.GetBytes(S, PByte(@patData.SurName[0]), SizeOf(patData.SurName));
  ...
end;

    The alternative would be to use System.LocaleCharsFromUnicode() instead, eg: 

    
var
  patData : TPatData;
  SurName : String;
begin
  FillChar(@patData, sizeof(patData), 0);
  SurName := ...;
  LocaleCharsFromUnicode(CP_UTF8, 0, PChar(SurName), Length(SurName), @patData.SurName[0], sizeof(patData.SurName), nil, nil);
  ...
end;

     

    Thank you very much much for the great insight!

×