Jump to content

Yaron

Members
 View Profile  See their activity

  • Content Count

    286

  • Joined

  • Last visited

  • Days Won

    2

Posts posted by Yaron

  1. Posted · Edited by Yaron

    @David Heffernan It's not that the code freezes forever, it stalls.

     

    Here are real-world examples,

    1. A 3rd party product called ODB Studios has a bug, it does not set a timestamp for it's audio stream, so when playing a video, the entire stream must be decoded to reach the seek point (the position where I take the thumbnail).

    This causes the (directshow) seek command to stall for upto 60 seconds.

     

    2. A user starts to download a video file and then pauses/stops the download (so the file is no longer locked, I postpone thumbnail generation for locked files).

    The video's frame-index block did not download yet, so the (directshow) file parser then scans the entire media file to reconstruct the frame-index which can easily stall for 30-60 seconds on multi-GB files.

     

    There are other cases where this issue can happen, there is no way to avoid it since I can't know in advance every case for every audio/video format that may pop-up.

     

     

  2. Posted · Edited by Yaron

    I have a usage case where my application scrapes a screenshot (thumbnail) from multiple video files in background threads.

    I grab the screenshot using a third party library.

     

    The problem is sometimes, the third party library will stall, usually due to a corrupt/incomplete media file where it tries to analyze the entire file (which can take upto 30 seconds).

    I have no way to notify the third party library to cease operations.

     

    This stalling can prevent my application from closing (It's waiting for threads to finish so the exit is clean) or even make the UI appear semi-broken because no thumbnails show up.

     

    I'd like to hear what you may have done to handle such cases?

  3. Posted

    Hi @Andrea Magni

    To improve security and statistics, I need access to the client's IP address when my function triggers in server.resources:

      

      [Path('projectX')]
  TProjectXServicesResource = class
  protected
  public
    [GET, Produces(TMediaType.TEXT_HTML)]
    function xHome([QueryParam] lang : String) : String;

    [POST, Consumes(TMediaType.MULTIPART_FORM_DATA), Produces(TMediaType.TEXT_HTML)]
    function xAction([FormParams] AParams: TArray<TFormParam>): String;
  end;

function TProjectXServicesResource.xHome([QueryParam] lang : String) : String;
begin
  // Need access to the client's IP here
end;

function TProjectXServicesResource.xAction([FormParams] AParams: TArray<TFormParam>) : String;
begin
  // Need access to the client's IP here
end;

    How can I obtain the client IP?

  4. Posted

    Looks like this issue is somewhat complex, I've actually read this article:
    https://www.the-art-of-web.com/sql/counting-article-views/

     

    In which they suggest creating a separate table just for view counting, with each row representing a view (sql INSERT) and every so often running aggregation code that converts the rows into actual pageview numbers to insert into the original table and erase the counted rows.

    Using this method means no deadlocks are possible when counting things, might even mean less DB overhead (they write that UPDATE is slower than INSERT, but it's not something I confirmed).

     

  5. Posted · Edited by Yaron
    more info

    59 minutes ago, Zoran Bonuš said:

    Exactly, just what I was about to recommend. You create deadlock because the new value of VIEW_COUNT depends on the old value, and due to the nature of multi-generation architecture a simultaneous transactions, there can be different old values, depending on the isolation settings of your transaction.

    Using generator/sequence for view_count circumvalents the problem for this specific update query. However, a proper transaction setting is preferred anyway. There is no magic solve-it-all setting, I recommend studing the docs too .. will most likely came in handy later anyway.

    I'm a complete novice working with databases and like dany, I want to know how as well.

    I actually tried reading the documentation (https://firebirdsql.org/refdocs/langrefupd25-nextvaluefor.html) but it's pretty alien me at this point.

  6. Posted · Edited by Yaron

    22 minutes ago, Dany Marmur said:

    The case above is not quite clear to me. Where do you get the actual value for count?

    IMHO - reading up on FB transaction handling will get you a long way. I think FireDAC has components for that, not sure though.

     

    If you would want to increase the count inside you SQL then deadlocks could definitely happen.

    Two ways to manage that; either handle the case in your Exception block, posting something that will retry the operation. That could prove messy depending on the rest of the application/service architecture.

    The second way is to tell FB that this trasaction should wait for a lock to be released. Now, the message indicates that the transaction is NOT waiting. You set a value of seconds for transaction wait.

     

    I'm not the best to help here since i do not use FireDAC and i have no idea of how to handle transaction parameter, autocommitts and such stuff using FireDAC:

     

    HTH,

     

    /Dany

     

    The view number is read earlier in the code, I actually replaced it with a more reliable SQL command to increase the value instead of setting it: 

    procedure ClubHouseDB_IncCampaignViewCount(sDebugFile,sGalleryUID : String);
var
  dbConn     : TFDConnection;
  dbQuery    : TFDQuery;
  deadLocked : Boolean;
begin
  deadLocked := False;
  dbConn  := TFDConnection.Create(nil);  dbConn.ConnectionDefName := dbPoolName;
  dbQuery := TFDQuery.Create(nil);       dbQuery.Connection       := dbConn;
  Try
    dbQuery.SQL.Text   := 'UPDATE GALLERY_TABLE SET VIEW_COUNT=VIEW_COUNT+1 WHERE GALLERY_UID=:galleryuid;';
    Try
      dbQuery.Prepare;
      dbQuery.ParamByName('galleryuid').AsString := sGalleryUID;
      dbQuery.ExecSQL;
    Except
      on E : Exception do If Pos('deadlock',Lowercase(E.Message)) > 0 then deadLocked := True;
    End;
  Finally
    dbQuery.Free;
    dbConn.Free;
  End;
  If deadLocked = True then ClubHouseDB_IncCampaignViewCount(sDebugFile,sGalleryUID);
end;

    As you can see from the updated code, I'm catching the deadlock by looking at the returned exception message, I'm not sure it's the best approach, but I haven't been able to find another way to identify the deadlock.

     

    I don't have a problem increase the deadlock timeout, but I couldn't find anything on how to do that, I'm not even sure if it's part of the Firebird SQL statement or if I need to specify it through the Firedac components.

     

  7. Posted · Edited by Yaron

    I'm new to databases, is there any Delphi Firedac sample code that shows best practices when handles deadlocks?

     

    My deadlock is a simple integer page-view counter, here's the actual code: 

    procedure SetCampaignViewCount(sDebugFile,sGalleryUID : String; nCount : Integer);
var
  dbConn  : TFDConnection;
  dbQuery : TFDQuery;
begin
  dbConn  := TFDConnection.Create(nil);  dbConn.ConnectionDefName := dbPoolName;
  dbQuery := TFDQuery.Create(nil);       dbQuery.Connection       := dbConn;
  Try
    dbQuery.SQL.Text   := 'UPDATE GALLERY_TABLE SET VIEW_COUNT=:viewcount WHERE GALLERY_UID=:galleryuid;';
    Try
      dbQuery.Prepare;
      dbQuery.ParamByName('galleryuid').AsString := sGalleryUID;
      dbQuery.ParamByName('viewcount').AsInteger := nCount;
      dbQuery.ExecSQL;
    Except
      on E : Exception do
      Begin
        {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'SetCampaignViewCount Exception : '+E.Message){$ENDIF};
      End;
    End;
  Finally
    dbQuery.Free; dbConn.Free;
  End;
end;

     

  8. Posted

    This happens on a particular project,

     

    Simply opening the project and clicking on "refactor" from the top menu causes the entire IDE to crash with "Embarcadero RAD Studio for Windows has stopped working, A problem caused the program to stop working correctly. Please close the program.".

    The same thing happens if I do "Run -> Step Over" from the top menu followed by clicking the "search" top menu entry.

    In both cases the top menu's pop-up doesn't show up before the crash.

     

    Doesn't happen if I open another project, but can be reliably reproduced on the problematic (larger) project every time.

    Anyone encountered anything like this before?  I'm not even sure how to track down this issue.

  9. Posted

    8 minutes ago, rvk said:

    But I see you update the table in the thread. You didn't do that in your original post.

     

    The deadlock can come from using transactions incorrectly (or updating the same record in different transactions/threads),

    http://www.firebirdfaq.org/faq151/

    Thanks, I read the FAQ entry.

    I'm new to working with databases, what is the best approach to deal with deadlocks?

     

    The actual DB call that triggered the deadlock was a view counter, two users viewing the same web page.

    What is the best approach to handle such deadlocks?

     

  10. Posted

    My code now creates and frees both TFDConnection and TFDQuery: 

    procedure DB_SetUserFlags(sDebugFile,sUserUID : String; iFlags : Integer);
var
  dbConn  : TFDConnection;
  dbQuery : TFDQuery;
begin
  dbConn := TFDConnection.Create(nil);
  dbConn.ConnectionDefName := dbPoolName;
  dbQuery := TFDQuery.Create(nil);
  dbQuery.Connection := dbConn;
  Try
    dbQuery.SQL.Text   := 'UPDATE USERS SET USER_FLAGS=:userflags WHERE USER_UID=:useruid;';

    Try
      dbQuery.Prepare;
      dbQuery.ParamByName('userflags').AsInteger := iFlags;
      dbQuery.ParamByName('useruid').AsString    := sUserUID;
      dbQuery.ExecSQL;
    Except
      on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'Exception : '+E.Message){$ENDIF};
    End;
  Finally
    dbQuery.Free;
    dbConn.Free;
  End;
end;

     

  11. Posted

    After rewriting every DB entry point using "TFDConnections", I managed to resolve most of the errors, now I'm only receiving these 2:

    Project GameServicesServerApplication.exe raised exception class EIdSocketError with message 'Socket Error # 10053 Software caused connection abort.

    Project GameServicesServerApplication.exe raised exception class EIBNativeException with message '[FireDAC][Phys][FB]deadlock

     

    I'm assuming the first error is due to constant reloads in browser (it only happens in a browser, doesn't happen for me when load-testing using threads).

    However, the "deadlock" exception is an issue, any ideas why it's triggered and how to avoid it?

  13. Posted · Edited by Yaron

    I even created a simple load-testing app (https://github.com/bLightZP/Web-Server-Load-Tester) that opens the URL in multiple threads simultaneously (triggers the crashes instantly).

     

    With 10 thread it crashes instantly if there's DB access, if I disable the DB access code, even 1000 threads work just fine.

    And even with just 2 threads it can occasionally crash.

  14. Posted · Edited by Yaron

    Using Delphi 10.3.3 I built a simple web server using MARS Curiosity (https://en.delphipraxis.net/forum/34-mars-curiosity-rest-library/) and FireBird DB v3.0.4.

     

    However, sometimes a simple refresh in the web browser (Ctrl+R) will trigger multiple different errors:

    Project GameServicesServerApplication.exe raised exception class EIdSocketError with message 'Socket Error # 10053 Software caused connection abort.

    Project GameServicesServerApplication.exe raised exception class $C0000005 with message 'access violation at 0x00730647: read of address 0x00000000'.
    Project GameServicesServerApplication.exe raised exception class EDatabaseError with message 'Field 'CONSUMERNAME' not found'.

    Project GameServicesServerApplication.exe raised exception class EArgumentOutOfRangeException with message 'Argument out of range'.
     

    Holding Ctrl+R in the web browser easily reproduces the issue within a few seconds (the more DB calls I preform in the MARS web server function, the easier it is to trigger the crash).

    The fault is not in MARS, I tested by disabling all DB calls and verifying the code doesn't crash (I even threw a Sleep(1000) in there to see if it may be related to the duration it takes the function to process).

     

    Am I doing something wrong?

     

    I use a connection pool to support multiple threads connecting at once, here is how I connect to the DB: 

      dbParams := TStringList.Create;
  try
    dbParams.Add('Server=localhost');
    dbParams.Add('Database=c:\DB\Database.FDB');
    dbParams.Add('User_Name=SYSDBA');
    dbParams.Add('Password=SomePassword');
    dbParams.Add('CharacterSet=UTF8');
    dbParams.Add('Pooled=True');
    FDManager.AddConnectionDef(dbPoolName,'FB',dbParams);
    Try
      FDManager.Open;
    Except
      on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(debugFileSystem,'Exception connecting to DB : '+E.Message){$ENDIF};
    End;
  finally
    dbParams.Free;
  end;

     

    Here's the MARS server definition: 

    Type
  [Path('test')]
  TGalleryServicesResource = class
  protected
  public
    [GET , Path('/gallery'), Produces(TMediaType.TEXT_HTML)]
    function Gallery_From_External_Source([QueryParam] code : String) : String;
  end;

     

    Here is the function MARS triggers: 

    function TGalleryServicesResource.Gallery_From_External_Source(code : String) : String;
begin
  If GetGalleryCodeDetails(sDebugFile,code,nGalleryCode) = True then
  Begin
  End;
end;

     

    And here is the database access: 

    function  GetGalleryCodeDetails(sDebugFile,sGalleryCode : String; var nGalleryCode : TGalleryCodeRecord) : Boolean;
var
  dbQuery : TFDQuery;
begin
  Result := False;
  dbQuery := TFDQuery.Create(nil);
  Try
    dbQuery.ConnectionName := dbPoolName;
    dbQuery.SQL.Text := 'SELECT * FROM GALLERY_CODES WHERE CODE_UID=:gallerycodeuid;';

    Try
      dbQuery.Prepare;
      dbQuery.ParamByName('gallerycodeuid').AsString := sGalleryCode;
      dbQuery.Open;
      If dbQuery.RecordCount > 0 then
      Begin
        Result := True;
        With nGalleryCode do
        Begin
          gcUserName              := dbQuery.FieldByName('CONSUMERNAME').AsString;
          gcUserEMail             := dbQuery.FieldByName('CONSUMEREMAIL').AsString;
          gcCodeUID               := dbQuery.FieldByName('CODE_UID').AsString;
          gcImageURL              := dbQuery.FieldByName('IMAGEURL').AsString;
          gcCreateTimeStamp       := dbQuery.FieldByName('CREATE_TIMESTAMP').AsFloat;
          gcOpenTimeStamp         := dbQuery.FieldByName('OPEN_TIMESTAMP').AsFloat;
          gcUseTimeStamp          := dbQuery.FieldByName('USE_TIMESTAMP').AsFloat;
          gcStatus                := dbQuery.FieldByName('STATUS').AsInteger;
        End;
      End;
    Except
      on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'Exception : '+E.Message){$ENDIF};
    End;
  Finally
    dbQuery.Free;
  End;
end;

     

  17. Posted

    I was able to modify the style to me desire, called it "mytheme.vsf",

     

    When trying to apply it in the registry by setting "Theme" to "Custom" and "VCLStyle" to "mytheme.vsf" (after placing the file in "c:\Program Files (x86)\Embarcadero\Studio\20.0\Redist\styles\vcl\").

     

    However, Delphi doesn't seem to be using the style, what's more, after restoring the registry setting, if i look under the themes pop-up-menu (clicking the little window-moon icon), the "custom" entry is grayed out.

  19. Posted · Edited by Yaron

    14 minutes ago, Dalija Prasnikar said:

    There are few problems here.

     

    First, IDE themes are slightly different from general VCL Styles so you cannot use them because some parts of IDE will not be painted properly. Next VCL Style editor is rather hard to use for creating any theme from scratch. 

     

    Your best option would be to extract theme resource from darktheme260.bpl (haven't checked if this is the right bpl, as I have customized light theme), and then use that resource as starting point for your customization. 

     

    IDE themes are copyrighted, so you should not distribute such hacked theme.

    What do I use to extract the resources and what do I use to them rebuilt the file and make it accessible in the IDE?

    Remember, I only need to change a few RGB values, not really anything too complex.

     

    I guess if I can't share my work, I'll just document which values I changed.

  20. Posted

    I'm aware that with recent versions, the theming has been rewritten so I'm wondering if there's any tool/editor supporting the latest Delphi theming features that would let me specify my own RGB values for each UI element.

     

    I would like to create an alternative dark theme that's more utilitarian.

    While I prefer a dark theme for eye strain, I find the current dark theme too flat with UI elements that I expect to be on a different plane (color) becoming merged visually (e.g. Scrollbars widgets).

    I'm aware there are a few other dark'ish themes included, but they too suffer from their own issues so I would rather create (and share of course) my own color-theme.

  22. Posted

    I figured out why it's crashing in my code and not in the simple sample page,

    If the HTML form file input has an "accept" field (e.g. "<input type="file" accept=".jpg, .jpeg" name="UploadImage" required>"), the App will crash.

     

    Since I may not have control over the form's design, I am still looking for a solution around this issue.

     

    You can easily test this with the code I linked in the previous post by replacing: 

      WebBrowser.Navigate('https://ps.uci.edu/~franklin/doc/file_upload.html');

    With: 

      WebBrowser.LoadFromStrings(
  '<HTML><HEAD><TITLE>Test</TITLE></HEAD><BODY>'+
  '<form method="post" enctype="multipart/form-data" action="https://bla.com">'+
  '<input type="file" accept=".jpg, .jpeg" name="UploadImage" required>'+
  '<input type="submit" value="Save">'+
  '</form>'+
  '</BODY></HTML>','');

     

    • Like 1

  23. Posted

    I also tried recreating the FManager component after each page load like this:
      

    procedure TMainForm.WebBrowserDidFinishLoad(ASender: TObject);
begin
  FManager := nil;
    {$IFDEF TRACEDEBUG}AddDebugEntry('Create FWebManager (before)');{$ENDIF}
    Try
      FManager := TWebChromeClientManager.Create(WebBrowser);
    Except
      on E: Exception do
      begin
        {$IFDEF TRACEDEBUG}AddDebugEntry('Error creating FWebManager : '+E.Message);{$ENDIF}
      end;
    End;
    {$IFDEF TRACEDEBUG}AddDebugEntry('Create FWebManager (after)');{$ENDIF}
end;

    But it didn't help, still crashes instantly after pressing the button.

  24. Posted · Edited by Yaron

    14 hours ago, Dave Nottage said:

    It seems that setting the WebChromeClient works only when the TWebBrowser is visible. Move your code that creates FWebManager to just after you set VoucherimWebBrowser.Visible to True.

    Sadly, this issue is not resolved.

    Like you wrote, it only works if the TWebBrowser component is visible and I verified that it works using the very simple upload form you linked to originally.

    However, when using it in a more complicated form, after browsing through several pages within the WebView to reach the form, clicking the button just terminates the App instantly (no UI error message).

    Looking at the logcat, here's the output:

      

    11-28 12:44:35.138  3138  3240 I InputDispatcher: Delivering touch to (3807): action: 0x0, toolType: 1
11-28 12:44:35.138  3807  3807 D ViewRootImpl: ViewPostImeInputStage processPointer 0
11-28 12:44:35.208  3138  3241 D InputReader: Input event(9): value=0 when=3853093882434000
11-28 12:44:35.208  3138  3241 D InputReader: Input event(9): value=0 when=3853093882434000
11-28 12:44:35.208  3138  3241 I InputReader: Touch event's action is 0x1 (deviceType=0) [pCnt=1, s=] when=3853093882434000
11-28 12:44:35.208  3138  3240 I InputDispatcher: Delivering touch to (3807): action: 0x1, toolType: 1
11-28 12:44:35.208  3807  3807 D ViewRootImpl: ViewPostImeInputStage processPointer 1
11-28 12:44:35.238  3807  3807 D Instrumentation: checkStartActivityResult() : Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg }
11-28 12:44:35.238  3807  3807 D Instrumentation: checkStartActivityResult() : intent is instance of [Intent].
11-28 12:44:35.238  3138  4840 D ApplicationPolicy: isIntentDisabled start :Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg }
11-28 12:44:35.238  3138  4840 D ApplicationPolicy: isIntentDisabled return :false
11-28 12:44:35.238  3807  3807 W System.err: android.content.ActivityNotFoundException: No Activity found to handle Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg }
11-28 12:44:35.238  3807  3807 W System.err: 	at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:1878)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.app.Instrumentation.execStartActivity(Instrumentation.java:1545)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.app.Activity.startActivityForResult(Activity.java:4283)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.app.Activity.startActivityForResult(Activity.java:4230)
11-28 12:44:35.238  3807  3807 W System.err: 	at com.embarcadero.rtl.ProxyInterface.dispatchToNative(Native Method)
11-28 12:44:35.238  3807  3807 W System.err: 	at com.embarcadero.rtl.ProxyInterface.invoke(ProxyInterface.java:21)
11-28 12:44:35.238  3807  3807 W System.err: 	at java.lang.reflect.Proxy.invoke(Proxy.java:393)
11-28 12:44:35.238  3807  3807 W System.err: 	at $Proxy12.onFileChooserIntent(Unknown Source)
11-28 12:44:35.238  3807  3807 W System.err: 	at com.delphiworlds.kastri.DWWebChromeClient.onShowFileChooser(DWWebChromeClient.java:29)
11-28 12:44:35.238  3807  3807 W System.err: 	at N6.a(PG:145)
11-28 12:44:35.238  3807  3807 W System.err: 	at xp.runFileChooser(PG:2)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.os.MessageQueue.nativePollOnce(Native Method)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.os.MessageQueue.next(MessageQueue.java:323)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.os.Looper.loop(Looper.java:143)
11-28 12:44:35.238  3807  3807 W System.err: 	at android.app.ActivityThread.main(ActivityThread.java:7225)
11-28 12:44:35.238  3807  3807 W System.err: 	at java.lang.reflect.Method.invoke(Native Method)
11-28 12:44:35.238  3807  3807 W System.err: 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
11-28 12:44:35.238  3807  3807 W System.err: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)
11-28 12:44:35.248  1343  1516 D libEGL  : eglTerminate EGLDisplay = 0xb690164c
11-28 12:44:35.258  1343  4318 I SurfaceFlinger: id=17304 Removed TurfaceView (4/10)
11-28 12:44:35.258  1343  1522 I SurfaceFlinger: id=17304 Removed TurfaceView (-2/10)
11-28 12:44:35.268  1343  1343 D libEGL  : eglTerminate EGLDisplay = 0xbe93d3ac
11-28 12:44:35.888  3138  3240 W InputDispatcher: channel ~ Consumer closed input channel or an error occurred.  events=0x9
11-28 12:44:35.888  3138  3240 E InputDispatcher: channel ~ Channel is unrecoverably broken and will be disposed!
11-28 12:44:35.888  3138  4840 I WindowState: WIN DEATH: Window{891996c u0 d0 PopupWindow:8f2b053}
11-28 12:44:35.898  3138  3914 D GraphicsStats: Buffer count: 11
11-28 12:44:35.898  1343  1522 I SurfaceFlinger: id=17305 Removed QopupWindow (5/9)
11-28 12:44:35.898  1343 19954 D libEGL  : eglTerminate EGLDisplay = 0xb09a86fc
11-28 12:44:35.898  1343  1522 I SurfaceFlinger: id=17305 Removed QopupWindow (-2/9)
11-28 12:44:35.898  1343 19954 D libEGL  : eglTerminate EGLDisplay = 0xb09a86fc
11-28 12:44:35.898  3138  3920 I ActivityManager: Process com.inmatrix.Voucherim (pid 3807)(adj 0) has died(274,734)
11-28 12:44:35.898  3138  3920 D ActivityManager: cleanUpApplicationRecord -- 3807

     

  25. Posted

    Thanks Eli, Adding the library resolved the exception, however clicking on the choose file button still does nothing, so something else is preventing this from working in my own code and I can't seem to find out what.

     

    I posted the App's source here:

    https://inmatrix.com/temp/TestApp_src.zip

     

    It just wraps around a website, simply run it and click 'connect' (no need for name and pass, it just opens a sample file submission page)

×