Yaron

Using Delphi 10.3.3 I built a simple web server using MARS Curiosity (https://en.delphipraxis.net/forum/34-mars-curiosity-rest-library/) and FireBird DB v3.0.4. However, sometimes a simple refresh in the web browser (Ctrl+R) will trigger multiple different errors: Project GameServicesServerApplication.exe raised exception class EIdSocketError with message 'Socket Error # 10053 Software caused connection abort. Project GameServicesServerApplication.exe raised exception class $C0000005 with message 'access violation at 0x00730647: read of address 0x00000000'. Project GameServicesServerApplication.exe raised exception class EDatabaseError with message 'Field 'CONSUMERNAME' not found'. Project GameServicesServerApplication.exe raised exception class EArgumentOutOfRangeException with message 'Argument out of range'. Holding Ctrl+R in the web browser easily reproduces the issue within a few seconds (the more DB calls I preform in the MARS web server function, the easier it is to trigger the crash). The fault is not in MARS, I tested by disabling all DB calls and verifying the code doesn't crash (I even threw a Sleep(1000) in there to see if it may be related to the duration it takes the function to process). Am I doing something wrong? I use a connection pool to support multiple threads connecting at once, here is how I connect to the DB: dbParams := TStringList.Create; try dbParams.Add('Server=localhost'); dbParams.Add('Database=c:\DB\Database.FDB'); dbParams.Add('User_Name=SYSDBA'); dbParams.Add('Password=SomePassword'); dbParams.Add('CharacterSet=UTF8'); dbParams.Add('Pooled=True'); FDManager.AddConnectionDef(dbPoolName,'FB',dbParams); Try FDManager.Open; Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(debugFileSystem,'Exception connecting to DB : '+E.Message){$ENDIF}; End; finally dbParams.Free; end; Here's the MARS server definition: Type [Path('test')] TGalleryServicesResource = class protected public [GET , Path('/gallery'), Produces(TMediaType.TEXT_HTML)] function Gallery_From_External_Source([QueryParam] code : String) : String; end; Here is the function MARS triggers: function TGalleryServicesResource.Gallery_From_External_Source(code : String) : String; begin If GetGalleryCodeDetails(sDebugFile,code,nGalleryCode) = True then Begin End; end; And here is the database access: function GetGalleryCodeDetails(sDebugFile,sGalleryCode : String; var nGalleryCode : TGalleryCodeRecord) : Boolean; var dbQuery : TFDQuery; begin Result := False; dbQuery := TFDQuery.Create(nil); Try dbQuery.ConnectionName := dbPoolName; dbQuery.SQL.Text := 'SELECT * FROM GALLERY_CODES WHERE CODE_UID=:gallerycodeuid;'; Try dbQuery.Prepare; dbQuery.ParamByName('gallerycodeuid').AsString := sGalleryCode; dbQuery.Open; If dbQuery.RecordCount > 0 then Begin Result := True; With nGalleryCode do Begin gcUserName := dbQuery.FieldByName('CONSUMERNAME').AsString; gcUserEMail := dbQuery.FieldByName('CONSUMEREMAIL').AsString; gcCodeUID := dbQuery.FieldByName('CODE_UID').AsString; gcImageURL := dbQuery.FieldByName('IMAGEURL').AsString; gcCreateTimeStamp := dbQuery.FieldByName('CREATE_TIMESTAMP').AsFloat; gcOpenTimeStamp := dbQuery.FieldByName('OPEN_TIMESTAMP').AsFloat; gcUseTimeStamp := dbQuery.FieldByName('USE_TIMESTAMP').AsFloat; gcStatus := dbQuery.FieldByName('STATUS').AsInteger; End; End; Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'Exception : '+E.Message){$ENDIF}; End; Finally dbQuery.Free; End; end;

If anyone else is wondering how it's done, you need to add "[Context] marsRequest: TWebRequest;" and then the client's IP is available in "marsRequest.RemoteIP". [Path('projectX')] TProjectXServicesResource = class protected [Context] marsRequest: TWebRequest; public [GET, Produces(TMediaType.TEXT_HTML)] function xHome([QueryParam] lang : String) : String; [POST, Consumes(TMediaType.MULTIPART_FORM_DATA), Produces(TMediaType.TEXT_HTML)] function xAction([FormParams] AParams: TArray<TFormParam>): String; end; function TProjectXServicesResource.xHome([QueryParam] lang : String) : String; begin ShowMessage(marsRequest.RemoteIP); end; function TProjectXServicesResource.xAction([FormParams] AParams: TArray<TFormParam>) : String; begin ShowMessage(marsRequest.RemoteIP); end;

@David Heffernan It's not that the code freezes forever, it stalls. Here are real-world examples, 1. A 3rd party product called ODB Studios has a bug, it does not set a timestamp for it's audio stream, so when playing a video, the entire stream must be decoded to reach the seek point (the position where I take the thumbnail). This causes the (directshow) seek command to stall for upto 60 seconds. 2. A user starts to download a video file and then pauses/stops the download (so the file is no longer locked, I postpone thumbnail generation for locked files). The video's frame-index block did not download yet, so the (directshow) file parser then scans the entire media file to reconstruct the frame-index which can easily stall for 30-60 seconds on multi-GB files. There are other cases where this issue can happen, there is no way to avoid it since I can't know in advance every case for every audio/video format that may pop-up.

Looks like this issue is somewhat complex, I've actually read this article: https://www.the-art-of-web.com/sql/counting-article-views/ In which they suggest creating a separate table just for view counting, with each row representing a view (sql INSERT) and every so often running aggregation code that converts the rows into actual pageview numbers to insert into the original table and erase the counted rows. Using this method means no deadlocks are possible when counting things, might even mean less DB overhead (they write that UPDATE is slower than INSERT, but it's not something I confirmed).

I'm a complete novice working with databases and like dany, I want to know how as well. I actually tried reading the documentation (https://firebirdsql.org/refdocs/langrefupd25-nextvaluefor.html) but it's pretty alien me at this point.

The view number is read earlier in the code, I actually replaced it with a more reliable SQL command to increase the value instead of setting it: procedure ClubHouseDB_IncCampaignViewCount(sDebugFile,sGalleryUID : String); var dbConn : TFDConnection; dbQuery : TFDQuery; deadLocked : Boolean; begin deadLocked := False; dbConn := TFDConnection.Create(nil); dbConn.ConnectionDefName := dbPoolName; dbQuery := TFDQuery.Create(nil); dbQuery.Connection := dbConn; Try dbQuery.SQL.Text := 'UPDATE GALLERY_TABLE SET VIEW_COUNT=VIEW_COUNT+1 WHERE GALLERY_UID=:galleryuid;'; Try dbQuery.Prepare; dbQuery.ParamByName('galleryuid').AsString := sGalleryUID; dbQuery.ExecSQL; Except on E : Exception do If Pos('deadlock',Lowercase(E.Message)) > 0 then deadLocked := True; End; Finally dbQuery.Free; dbConn.Free; End; If deadLocked = True then ClubHouseDB_IncCampaignViewCount(sDebugFile,sGalleryUID); end; As you can see from the updated code, I'm catching the deadlock by looking at the returned exception message, I'm not sure it's the best approach, but I haven't been able to find another way to identify the deadlock. I don't have a problem increase the deadlock timeout, but I couldn't find anything on how to do that, I'm not even sure if it's part of the Firebird SQL statement or if I need to specify it through the Firedac components.

I'm new to databases, is there any Delphi Firedac sample code that shows best practices when handles deadlocks? My deadlock is a simple integer page-view counter, here's the actual code: procedure SetCampaignViewCount(sDebugFile,sGalleryUID : String; nCount : Integer); var dbConn : TFDConnection; dbQuery : TFDQuery; begin dbConn := TFDConnection.Create(nil); dbConn.ConnectionDefName := dbPoolName; dbQuery := TFDQuery.Create(nil); dbQuery.Connection := dbConn; Try dbQuery.SQL.Text := 'UPDATE GALLERY_TABLE SET VIEW_COUNT=:viewcount WHERE GALLERY_UID=:galleryuid;'; Try dbQuery.Prepare; dbQuery.ParamByName('galleryuid').AsString := sGalleryUID; dbQuery.ParamByName('viewcount').AsInteger := nCount; dbQuery.ExecSQL; Except on E : Exception do Begin {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'SetCampaignViewCount Exception : '+E.Message){$ENDIF}; End; End; Finally dbQuery.Free; dbConn.Free; End; end;

Thanks, I read the FAQ entry. I'm new to working with databases, what is the best approach to deal with deadlocks? The actual DB call that triggered the deadlock was a view counter, two users viewing the same web page. What is the best approach to handle such deadlocks?

My code now creates and frees both TFDConnection and TFDQuery: procedure DB_SetUserFlags(sDebugFile,sUserUID : String; iFlags : Integer); var dbConn : TFDConnection; dbQuery : TFDQuery; begin dbConn := TFDConnection.Create(nil); dbConn.ConnectionDefName := dbPoolName; dbQuery := TFDQuery.Create(nil); dbQuery.Connection := dbConn; Try dbQuery.SQL.Text := 'UPDATE USERS SET USER_FLAGS=:userflags WHERE USER_UID=:useruid;'; Try dbQuery.Prepare; dbQuery.ParamByName('userflags').AsInteger := iFlags; dbQuery.ParamByName('useruid').AsString := sUserUID; dbQuery.ExecSQL; Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry(sDebugFile,'Exception : '+E.Message){$ENDIF}; End; Finally dbQuery.Free; dbConn.Free; End; end;

After rewriting every DB entry point using "TFDConnections", I managed to resolve most of the errors, now I'm only receiving these 2: Project GameServicesServerApplication.exe raised exception class EIdSocketError with message 'Socket Error # 10053 Software caused connection abort. Project GameServicesServerApplication.exe raised exception class EIBNativeException with message '[FireDAC][Phys][FB]deadlock I'm assuming the first error is due to constant reloads in browser (it only happens in a browser, doesn't happen for me when load-testing using threads). However, the "deadlock" exception is an issue, any ideas why it's triggered and how to avoid it?

You may be right, I was basing my code on a comment by Jacek here It's quite possible he was wrong, initial testing looks to confirm this.

I even created a simple load-testing app (https://github.com/bLightZP/Web-Server-Load-Tester) that opens the URL in multiple threads simultaneously (triggers the crashes instantly). With 10 thread it crashes instantly if there's DB access, if I disable the DB access code, even 1000 threads work just fine. And even with just 2 threads it can occasionally crash.

The painful thing about this experimentation is that it wipes my editor color selection each time I try something. I simply can't understand some Embarcadero's UI choices 😞

I'm aware that with recent versions, the theming has been rewritten so I'm wondering if there's any tool/editor supporting the latest Delphi theming features that would let me specify my own RGB values for each UI element. I would like to create an alternative dark theme that's more utilitarian. While I prefer a dark theme for eye strain, I find the current dark theme too flat with UI elements that I expect to be on a different plane (color) becoming merged visually (e.g. Scrollbars widgets). I'm aware there are a few other dark'ish themes included, but they too suffer from their own issues so I would rather create (and share of course) my own color-theme.

Sadly using the value from the "Name" field didn't work. Something else is wrong, I can no longer see any of the custom pre-installed themes from the menu.

I was able to modify the style to me desire, called it "mytheme.vsf", When trying to apply it in the registry by setting "Theme" to "Custom" and "VCLStyle" to "mytheme.vsf" (after placing the file in "c:\Program Files (x86)\Embarcadero\Studio\20.0\Redist\styles\vcl\"). However, Delphi doesn't seem to be using the style, what's more, after restoring the registry setting, if i look under the themes pop-up-menu (clicking the little window-moon icon), the "custom" entry is grayed out.

Now that 64bit is just around the corner, I recently found out that I'm lacking the 64bit hardware to test it (even on hardware with 64bit processor, it was running 32bit android). What's more frustrating is that it's hard to even know in advance if a device supports 64bit. Sure, it may have a 64bit processor, but is that processor running a 64bit version of Android? I couldn't find out for a few models I looked into. I also tried the Nox emulator, but sadly, I can't get any Delphi APK to run on it (when emulating Android 7, the most recent version supported by Nox), the app just closes unexpectedly right after the splash screen on an empty app (new blank project -> compile). Any cheap & recommended device for testing 64bit? Any other 64bit android emulation framework that works reliably?

I'm not sure it's compatible with recent changes in v10.3.2 and 10.3.3.

What do I use to extract the resources and what do I use to them rebuilt the file and make it accessible in the IDE? Remember, I only need to change a few RGB values, not really anything too complex. I guess if I can't share my work, I'll just document which values I changed.

It seems the problem is specific accept=".jpg, .jpeg" or even accept=".jpg" using accept="image/jpeg" works just fine.

I figured out why it's crashing in my code and not in the simple sample page, If the HTML form file input has an "accept" field (e.g. "<input type="file" accept=".jpg, .jpeg" name="UploadImage" required>"), the App will crash. Since I may not have control over the form's design, I am still looking for a solution around this issue. You can easily test this with the code I linked in the previous post by replacing: WebBrowser.Navigate('https://ps.uci.edu/~franklin/doc/file_upload.html'); With: WebBrowser.LoadFromStrings( '<HTML><HEAD><TITLE>Test</TITLE></HEAD><BODY>'+ '<form method="post" enctype="multipart/form-data" action="https://bla.com">'+ '<input type="file" accept=".jpg, .jpeg" name="UploadImage" required>'+ '<input type="submit" value="Save">'+ '</form>'+ '</BODY></HTML>','');

I also tried recreating the FManager component after each page load like this: procedure TMainForm.WebBrowserDidFinishLoad(ASender: TObject); begin FManager := nil; {$IFDEF TRACEDEBUG}AddDebugEntry('Create FWebManager (before)');{$ENDIF} Try FManager := TWebChromeClientManager.Create(WebBrowser); Except on E: Exception do begin {$IFDEF TRACEDEBUG}AddDebugEntry('Error creating FWebManager : '+E.Message);{$ENDIF} end; End; {$IFDEF TRACEDEBUG}AddDebugEntry('Create FWebManager (after)');{$ENDIF} end; But it didn't help, still crashes instantly after pressing the button.

Sadly, this issue is not resolved. Like you wrote, it only works if the TWebBrowser component is visible and I verified that it works using the very simple upload form you linked to originally. However, when using it in a more complicated form, after browsing through several pages within the WebView to reach the form, clicking the button just terminates the App instantly (no UI error message). Looking at the logcat, here's the output: 11-28 12:44:35.138 3138 3240 I InputDispatcher: Delivering touch to (3807): action: 0x0, toolType: 1 11-28 12:44:35.138 3807 3807 D ViewRootImpl: ViewPostImeInputStage processPointer 0 11-28 12:44:35.208 3138 3241 D InputReader: Input event(9): value=0 when=3853093882434000 11-28 12:44:35.208 3138 3241 D InputReader: Input event(9): value=0 when=3853093882434000 11-28 12:44:35.208 3138 3241 I InputReader: Touch event's action is 0x1 (deviceType=0) [pCnt=1, s=] when=3853093882434000 11-28 12:44:35.208 3138 3240 I InputDispatcher: Delivering touch to (3807): action: 0x1, toolType: 1 11-28 12:44:35.208 3807 3807 D ViewRootImpl: ViewPostImeInputStage processPointer 1 11-28 12:44:35.238 3807 3807 D Instrumentation: checkStartActivityResult() : Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg } 11-28 12:44:35.238 3807 3807 D Instrumentation: checkStartActivityResult() : intent is instance of [Intent]. 11-28 12:44:35.238 3138 4840 D ApplicationPolicy: isIntentDisabled start :Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg } 11-28 12:44:35.238 3138 4840 D ApplicationPolicy: isIntentDisabled return :false 11-28 12:44:35.238 3807 3807 W System.err: android.content.ActivityNotFoundException: No Activity found to handle Intent { act=android.intent.action.GET_CONTENT cat=[android.intent.category.OPENABLE] typ=.jpg } 11-28 12:44:35.238 3807 3807 W System.err: at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:1878) 11-28 12:44:35.238 3807 3807 W System.err: at android.app.Instrumentation.execStartActivity(Instrumentation.java:1545) 11-28 12:44:35.238 3807 3807 W System.err: at android.app.Activity.startActivityForResult(Activity.java:4283) 11-28 12:44:35.238 3807 3807 W System.err: at android.app.Activity.startActivityForResult(Activity.java:4230) 11-28 12:44:35.238 3807 3807 W System.err: at com.embarcadero.rtl.ProxyInterface.dispatchToNative(Native Method) 11-28 12:44:35.238 3807 3807 W System.err: at com.embarcadero.rtl.ProxyInterface.invoke(ProxyInterface.java:21) 11-28 12:44:35.238 3807 3807 W System.err: at java.lang.reflect.Proxy.invoke(Proxy.java:393) 11-28 12:44:35.238 3807 3807 W System.err: at $Proxy12.onFileChooserIntent(Unknown Source) 11-28 12:44:35.238 3807 3807 W System.err: at com.delphiworlds.kastri.DWWebChromeClient.onShowFileChooser(DWWebChromeClient.java:29) 11-28 12:44:35.238 3807 3807 W System.err: at N6.a(PG:145) 11-28 12:44:35.238 3807 3807 W System.err: at xp.runFileChooser(PG:2) 11-28 12:44:35.238 3807 3807 W System.err: at android.os.MessageQueue.nativePollOnce(Native Method) 11-28 12:44:35.238 3807 3807 W System.err: at android.os.MessageQueue.next(MessageQueue.java:323) 11-28 12:44:35.238 3807 3807 W System.err: at android.os.Looper.loop(Looper.java:143) 11-28 12:44:35.238 3807 3807 W System.err: at android.app.ActivityThread.main(ActivityThread.java:7225) 11-28 12:44:35.238 3807 3807 W System.err: at java.lang.reflect.Method.invoke(Native Method) 11-28 12:44:35.238 3807 3807 W System.err: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230) 11-28 12:44:35.238 3807 3807 W System.err: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120) 11-28 12:44:35.248 1343 1516 D libEGL : eglTerminate EGLDisplay = 0xb690164c 11-28 12:44:35.258 1343 4318 I SurfaceFlinger: id=17304 Removed TurfaceView (4/10) 11-28 12:44:35.258 1343 1522 I SurfaceFlinger: id=17304 Removed TurfaceView (-2/10) 11-28 12:44:35.268 1343 1343 D libEGL : eglTerminate EGLDisplay = 0xbe93d3ac 11-28 12:44:35.888 3138 3240 W InputDispatcher: channel ~ Consumer closed input channel or an error occurred. events=0x9 11-28 12:44:35.888 3138 3240 E InputDispatcher: channel ~ Channel is unrecoverably broken and will be disposed! 11-28 12:44:35.888 3138 4840 I WindowState: WIN DEATH: Window{891996c u0 d0 PopupWindow:8f2b053} 11-28 12:44:35.898 3138 3914 D GraphicsStats: Buffer count: 11 11-28 12:44:35.898 1343 1522 I SurfaceFlinger: id=17305 Removed QopupWindow (5/9) 11-28 12:44:35.898 1343 19954 D libEGL : eglTerminate EGLDisplay = 0xb09a86fc 11-28 12:44:35.898 1343 1522 I SurfaceFlinger: id=17305 Removed QopupWindow (-2/9) 11-28 12:44:35.898 1343 19954 D libEGL : eglTerminate EGLDisplay = 0xb09a86fc 11-28 12:44:35.898 3138 3920 I ActivityManager: Process com.inmatrix.Voucherim (pid 3807)(adj 0) has died(274,734) 11-28 12:44:35.898 3138 3920 D ActivityManager: cleanUpApplicationRecord -- 3807

Thanks Eli, Adding the library resolved the exception, however clicking on the choose file button still does nothing, so something else is preventing this from working in my own code and I can't seem to find out what. I posted the App's source here: https://inmatrix.com/temp/TestApp_src.zip It just wraps around a website, simply run it and click 'connect' (no need for name and pass, it just opens a sample file submission page)

For some reason, any data I entered under the Project Options version info's "all configuration" entries is not actually applied to the release/debug sections. And later on, I couldn't upload to the play store, because it was seeing the old version number from the "release" section. Is it just me, or am I misunderstanding what the "all configuration" entries actually do (I was assuming it was applying whatever I entered into both release/debug profiles so I don't have to do everything twice).