Jump to content

Yaron

Members
  • Content Count

    101
  • Joined

  • Last visited

  • Days Won

    1

Everything posted by Yaron

  1. I am trying to build a system that will handle concurrent DB requests efficiently, there's not a lot of load, but it should be able to handle more than 1 query concurrently. I am using FireBird v3.0.4 and Delphi 10.3.2 with the TFDConnection component. I am hosting the code in an ISAPI DLL that runs on IIS 7.5. Right now I'm creating the DB connection using TFDConnection in the DLL's "initialization" section. However, I noticed that even under low-load (2-3 users), I occasionally get an "[FireDAC][Phys][FB]Error reading data from the connection." exception when performing a DB query (not the same DB query, it seems pretty random). I tried to research the error and didn't find anything clear, the best I got was : https://forums.embarcadero.com/thread.jspa?threadID=245750 Which seems to indicate that I should instance a copy of TFDConnection for every query. Before I make significant code changes to test this, I would welcome any tips on the best approach to handling DB concurrency.
  2. Yaron

    FireBird, TFDConnection and concurrency

    You beat me to it, I just found this: https://mathiaspannier.wordpress.com/2016/07/17/how-to-properly-cleanupshutdown-a-delphi-isapi-which-uses-threads/ Which is essentially the same answer, can't close the connection in the finalization section when dealing with ISAPI.
  3. Yaron

    FireBird, TFDConnection and concurrency

    Looks like I'm encountering an IIS issue when using this scheme. In the ISAPI's DLL's finalization code I call "FDManager.Close" and in IIS, when stopping the application pool associated with the DLL it can take ~60 seconds before I can restart the pool, any ideas? This is the error I get: When checking my logs, It doesn't seem to exit cleanly (the debug message that the "DB Closed" does not show up in the log), but there doesn't seem to be an exception, here's the code I use: Try FDManager.Close; Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('Exception disconnecting from DB : '+E.Message){$ENDIF}; End; {$IFDEF TRACEDEBUG}AddDebugEntry('DB Closed');{$ENDIF} When I run the same code locally as an EXE that handles the HTTP requests, there are no issues, any ideas?
  4. Yaron

    FireBird, TFDConnection and concurrency

    Thank you! Took me a bit to understand the mechanics, but with your help I managed to rewrite the code with minimal changes and so far it's working. P.S. "FDManager.Active :=True;" no longer works in Delphi 10.3, it simply doesn't exist anymore, but "FDManager.Open" seems to be the replacement.
  5. Yaron

    FireBird, TFDConnection and concurrency

    Does that mean that for every http request (that requires DB access) I have to create a new instance of TFDConnection and then connect to the DB? This is the first time I'm writing an ISAPI dll with DB access and the threading model is not exactly clear to me. My original (perhaps flawed) assumption was that IIS will load additional DLLs in separate threads to handle concurrent http requests. I would like to avoid the 100ms penalty for connecting to the DB on each request, any tips?
  6. It took me hours of research to find the tools & documentation and haven't found any up-to-date text describing a similar process, so I decided to post my findings here in case someone else has the same requirement. I wrote (in Delphi) a simple windows service that runs the FireSQL "nbackup.exe" command line tool to create a DB backup every [x] hours using this command line: "c:\Program Files\Firebird\Firebird_3_0\nbackup.exe" -U SYSDBA -P [password] -B 0 "c:\path\database.FDB" "[output_file]" The windows service keeps [x] number of backup files (erasing old files) and creates a new backup every [y] minutes. Personally, I used 1 full-db backup per hour, covering an entire week (168 files). I then use the open-source SyncThing P2P file sync tool (also running as a windows service) to securely & automatically sync the DB backups with every PC assigned as off-site backup. 100% automated, secure, zero cost.
  7. Not my quote: I basically used it cause that's the example I was able to find in the documentation for exporting the DB. In theory I could have used the windows task scheduler, but I needed more control and I'm using the service to perform other timed DB actions. And I couldn't just call nbackup directly from the task scheduler as I'm erasing old DB files and numbering new backups based on existing backup file names (a running counter in the file name) and some more logic is required for this.
  8. SyncThing has a Linux/Mac/FreeBSD/etc... client, so in theory the off-site backups can be on pretty much any widely used OS.
  9. To give me the largest work-space, I designed the Delphi IDE UI to look like the attached "delphi_before.jpg". However, under some conditions, windows throws apps a resize event (e.g. when turning on a secondary monitor), and then the Delphi IDE UI automatically changes to the attached "delphi_after.jpg" (the tool-bar drops down a row). Is there any way to lock the UI so my preferred layout remains fixed? P.S. This is on Win7 64bit.
  10. This is a bit old, but I've been blocking the windows screen-saver for media playback in every version of windows since 1999. I'm actually handling several actions, first I capture the WM_POWERBROADCAST message in the form: procedure WMPOWERBROADCAST(var M : TMessage); message WM_POWERBROADCAST; const PBT_APMQUERYSUSPEND = $0000; PBT_APMQUERYSTANDBY = $0001; PBT_APMQUERYSUSPENDFAILED = $0002; PBT_APMQUERYSTANDBYFAILED = $0003; PBT_APMSUSPEND = $0004; PBT_APMSTANDBY = $0005; PBT_APMRESUMECRITICAL = $0006; PBT_APMRESUMESUSPEND = $0007; PBT_APMRESUMESTANDBY = $0008; PBT_APMBATTERYLOW = $0009; PBT_APMPOWERSTATUSCHANGE = $000A; PBT_APMOEMEVENT = $000B; PBT_APMRESUMEAUTOMATIC = $0012; begin Inherited; Case M.WParam of PBT_APMQUERYSUSPEND, PBT_APMSUSPEND : Begin If SystemSuspended = False then Begin SystemSuspended := True; End; End; PBT_APMRESUMEAUTOMATIC : Begin SystemSuspended := False; End; End; M.Result := Integer(True); end; Then on WinMsg (Application.OnMessage := WinMsg;) I do: procedure TMainForm.WinMsg(var Msg: TMsg; var Handled: Boolean); var ModWnd : HWnd; I : Integer; mPos : TPoint; begin If (Msg.Message = SC_SCREENSAVE) or (Msg.Message = SC_MONITORPOWER) then Begin Msg.wParam := 0; Msg.Message := SC_Move; Handled := True; End else If (Msg.Message = WM_SYSCOMMAND) then Begin Case Msg.WPARAM of SC_MONITORPOWER, SC_SCREENSAVE : Begin Msg.wParam := 0; Msg.Message := SC_Move; Handled := True; End; End; End; End; Then on WndProc ( procedure WndProc(var Msg : TMessage); override;) : Hope this helps.
  11. Took your advice and created a QC report: https://quality.embarcadero.com/browse/RSP-26253
  12. It's not an issue restoring the IDE, just a little drag on that line, but it happens about once or twice every day which is a bit annoying.
  13. Using Delphi 10.3.2: I use the function below to send HTML eMails through Amazon SES using Indy. However, no matter what I tried, non-latin characters appear as "�" (for example, instead of "Oriënt Express", gMail displays "Ori�nt Express"). I need to be able to include non-latin characters in both the subject and the eMail's HTML body, what am I doing wrong? P.S. Not sure if it matters, but the HTML does contain "<meta charset="UTF-8">" in the "<HEAD>" block and I verified the source was properly encoded by saving it to a local utf8 html file which displays the encoded characters correctly. procedure SendEmailIndy( const SMTPServer: string; const SMTPPort: integer; const SMTPUserName : string; const SMTPPassword : string; const FromName, FromAddress: string; const ToAddresses: string; //comma "," separated list of e-mail addresses const CCAddresses: string; //comma "," separated list of e-mail addresses const BCCAddresses: string; //comma "," separated list of e-mail addresses const Subject: string; const EmailBody: string; const IsBodyHtml: Boolean; //verses Plain Text const Attachments: TStrings; UseTLS : Boolean); var smtp: TIdSMTP; // IdSmtp.pas TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support msg: TidMessage; // IdMessage.pas builder: TIdCustomMessageBuilder; //IdMessageBuilder.pas s: string; emailAddress: string; begin TLSHandler := nil; msg := TidMessage.Create(nil); msg.ContentType := 'text/html'; msg.CharSet := 'UTF-8'; msg.ContentTransferEncoding := '8bit'; try if IsBodyHtml then begin builder := TIdMessageBuilderHtml.Create; TIdMessageBuilderHtml(builder).Html.Text := EmailBody; end else begin builder := TIdMessageBuilderPlain.Create; end; try Try if Attachments <> nil then for s in Attachments do builder.Attachments.Add(s); builder.FillMessage(msg); Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('Exception : '+E.Message){$ENDIF}; End; finally builder.Free; end; msg.From.Name := FromName; msg.From.Address := FromAddress; msg.Subject := Subject; //If the message is plaintext then we must fill the body outside of the PlainText email builder. //(the PlainTextBuilder is unable to build plaintext e-mail) if not IsBodyHtml then msg.Body.Text := EmailBody; for s in ToAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then begin with msg.recipients.Add do begin //Name := '<Name of recipient>'; Address := emailAddress; end; end; end; for s in CCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.CCList.Add.Address := emailAddress; end; for s in BCCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.BccList.Add.Address := emailAddress; end; smtp := TIdSMTP.Create(nil); try smtp.Host := SMTPServer; smtp.Port := SMTPPort; smtp.Username := SMTPUserName; smtp.Password := SMTPPassword; If UseTLS = True then Begin TLSHandler := TIdSSLIOHandlerSocketOpenSSL.Create; smtp.IOHandler := TLSHandler; smtp.UseTLS := TIdUseTLS.utUseRequireTLS; End; Try smtp.Connect; try try smtp.Send(msg) Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('SMTP Send Exception : '+E.Message){$ENDIF}; End; finally smtp.Disconnect; end; Except on E : Exception do {$IFDEF TRACEDEBUG}AddDebugEntry('SMTP Connect Exception : '+E.Message){$ENDIF}; End; finally smtp.Free; If TLSHandler <> nil then TLSHandler.Free; end; finally msg.Free; end; end;
  14. Nevermind, I figured it out, I had to move the charset code after setting the email body.
  15. Another bit of information, when looking at the headers of the delivered email, there's this: Content-Type: text/html; charset=us-ascii Even though CharSet is set to UTF8. According at this post https://stackoverflow.com/questions/9844250/not-able-to-send-utf-8-email-using-delphi-indy , I'm setting the parameters in the right order.
  16. If I try to use UTF8Encode in Delphi 10.3 I get a warning "[dcc32 Warning] xxx.pas(683): W1057 Implicit string cast from 'RawByteString' to 'string'".
  17. I am having a really weird issue with Firebird SQL 3.0.4, I have two servers, one local which I use for development and one production, both are running the exact same version of Firebird, using the same "firebird.conf" configuration file. On my production server, if I enter Hebrew text (e.g. "בדיקה"), I get "?????" in the DB. However, it works just fine on my development server. On my development server, if I enter German umlauts (e.g. "äöü"), I get "???" in the DB. However, it works just fine on my production server. Things I tried: 1. I verified the text sent to the DB is valid (appears correctly in UTF8 encoded log files). 2. I file-compared "firebird.conf" on both production and development server and they are identical. 3. I tried using this Firebird SQL command to reset the collation "alter character set utf8 set default collation unicode_ci_ai;". 4. Adding 'CharacterSet=utf8' to my connection params (full connection code below). 5. If I use DBeaver to view the database structure, I see all the "varchar" columns have a "charset" set to "NONE". dbMain := TFDConnection.Create(nil); with dbMain.Params do Begin Add('DriverID=FB'); Add('Server=localhost'); Add('Database=c:\DB\myDB.FDB'); Add('User_Name=SYSDBA'); Add('CharacterSet=utf8'); Add('Password=my_password'); end; dbMain.Open; Any ideas?
  18. Yaron

    Firebird SQL 3.0.4 content encoding issue

    Just to conclude, after scouring the internet to find an elegant solution, I eventually had to brute-force it by creating new columns using this command : ALTER TABLE [TableName] ADD [NewColumnName] VARCHAR(100) CHARACTER SET UTF8; Then I used DBeaver's "Copy" & "Advanced Paste" to copy over the string data from the old columns to the new columns. I had to do this for approximately 10 tables with a combined total of around 60 columns. The whole process took about 1.5-2 hours.
  19. Yaron

    Firebird SQL 3.0.4 content encoding issue

    I think I figured it out, when I created the DB using DBeaver, it simply created all columns with a CHARSET of "NONE". Now I'm trying to figure out a way to automate the conversion from NONE to UTF8 as there doesn't seem to be any straight-forward way of making it happen.
  20. Yaron

    My open-source portfolio

    I released quite a bit of interesting code to github over the years: YouTube DATA API v3 parsing: https://github.com/bLightZP/Delphi-YouTube-Channel-parsing-plugin-for-Zoom-Player Basic RSS feed parsing code: https://github.com/bLightZP/Delphi-RSS-feed-parsing-plugin-for-Zoom-Player TheAudioDB MetaData/Image scraping code: https://github.com/bLightZP/Delphi-theaudiodb.com-Zoom-Player-media-scraping-plug-in TheMovieDB MetaData/Image scraping code: https://github.com/bLightZP/Delphi-themoviedb.org-Zoom-Player-media-scraping-plug-in OpenSubtitles.org subtitle search & scrape code: https://github.com/bLightZP/Delphi-OpenSubtitles.org-API-support-for-Zoom-Player A basic cross-platform calculator https://github.com/bLightZP/ElegantCalculator https://play.google.com/store/apps/details?id=com.inmatrix.ElegantCalculator Adapted old code to work as cross-platform pure-pascal image scaling with filters (bicubic, bilinear, etc): https://github.com/bLightZP/ImageInterpolation Adapted old code to work as a cross-platform drawing of an anti-aliased circle (can be modified to draw rount-rect as well): https://github.com/bLightZP/AntiAliasedCircle I forked a QRCode generating source code and greatly optimized it (~ x50 faster): https://github.com/bLightZP/DelphiZXingQRCode The original Delphi scanline color-conversion implementation was very slow, so I optimized it: https://github.com/bLightZP/OptimizedDelphiFMXScanline
  21. Using "with" can clean up the code to make it easier to read, but ever since the dawn of Delphi, it had a severe downside that "with" hinders debugging. You can't simply hover with the mouse cursor over a variable to get its value or use evaluate/modify on the variable itself without having to copy & paste the record/class name first. And now I've encountered a more recent issue with refactoring. If you use refactoring to rename a variable inside a record, it will miss any instances where the record was used with a "with" statement. Will this ever be improved? If not, what are your recommendations?
  22. I am using MARS as a back-end for a Web Application (processing post/get requests from HTML forms and outputting HTML based on the form input). I am currently deploying MARS as an IIS module DLL simply because I already have an IIS7.5 server installed and I wanted the free SSL security that comes with this type of setup. However, I had one instance where I was getting a "500" error from IIS and the only way to resolve it was to restart the application pool. Assuming my code is bug-free, what is the most reliable and best performance (for concurrent usage by multiple users) method to deploy MARS with SSL on a windows machine?
  23. Yaron

    With's the deal with "With"?

    I'm aware of past discussion with regards to debugging, I was just not aware that it also broke refactoring which is why I started the topic. Was there ever an official reply on an intent to resolve some of these issues?
  24. Due to Delphi's inability to register changes in included ({$I filename}) files when performing a standard compile, I am forced to do a full build which brings up warning/hints from underlying units. With MARS, I always see these 9 Hints: [dcc32 Hint] MARS.Core.JSON.pas(264): H2443 Inline function 'TJSONArray.GetValue' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Core.JSON.pas(384): H2443 Inline function 'TJSONArray.GetValue' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Core.JSON.pas(473): H2443 Inline function 'TJSONObject.GetPair' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Core.JSON.pas(822): H2443 Inline function 'TJSONArray.GetValue' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Core.MessageBodyWriters.pas(129): H2443 Inline function 'TJSONArray.AddElement' has not been expanded because unit 'System.JSON' is not specified in USES list [dcc32 Hint] MARS.Core.MessageBodyWriters.pas(275): H2443 Inline function 'TJSONArray.AddElement' has not been expanded because unit 'System.JSON' is not specified in USES list [dcc32 Hint] MARS.Core.MessageBodyReaders.pas(302): H2443 Inline function 'TJSONArray.GetValue' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Core.MessageBodyReaders.pas(366): H2443 Inline function 'TJSONArray.GetValue' has not been expanded because unit 'System.Generics.Collections' is not specified in USES list [dcc32 Hint] MARS.Data.Utils.pas(213): H2443 Inline function 'TJSONArray.AddElement' has not been expanded because unit 'System.JSON' is not specified in USES list Any merit to these hints?
  25. Yaron

    The 9 Hints in MARS

    I did, but it will get overwritten the next time I pull the code to get the latest version...
×