Brian Evans

The zip file came from another computer (the Internet) and is probably marked as such. Perhaps some copied files are getting the flag set in some circumstances? Try checking the unblock checkbox before extracting/copying.

FireDac has Local SQL (FDLocalSQL) which can be used to run SQL against TDataSets. Uses SQLite underneath the hood with some extra bits. Using with some FDMemTable components would keep it all in memory. Local SQL (FireDAC) - RAD Studio (embarcadero.com)

There is no built-in query editor in Delphi for TDataSet descendants. Some third party components provide them and some include being able to run the query. One example is FireDac where it accessed by double left clicking or right clicking the component. The dbGo components provide no such editor.

A lot of SQL is executed with parameters which would be a much easier to use form of the data than logs containing SQL with values inline. It also reduces clutter as static values show up as such vs looking the same as a parameter that might change. Where are these logs coming from? Can they be updated to log SQL + parameter value list?

Likely the reconnect failed and latter on after the error dialog is closed the connection gets opened again and the application continues on. A FDConnection has some events to give you a chance to handle exceptions or log what is happening: OnError, OnLogin, OnLost, OnRecover and OnRestored. If you add some logging to OnLost with a timestamp you should be able to tell what is going on and when.

A quick look at the file (searched Bing for the 2022VAERSDATA.csv) and it looks like double quotes within a string field are escaped with a double quote. As well only string fields that contain commas or double quotes appear to be enclosed in double quotes. Will take a more detailed look when I get a chance. LOAD DATA local INFILE 'c:/programdata/mysql/mysql server 8.0/uploads/2022VAERSDATA.csv' INTO TABLE data CHARACTER SET latin1 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"' IGNORE 1 LINES

The MySQL LOAD DATA statement can certainly handle all of that not sure why you think otherwise? MySQL :: MySQL 5.7 Reference Manual :: 13.2.6 LOAD DATA Statement The [{FIELDS | COLUMNS} [TERMINATED BY 'string'] [[OPTIONALLY] ENCLOSED BY 'char'] [ESCAPED BY 'char'] can handle double quotes. The SET col_name={expr | DEFAULT} section can be used for date, Boolean or other field conversions as needed.

I tend to move it all into a DataModule (one per thread type/definition) so it ends up under an instance variable for that namespace wise.