TDFBatchMove.GuessFormat - How Many Rows Does It Read?

[weirdo12 16]

If the TDFBatchMove.Reader property is a TFDBatchMoveTextReader when GuessFormat is called, how many rows does it read from a file to determine the format of the columns?

 

The issue I have is an import file where a column normally has the value 1 and then say 10,000 rows in it will change to 2.2046 which kills the import.

Edited February 23, 2023 by weirdo12
typo

[programmerdelphi2k 235]

my "TFDBatchMoveTextReader" = option = WithFieldNames = true, but you can config it for another approach (... all FDBatchMoveXXXX  too!)

The procedure will read the first line of text, to check which is the delimiter used, then each "text" collected will be used to check its "type", if it is not of type "string", then it is not the list of fields, but the data.
then a new check to see some data from the lines that represent the records...
This will create a new list to compare the types of each column in the record.
in tests, only 8 records are read before completing the final analysis.

each line is readed and analized using a stream, like TStream way. Then, basically there is not limit, just 1 line each iteration! (until 8191 chars or EOL/#13/#10 chars)

Tested using table "Employee" demo!

 

Quote

FireDAC.Comp.BatchMove.Text.pas, line 1360 (RAD 11.2)  

 

Edited February 23, 2023 by programmerdelphi2k

[Uwe Raabe 1891]

7 hours ago, weirdo12 said:

The issue I have is an import file where a column normally has the value 1 and then say 10,000 rows in it will change to 2.2046 which kills the import.

TFDBatchMove provides the AnalyzeSample property to control that.

[programmerdelphi2k 235]

AnalizeSample is used in "FireDAC.Comp.BatchMove.Text.pas, line 1360 (RAD 11.2) " for that, for sure

[weirdo12 16]

Oh boy, how did I miss that TFDBatchMove.AnalyzeSample property!?

 

After more thought this morning, I doubt it's going to help with my problem. For example if the text file was like like this:

 

"row_id","float_col"

1,1

2,2.2046

3,1

 

float_col will likely still be guessed to be an integer. I will have to look at the source.

 

 

[programmerdelphi2k 235]

in "TFDBatchMoveTextReader" you can define the Fields/Format expected!!!

[weirdo12 16]

26 minutes ago, programmerdelphi2k said:

in "TFDBatchMoveTextReader" you can define the Fields/Format expected!!!

 

Yep. That particular property did not elude me.