Stéphane Wierzbicki

Fwiw (closed as worked as expected) 

 

https://quality.embarcadero.com/browse/RSP-18337

 

Emb, please update your documentation or raise an exception on those methods not working with beginbatch/endbatch

Hello, 

 

I tried to call a TFDmemtable.locate function within a beginbatch/endbatch block and the function always return true (even when no record is in the table nor exists). 

 

Locate function is working as soon as I remove the beginbatch/endbatch block. 

 

Is this expected? 

I'll say Delphi 2007.

I'm using XE 10.3.1 and it is slow as a pig. 

Not to mention that *insights stuffs never ever fully worked... 

@Dmitry Arefiev yes I know 🙂

I'm so far pleased with the current speed. I'll check if I can use the Array DML within TMS scripter. 

 

You wrote a very efficient library... Kudos! 

@edwinyzh the culprit was the use of TFDTable instead of TFDQuery objects.

 

Speed is now almost the same. 

 

Speed difference is so huge between TFDTable and TFDQuery. I naively thought that TFDTable was the right choice when dealing with tables.

 

Do you know why there is such speed differences ? 

 

Anyway thank you for your help.

Hi,

 

I need to import some Excel data, modify it and export it into another format.

Input fields defs, output fields defs and script can be changed ar run time (lets say by a smart end user.

Data is persisted into an SQLite database.

 

Basically here is what I did: 

• Creating Input and Output TFDMemtable at run time (based on fields defs previously defined
• Injecting XLS data into Input TFDmemtable objects
• Passing Input and output TFDmemtable objects to scripter object
• Running script (cannot use DML for various reasons)

input.beginbatch;
Output.beginbatch;
try
 input.fisrt;
 while not input.eof do 
 begin 
   output.append;
   output.fieldbynem('blabla').AsString := input.fieldbyname('llkll').AsString;
   Output.post;
   input.next;
 end;
finally
 input.endbatch;
 Output.endbatch;
end;

• Persisting TFDMemtable data into SQLite database thanks to TBatchmove
• Exporting data into another format

 

This works pretty well (few secs for 10000 rows)

I decided then to remove all those TFDmemtable objects and replace them with TFDTable objects. Now it take 15x the time to complete. This is true specially on the script side.

 

To speed up the things I tried to wrap my script on a transaction

 

    try
      FLocalConnection.StartTransaction;
      fScripter.Execute;
        if FLocalConnection.inTransaction then
          FLocalConnection.commit;
        Log('Script Executed')

    except
      on e: exception do
      begin
        if FLocalConnection.inTransaction then
          FLocalConnection.rollback;
        Log('An error occured while executing the script: ' + e.Message);
      end;
    end;

I  tried to set TFDTable with this options

 

    FetchOptions.RecsMax := 500000; // Sample value
    ResourceOptions.SilentMode := true;
    UpdateOptions.LockMode := lmNone;
    UpdateOptions.LockPoint := lpDeferred;
    UpdateOptions.FetchGeneratorsPoint := gpImmediate;

And at least I tried to work with an :memory: database and pooled connection

 

  // close fdmanager
  FDManager.Close;
  // check if connection definition exists
  condef := FDManager.ConnectionDefs.FindConnectionDef(ConDefName);
  if not Assigned(condef) then
    condef := FDManager.ConnectionDefs.AddConnectionDef;
  // add connection definition
  condef.Name := ConDefName;
  condef.Params.DriverID := 'SQLite';
  condef.Params.Database := ':memory:';
  condef.Params.Pooled := true;
  condef.Params.UserName := '';
  condef.Params.Password := '';
  condef.Params.Add('SharedCache=False');
  condef.Params.Add('LockingMode=Normal');
  condef.Params.Add('Synchronous=Off');
  condef.Params.Add('JournalMode=Off');

  condef.Params.Add
    ('SQLiteAdvanced=auto_vacuum = 1;page_size = 4096;temp_store = MEMORY');

  condef.Params.Add('LockingMode=Normal');
  condef.Params.Add('CacheSize=60000');
  condef.Params.Add('BusyTimeOut=30000');
  condef.Apply;
  FDManager.Active := true;

 

Nothing changed so far.

 

Do some of you have any recommendation ?

 

 

 

1 hour ago, Dmitry Arefiev said:

http://docwiki.embarcadero.com/RADStudio/Tokyo/en/Using_SQLite_with_FireDAC#Adjusting_FireDAC_Mapping

Read sentence "For an expression in a SELECT list ..."

Well, thank you so much, I wasn't aware of this. This worked.

@Dmitry Arefiev do you have some advice on how to get this working? 

Testing 

 

cast(date('now') as date),

 

will return 2019

Thank you Dany, forgot about CAST (and I heavily used it on mySQL many years ago...) 

That's strange. I'm getting year value "2018" instead of "01/02/2018" for example... 

select CAST(date(DATE_OF_LOSS,'start of month','+1 month','-1 day') as DATE)   As DATE_OF_LOSS

Hello,

 

I have a problem that is driving me nuts. FireDac somehow "lost" the ftDate type after using the SQLite date function.

My SQLite table does have one field "DATE_OF_LOSS" defined as "DATE".

 

Field "DATE_OF_LOSS" will correctly be retrieved as a ftDate field type (for exemple 01/02/2018)

Select DATE_OF_LOSS from TABLE

This is no more the case when doing this. Field "DATE_OF_LOSS" will be retrieved as a ftString field type (for example 2018-02-01).

 select date(DATE_OF_LOSS,'start of month','+1 month','-1 day')   As DATE_OF_LOSS

Does anyone knows how to solve this ?

 

PS: using the latest Delphi RIO version, default SQLite connection parameters leaved to default. My OS Date time format is "dd/MM/yyyy"

It also happend here... but no way to reproduce !

@jbg many thanks

@jbg I've an issue while using the latest IDE Fixpack version.

It seems there is an incompatibility with TMS Aurelius: I'm getting EInvalidPointer exception after validating the TAureliusDataset "Load field definition" dialog box.

The error disappears after removing IDE FixPack 

5 hours ago, sjordi said:

... just to be answered that this is a known bug in the Quality Portal and I should follow the ticket to be notified when solved...

That's a *SHAME*

Is EMB a 5 guys running business ?

 

 

On 2/17/2019 at 12:31 PM, Dmitry Arefiev said:

 And more is comming in Updates.

Can you please eloborate ?

@mijn I guess that setup files are downloaded on the C drive (certainly in %temp% folder). Do you have enough free space? 

BTW where is your component available? 

Well, there is no more possible to disable this stupid theming stuff (from the IDE).

I've choose a custom color schema and as you see IDE is not correctly painted...

 

And what to say about all these flikers and time needed to display the new color schema.

 

 

Why don't you try by yourself? Download and install the Community version (or even the trial one) 

@jbg thank you for the confirmation.

I less and less understand Emb. This "new" IDE is so slow, flicker everywhere... 

 

Did they use their software? They should focus on quality rather than getting us such half backed solutions... 

 

 

Oops, they managed to put 1000 lines of code into 10.

 

Is this with the new update? 

Do we still need IDE fix pack for RIO 10.3.1 ?

Let's see after the installation 🙂