Mark Williams

Ran some tests on an in memory dataset of 25K records.

Without filter 31ms to iterate the whole dataset (whilst also querying each record to see if it matched the required "document_category_id)." .

Filtering first took twice as long to filter and iterate  even though the resulting dataset was only 30 records. That sort of answers my question. But would this still hold true with say a million plus records?

I need to work with a potentially large dataset in memory. It has an integer field called "document_category_id". On first loading the table it is sometimes necessary to update records in the table from one document_category_id to another. This might need to be done for several, but not all of the various document_category_ids.

Trying to work out what is the most efficient way of iterating.

Would it be more efficient to filter on the requored document_category_ids first and then iterate or to iterate on an unfiltered table?

Managed to work out a solution in the end. Sure there will be some issues I haven't considered, but seems to work ok. If anyone is interested here's the code:

procedure TForm3.TabControl2DrawTab(Control: TCustomTabControl;
  TabIndex: Integer; const Rect: TRect; Active: Boolean);
  var
    TabName:String;
    Pt : TPoint;
    R: TRect;
    TabUnderMouse:Integer;


begin
  Pt := TabControl2.ScreenToClient(Mouse.CursorPos);
  TabUnderMouse := TabControl1.IndexOfTabAt(Pt.X, Pt.Y);

  with (Control as TTabControl).canvas do
    begin
      R := Rect;

      if TabIndex=1 then
        Font.Style := [fsStrikeout];

      if (TabUnderMouse = TabIndex)and Active  then
        Font.Color := TStyleManager.ActiveStyle.GetStyleFontColor(sfTabTextActiveHot)
      else if Active then
        Font.Color := TStyleManager.ActiveStyle.GetStyleFontColor(sfTabTextActiveNormal)
      else if (TabUnderMouse = TabIndex) then
        Font.Color := TStyleManager.ActiveStyle.GetStyleFontColor(sfTabTextInActiveHot)
      else
        Font.Color := TStyleManager.ActiveStyle.GetStyleFontColor(sfTabTextInActiveNormal);

      tabName := TTabControl(Control).Tabs[TabIndex];

      Brush.Style := bsClear;

      DrawText(Handle, PChar(TabName), Length(TabName), R,  DT_SINGLELINE or DT_VCENTER or DT_CENTER)
  end;
end;

I want to show a TabControl with tabs with different font styles. Basically, I want the font in some of the tabs to be shown in strikeout. The app uses a couple of different themes. I thought I'd use the StyleManager to achieve this with the following code:

 with (Control as TTabControl) do
    begin
      tabName := TTabControl(Control).Tabs[TabIndex];

	  if TabIndex=1 then
        Canvas.Font.Style:=[fsStrikeout];


      Pt := TabControl2.ScreenToClient(Mouse.CursorPos);
      TabUnderMouse := TabControl1.IndexOfTabAt(Pt.X, Pt.Y);

      if Active then
        Details := TStyleManager.ActiveStyle.GetElementDetails(ttTabItemSelected)
      else if (TabUnderMouse=TabIndex) then
        Details := TStyleManager.ActiveStyle.GetElementDetails(ttTabItemHot)
      else
        Details := TStyleManager.ActiveStyle.GetElementDetails(ttTabItemNormal);

      TStyleManager.ActiveStyle.DrawElement(Canvas.Handle, Details, Rect);

      TStyleManager.ActiveStyle.DrawText(Canvas.Handle, Details, TabName, Rect,
        DT_VCENTER or DT_CENTER, Canvas.Font.Color);
    end;

This sets the colour of the tab background perfectly depending on its state, but has no impact on the font which is always black. However, the font is always black whatever the state of the tab.

The color parameter in the DrawText function does not appear to do anything. I've tried setting the canvas font color before calling Drawtext to the stylefontcolor of the tab element in its various states. It has no effect, not does entering any color whatsoever in the color parameter.

4 hours ago, KryptoMed said:

Thank you Mark for the info. CoolSelector from Morten behaviours also very good . (http://coolselector.danfoss.com/ ) .

Seems to be spam.

I've used it on one smallish app so far. It works perfectly with just one line of code. You may need to tweak a few things if you want to open documents, but it is very straightforward.

It took a little bit of time to get my server set up correctly and I needed support from Thinfinity to do this, but the support was superb.

On 4/1/2024 at 7:57 AM, Cristian Peța said:

I suppose you are using TXMLDocument. Then why not using TXMLDocument.LoadFromStream and TXMLDocument.SaveToStream and let the library do the encoding work for you?

That seems to be the answer. Many thanks. I think I initially went with LoadXMLData because it created the XMLDocument for you in the same step. A false economy it seems!

I have an ISAPI DLL written in Delphi 10.4 running on my server. I exchange information in XML format with the DLL via an app also written in Delphi 10.4.

Both the server and the user computer are using Microsoft XML 6.0.

I use the following function to convert the exchange xml streams to a string and then to load into IXMLDocument.

Function ConvertStreamToString(stream:TMemoryStream):String;
  Var
    ss:TStringStream;
begin
  if (Stream <> nil) and (stream.Size>0) then
  begin
      Stream.Position:=0;
      ss:=TStringStream.Create;
      try
        ss.CopyFrom(stream, 0);
        Result:=ss.DataString;
      finally
        ss.Free;
      end;  
  end
  else
    Result := '';
end;

I load the xml data with:

Doc := LoadXMLData(ConvertStreamToString(Response));

This works fine for data received from the server, but not for data submitted to the server.

The data is received complete by the server (I save the received stream data to file to test), but something goes wrong with the convertStreamToString function and the LoadXMLData function throws up an error:

Quote

A Declaration was not closed.

Line: 1

<

If I change the stream conversion function to:

SetString(Result, PChar(Stream.Memory), Stream.Size div SizeOf(Char));

This works for the DLL on the server, but fails on the local app. It returns a load of junk throwing up an AV on the call to LoadXMLData.

I could have a different functions for the server and the local app, but I would rather have some idea as to why this is happening. I guess it is to do with encoding, but can anyone give me a steer as to how I resolve it please?

22 hours ago, weirdo12 said:

😄 So just supply the current value when the value doesn't change. I do like the statement that uses COALESCE because it makes it very obvious that that is what is happening.

Yes. As I explained that would work, but I would have to submit a mass of additional data to my server that is unnecessary if you use COALESCE.

22 hours ago, weirdo12 said:

Are you targeting a specific database server?

No. Trying to write it as flexible as poss

13 hours ago, Brian Evans said:

Null parameters and COALESCE () in the UDPATE statement would be one way.

SQL.Text :='UPDATE persons SET name=COALESCE(:NAME,name), email=COALESCE(:EMAIL,email) WHERE id=:ID'; 

 

You're a genius. I'm an idiot. Why didn't I think of COALESCE. Thanks

22 hours ago, weirdo12 said:

Why not just supply the current value where you don't want a value to change?

Possible, but I'm posting data to a server which then submits the query and it could be an awful lot of unnecessary data to post.

I'd like to use Array DML to update a table, however, I don't want to update all fields every time. In some instances I would like to leave the existing database value untouched for a particular record.

In a normal update query I would do something like "UPDATE persons set name=name". I can't see how it's possible to do that for Array DML.

      SQL.Text :='UPDATE persons SET name=:NAME, email=:EMAIL WHERE id=:ID';

      Params[0].DataType:=ftString;
      Params[1].DataType:=ftString;
      Params[2].DataType:=ftLargeInt;


      Params.ArraySize:=2;

      Params[0].AsStrings[0]:='Delphi';
      Params[1].AsStrings[0]:='praxis';
      Params[2].AsLargeInts[0]:=794;

      Params[0].AsStrings[1]:='Mark';      
      Params[2].AsLargeInts[1]:=795;

      Execute(Params.ArraySize);

The first set of values is intended to overwrite name and email fields.

The second set of value is intended to overwrite just the name field and leave the email field as is in the database. I thought (but not with much optimism) that not seeing the parameter for the email field may leave it as is. In fact it sets the value to an empty string.

I can't see any method or property of TFDParam that seems to fit the bill. Was hoping there might be an "AsIs" property, but sadly not.

I'm sure it can't be the case that I would need to submit multiple array DML's depending on which values I want to change and which ones I want to leave untouched.

Should have mentioned @J. Robroeks in the original post as J posted the solution referred to on SA.

I posted a question on this a while back. Trying to establish whether there was a way of retrieving the ids for new rows inserted using ArrayDML. I had tried using "returning id". That didn't work or rather it didn't work as I had hoped. From memory it inserted just one row and returned the id for that row,

By chance I came across this post on stackoverflow:

https://stackoverflow.com/questions/66704349/firedac-array-dml-and-returning-clauses

I have tried this and get "syntax error at or near {". 

I can't find anything on "INTO" in this context. 

Also, using PostgreSQL to test.

I'm trying to use TControlBar for the first time to host controls, mainly TToolBars

I have found that when you copy toolbars into the TControlBar it often breaks the toolbar over two rows. This seems to happen where there are 8 or more buttons.

I can't find any property either of the controlbar or toolbar to prevent this happening. Most hopeful was TToolbar's Wrappable property, but setting this to false does nothing when the toolbar is placed in a controlbar.

I have found the solution. I was working in idiot mode. The path I was trying to add was already in there.

I haven't tried to add a new library path for at least  year and now I can't add anything. Whereas the Add button is meant to enabled once you select a folder, it no longer does so.

Also, if I select an existing path the Replace button does not enable.

Anyone have any ideas what may cause this?

16 hours ago, Lajos Juhász said:

You should post a test application

Unit2.pas

Unit2.dfm

I have made slight changes. Both of my own functions are now called with the same parameters.

The strUtils one works as expected. The AnsiStrings one reports -1

Thanks

Thanks to Stano the function now compiles. However, I cannot get the AnsiStrings function to work as I would expect ie in the same way as the strUtils unit.

The code is:

	{Ansitrings:}
   Function FindWordInText(Const Text:PAnsiChar; Const FindText:AnsiString; index:integer):Integer;
{StrUtils:}
	Function FindWordInText(Const Text:PWideChar; Const FindText:AnsiString; index:integer):Integer;
  var
	{AnsiStrings:}
    	P:AnsiChar;
	{StrUtils}
		P:PWideChar;
    Size:word;
    Options:TStringSearchOptions;
begin
  Size := length(text);
  //Buffer := StrAlloc(Size+1);

  Options:=[soDown];
  
  P := {System.AnsiStrings.}SearchBuf(text, Size , 0, index,
  FindText, options);


  if P <> nil then
    Result:=P-text;
end;

StrUtils function correctly returns the position of a word within the text, AnsiStrings function always returns -1.

I assume it is to do with the difference between PWideChar and PAnsiChar, but how do I handle that please within this function?

Sorry should have spotted that myself, although the help file in the StrUtils unit only refers to the functuion being deprecated for C++.

I am now getting an "Ambiguous overloaded call to SearchBuf" error.

I am calling SearchBuf in a functions declared as follows:

Function FindWordInText(Const Text:PAnsiChar; Const FindText:AnsiString; caseSens, WholeW: boolean; index:integer):Integer;
  var
    P:PAnsiChar;    
    Size:word;
    Options:TStringSearchOptions;
begin
 {irrelevant code removed}
  P := SearchBuf(text, Size , 0, index,
  FindText, options);
end;

After changing my reference to strUtils in uses to AnsiStrings this would not compile. Reason being that TStringSearchOptions is in strUtils. So I also added StrUtils back in to my uses after AnsiStrings. As soon as I did so, I get the ambiguous error.

The parameters passed into my function are correct for the SearchBuf function in AnsiStrings unit. I also get the same error if I add the AnsiStrings unit after StrUtils in my uses clause.

If I change my functions parameters as follows:

Function FindWordInText(Const Text:PWideChar; Const FindText:AnsiString; caseSens, WholeW: boolean; index:integer):Integer;
  var
    P:PWideChar;    
    Size:word;
    Options:TStringSearchOptions;
begin
  P := SearchBuf(text, Size , 0, index,
  FindText, options);
end;

It compiles and executes fine. Presumably, it is calling the StrUtils function. 

But why am I getting an ambiguous error with PAnsiChar as a parameter, but not PWideChar?

I am using Searchbuf from SysUtils. I am getting the above warning. Often when the compiler warns that a function is deprecated it points you to a new function that performs the same or advanced role. Not in this case. Anyone know what if anything has replaced SearchBuf?

7 hours ago, Alexander Elagin said:

I have a zipped Virtual Treeview archive from 2004, it contains packages for Delphi 4 - Delphi 9 (2005?). If anybody is interested, I can share it. Also found the installer for VTV 4.8.7, dated 15.12.2010.

Yes please

On 1/27/2022 at 3:19 PM, Fr0sT.Brutal said:

Have you cloned whole repo? Mine starts from 2009 with 4.8 mentioned

I have cloned the repository, but can find no reference to 2009. You mention cloning the "whole" repository. I guess I am not doing that. How do you get the whole repository from Github?

8 hours ago, Dalija Prasnikar said:

If application worked correctly before the changes, then those code changes will be the cause of the issue, not the TreeView. I suggest going back through your changes history, finding out the last good version of your code. By comparing the code between good and bad version you can more easily pinpoint the real cause.

Because there are so many changes and the issue is random, I have been trying to avoid a bit by bit reversal, but it is looking as if I have no alternative!

On 1/27/2022 at 12:33 PM, Fr0sT.Brutal said:

Most likely - double free when VTV's OnFreeNode disposes linked object and it also gets freed somewhere else

Double-checked all tree code. The tree data does not link objects. They reference objectlists, but the tree data merely references an index to the objects in the objectlist. I've double-checked that I'm the object referenced by the index stored in tree data exists before attempting to refer to it. All that seems okay.

I've also wrapped all freenode event code in try except.

Still at a loss as to what is causing the problem.