ANN: Sempare Template Engine for Delphi

[darnocian 83]

I'd like to advertise the Sempare Template Engine for Delphi. The Sempare Template Engine for Delphi allows for flexible text manipulation. It can be used for generating email, html, source code, xml, configuration, etc.

 

It is available on github via https://github.com/sempare/sempare-delphi-template-engine

It is also available via Delphinus (https://github.com/Memnarch/Delphinus)

 

Simply add the 'src' directory to the search path to get started.

 

Sample usage:

program Example;
uses
    Sempare.Template;
type
    TInformation = record
        name: string;
        favourite_sport : string;
    end;
begin
    var tpl := Template.parse('My name is <% name %>. My favourite sport is <% favourite_sport %>.');
    var information : TInformation;
    information.name := 'conrad';
    information.favourite_sport := 'ultimate';
    writeln(Template.eval(tpl, information));	
end.

 

Features include:

• statements
  • if, elif, else statements
  • for and while statements
  • include statement
  • with statement
  • function/method calls
• expressions
  • simple expression evaluation (logical, numerical and string)
  • variable definition
  • functions and methods calls
  • dereference records, classes, arrays, JSON objects, TDataSet descendants and dynamic arrays
  • ternary operator
• safety
  • max run-time protection
• customisation
  • custom script token replacement
  • add custom functions
  • strip recurring spaces and new lines
• lazy template resolution
• parse time evaluation of expressions/statements
• allow use of custom encoding (UTF-8 with BOM, UTF-8 without BOM, ASCII, etc)
• extensible RTTI interface to easily dereference classes and interfaces (current customisations for ITemplateVariables, TDictionary, TJsonObject)
•  

There are numerous unit tests that can be reviewed as to how to use the various features.

 

Happy for all to play with it. Released under GPL and Commercial License.

 

Any feedback welcome.

[Edwin Yip 154]

This library fills the gap of the missing of a real powerful template engine in the Delphi community.

DMustache is good, but it's logic-less. I'm not talking about it's bad - on the contrary, it's good for its use cases, but we need more powerful scripting in other use cases.

Had been working with the developer for XE4 compatibility few days ago (initially more modern language features were used), and he was able to quickly get thing sorted out without an XE4 compiler. Really impressed!

[mvanrijnen 123]

Was looking for something like this.

Created myself a few half working "engines" (just for own needs), Hopefully can replace those with one good one 🙂

 

[darnocian 83]

It is now available via GetIt as well

 

[darnocian 83]

A new version (v1.3.1) is available on github with bug fixes (https://github.com/sempare/sempare-delphi-template-engine)

 

Hope to have the update in getit shortly too.

[mvanrijnen 123]

Looking now to implement the engine in an application here,  but when i have a template like:

 

Value: <% value %>
Description: <% description %>

 

I get double crl/lfs in the result?

 

 

[edit]

after inspection it seems like that the Template.Eval function adds an extra CR is added.   (e.g.  #$D#$A becomes #$D#$D#$A)  

 

quickfix: 🙂

Result := Template.Eval(FParsedTemplate, objrec).Replace(#13#13, #13, [rfReplaceAll]);

 

Edited January 18, 2021 by mvanrijnen

[darnocian 83]

Thanks for the report.

 

Could you please raise the issues on https://github.com/sempare/sempare-delphi-template-engine/issues

 

I'll check it out. A double #$D would not be ideal. The #$A is more easily noticeable, but most of the stuff I used was HTML where this would have been ignored.

[darnocian 83]

I just looked at the tests... mostly testing with #$A. 😉 But made a test to replicate the issue. 

[mvanrijnen 123]

13 hours ago, darnocian said:

I just looked at the tests... mostly testing with #$A. 😉 But made a test to replicate the issue. 

thnx.

 

other question, is it possible to use the engine multithreaded? 

(eg, can i have more engine instances in one process? )

 

[darnocian 83]

48 minutes ago, mvanrijnen said:

is it possible to use the engine multithreaded? 

Yes. Use in a multi threaded environment should be totally fine. I've even used it to serve html on an indy server.

 

There is locking used internally when accessing some internally managed collections as required.

 

You can load templates on the fly and the locks ensure the state of the structures don't get corrupted. You can load/parse templates upfront or on demand safely. There is no global/shared state that should cause any issues. 

 

There are a few different Template.Eval() overrides which allow you to use the templates that have been pre-parsed, or you can just reprocess a textual representation as required. When evaluation takes place, the state is totally independent of any other execution on other threads.

 

 

 

 

 

Edited January 19, 2021 by darnocian

[darnocian 83]

@mvanrijnen Just looking at the bug you highlighted, there is also an alternative workaround by setting the IContext.NewLine = #$A.

procedure TTestTemplate.TestNewLine;
type
  TRec = record
    Value: string;
    description: string;
  end;
var
  r: TRec;
  s: string;
  ctx: ITemplateContext;
begin
  ctx := Template.Context();
  ctx.newline := #10;
  r.Value := 'a value';
  r.description := 'some desc';
  s := Template.Eval(ctx, 'Value: <% value %>'#$D#$A'Description: <% description %>', r);
  Assert.AreEqual('Value: a value'#$D#$A'Description: some desc', s);
end;

I have a custom StreamWriter that allows for stripping repeated whitespace and new lines. I'll look into also providing a way to override that better.

[darnocian 83]

A fix is now in the dev branch with some additional tests and a way to also change the StreamWriter if required. Will look at new tagged release next week or so.

 

(with the fix in the dev branch, the the Context workaround above would be invalidated)

Edited January 19, 2021 by darnocian

[mvanrijnen 123]

ok, tnx for the quick response 🙂

For now, it's ok how it is, i will check next week for update. 

 

 

[darnocian 83]

Hi,

 

Just a small announcement that a new version (v1.4.0) has been released.

 

Changes:

1. NEW: Context.StreamWriterProvider which allows greater flexibility in providing custom overrides to the StreamWriter
2. NEW: Support for statement start and end tokens allowing for content to be swallowed (useful when statements are multi line)
3. NEW: Added helper variables and functions for spaces, newlines, tabs, chr() and ord()
4. FIX: Issue with custom text writer that supported a few options regarding newlines, but had a bug when it came to carriage returns. (not noticeable under html) 
5. FIX: Fixed a double free bug in a ParseFile helper.
6.  UPDATE: Documentation updates 

 

Context.StreamWriterProvider flexibility
 

  var ctx := Template.Context();
  ctx.StreamWriterProvider := function(const AStream: TStream; AContext: ITemplateContext): TStreamWriter
    begin
      result := TStreamWriter.Create(AStream);
   end;

The above is an example of how the override can be done on a per context basis. There is a GDefaultStreamProvider which is used when the context is initially created which can be used for global initialisation.

 

Allow swallowing of content between start and end statement tokens

Say you have a scenario like:

<% for i := 1 to 5 %>
  
  <% i %>
    
<% end %>

What you may notice is that there are many newlines that appear something like:

    1

    2

    3

    4

    5

 

Now consider the following:

<% for i := 1 to 5 |>
 <% i %>
<| end %>

In this scenario, all normal output between |> and <| are ignored, except statements such as explicit print() or variable references will work as normal.

 

Why? It just means that the template becomes responsible for any indenting and newlines within the end (|>) and start ( <|) tokens.

 

---

 

If you have any feature requests or bug reports, please raise them on https://github.com/sempare/sempare-delphi-template-engine/issues

 

 

 

 

 

 

 

 

[Rollo62 534]

I like the mustache notation {{}} a lot, is it possible to make this compatible (at least a bit), to Mustache ?

(Naive approach: Maybe its just replacimg "<%"   "%>" by "{{"   "}}"  ) ?

[mvanrijnen 123]

20 minutes ago, Rollo62 said:

I like the mustache notation {{}} a lot, is it possible to make this compatible (at least a bit), to Mustache ?

(Naive approach: Maybe its just replacimg "<%"   "%>" by "{{"   "}}"  ) ?

Yes, would be nice if this would be configurable. 

In the solution i  now have, i have to parse on server, and sometimes a second time on clients.  If this is configurable, i can decide in the template where things are resolved.

 

Edited January 25, 2021 by mvanrijnen

[darnocian 83]

3 hours ago, Rollo62 said:

I like the mustache notation {{}} a lot, is it possible to make this compatible (at least a bit), to Mustache ?

(Naive approach: Maybe its just replacimg "<%"   "%>" by "{{"   "}}"  ) ?

I tried to provide flexibility in many places. https://github.com/sempare/sempare-delphi-template-engine/blob/master/docs/configuration.md

 

begin
  var ctx := Template.Context;
  ctx.StartToken := '{{';
  ctx.EndToken := '}}';
  Assert.IsEqual('hello', Template.Eval(ctx, '{{ if true }}hello{{else}}bye{{end}}'));
end;

There is a restriction however. Must be 2 characters in length. It will work as long as it doesn't conflict with any of the other tokens (no validation is done on this however, so you just need to check if you do override)

 

There is also a global override where you can set GDefaultOpenTag and GDefaultCloseTag. These are defined in Sempare.Template.Context.pas. Once set, you don't have to explicitly create a context if you don't need one.

Edited January 25, 2021 by darnocian

[Rollo62 534]

Great, I will check that

[mvanrijnen 123]

Little problem with records, not sure if its Sempare or just delphi ande rtti on records

some quick test code:

 

 

TTestRecord=record
    ID : Integer;
    TypeID : integer;
    Description : string;
    Vehicle : string;
    ChildRecords : TArray<TTestRecord>;

    procedure Init;
    function SubResource(const AType : Integer; var AResource : TTestRecord) : Boolean;
    function SubResources(const AExceptTypes : TIntegerSet) : TArray<TTestRecord>;
  end;

 

const
  CNST_HTMLTEMPLATE_RESOURCE =
    '<font  face="calibri" size="12">'+
    '<b>(<% id %>) <% description %></b><br><br>'+
    '<% Vehicle %><br>'+
    '</font>'+
    '<font  face="calibri" size="10">'+
    '<% for subres in ChildRecords %>SubRes: |<% subres.Description %>|<br><%end%>'+
    '</font>';

 

var
  tpl : ITemplate;
  v1,v2 : TTestRecord;
  r : TTestRecord;
begin
  tpl := Template.Parse(CNST_HTMLTEMPLATE_RESOURCE);
  v1.Description := 'xx11-22-33';
  v1.ID := 1;
  v1.TypeID := 2;
  v2.Description := 'yy4455-66';
  v2.ID := 2;
  v2.TypeID := 2;

  r.Description := 'maurits';
  SetLength(r.ChildRecords, 2);
  r.ChildRecords[0] := v1;
  r.ChildRecords[1] := v2;
  r.Init;
  mmoResult.Text := Template.Eval(tpl, r);

  showmessage(r.Vehicle);
end;

 

i get not the result i expected on the subres.description, 

I get 2 times SubRes:

but the description field stays empty in the parsed result?

 

[darnocian 83]

I think subres should be an index into ChildRecords.

 

 Try change:

1 hour ago, mvanrijnen said:

'<% for subres in ChildRecords %>SubRes: |<% subres.Description %>|<br><%end%>'+

to

'<% for subresIdx in ChildRecords %>SubRes: |<% ChildRecords[subresIdx].Description %>|<br><%end%>'+

 

I've just renamed the variable as well to make it clearer.

 

It may seem undesirable, but did it this way as sometimes you want to know where you are in the array.

 

If you have many variables, you should be able to do:

<% with ChildRecords[subresIdx] %> <% Description %> <% Vehicle %> <% end %>

[darnocian 83]

BTW.  The new release v1.4.0 is now also available on GetIt (https://getitnow.embarcadero.com/?q=sempare&amp;product=rad-studio)

[mvanrijnen 123]

1 hour ago, darnocian said:

I think subres should be an index into ChildRecords.

 

 Try change:

to

'<% for subresIdx in ChildRecords %>SubRes: |<% ChildRecords[subresIdx].Description %>|<br><%end%>'+

 

I've just renamed the variable as well to make it clearer.

 

It may seem undesirable, but did it this way as sometimes you want to know where you are in the array.

 

If you have many variables, you should be able to do:

<% with ChildRecords[subresIdx] %> <% Description %> <% Vehicle %> <% end %>

thnx, works like a charm like this. 

 

[mvanrijnen 123]

New problem 🙂

 

Template:

Longitude1: AdrLongitude
Longitude2: <%AdrLongitude%>
Longitude3: <%fmt('%7.4f', AdrLongitude)%>
Longitude4: <%fmt('%7,4f', AdrLongitude)%>

 

Output:

Longitude1: AdrLongitude
Longitude2: 5,92406892776489
Longitude3: -?,?<E3523
Longitude4: 

 

Question:

How to get fmt to work properly? (using Dutch Windows, EU-VS-Keyboard, Dutch Settings, som the decimalsep is a comma on this system)

 

 

see also

<%x := 5%>
X=<%x%>

<%y := 5.1234%>
Y=<%y%>

the part with y can not be evaluated/parsed, wether i use a comma (,) or a dot (.)

 

 

 

Edited February 8, 2021 by mvanrijnen

[darnocian 83]

I created a test and the following is runs properly:
 

Assert.AreEqual('543.21', Template.Eval('<% x:= 543.21 %><% x %>'));  

Assert.AreEqual('5.1234', Template.Eval('<% x:= 5.1234 %><% x %>'));

floating point numbers should be using dot (.) as illustrated above.

 

I'm not sure about fmt() offhand. Will investigate. fmt() is just a wrapper around SysUtils.Format(fmt, ...). Might be some issue with TValue to TVarRec conversion, but will double check. Most of the templates I've worked on passed values into the engine via the input record/class, but appreciate this should not be a problem.

 

BTW. it is better to raise issues on github. https://github.com/sempare/sempare-delphi-template-engine/issues

Edited February 11, 2021 by darnocian

[darnocian 83]

 Assert.AreEqual('123.457', Template.Eval('<% x := 123.456789 %><% fmt("%6.3f", x) %>'));

here is a quick fix in Sempare.Template.Functions.pas:

class function TInternalFuntions.Fmt(const AArgs: TArray<TValue>): string; 
var
   LArgs: TArray<TVarrec>;
   LIdx: integer; 
begin
   setlength(LArgs, high(AArgs));   
   for LIdx := 1 to high(AArgs) do     
       LArgs[LIdx - 1] := AArgs[LIdx].AsVarRec;   
   exit(format(AsString(AArgs[0]), LArgs)); 
end;

I need to check into why the current helper does something different. Will get a fix into the main release soon. (fix is on dev branch)

 

 

Edited February 11, 2021 by darnocian