darnocian

For reference, I raised https://quality.embarcadero.com/browse/RSP-40056 in Nov 22 to include the missing fiber functions.

@Remy LebeauYes, I've seen Raymond Chen's blog. 

However, in many languages, coroutines are popular. Fibers is one way to implement them.

Generally, I would say the argument would be that most people would not use them directly, but possibly a wrapper or language feature would utilise fibers to provide some interesting behaviour. I would say, it is pointless including fiber functions in the windows unit without these as they provide context around whatever needs to be done. Agreed - nobody but me has missed probably them as they have been excluded for so long, or nobody bothered raised a QP ticket for the omission. Unfortunately, Delphi is not in the list of languages supporting coroutines either. ;(

Hi

I created a ticket a while back on Quality Portal for some missing Windows SDK functions for Fibers. I created a small test project to illustrate them working in https://github.com/sempare/sempare-delphi-fiber-generator.

For those that don't know, fibers essentially are light weight threads, where context switching between fibers is managed by the developer and not by the OS, as is done with threads. It can be complex to get your head around them if you are not familiar with them. For a more detailed discussion, here is a nice article: https://nullprogram.com/blog/2019/03/28/ and https://learn.microsoft.com/en-us/windows/win32/procthread/fibers. I think the reason they were omitted in the Delphi Windows unit is because the tool that did the main conversion of the Windows SDK functions was working with normal C functions, where the missing functions I've included were defined as C macros.

From a Delphi perspective, GetFiberData annd GetCurrentFiber are available in the unit https://github.com/sempare/sempare-delphi-fiber-generator/blob/develop/src/Sempare.Win32.Fiber.pas for win32 and win64.

In the project, the unit https://github.com/sempare/sempare-delphi-fiber-generator/blob/develop/src/Sempare.FiberGenerator.pas contains a TGenerator<T> which switches between the procedure producing values and the main method that is consuming them. Here is an example of it in use with a fibonacci generator.

Quote

procedure TGeneratorTest.TestFibonacci;
var
  LGenerator: TGenerator<integer>;
  LSeq: TArray<integer>;
  i, j: integer;
begin
  LGenerator.Producer := procedure
    var
      i, j, t: integer;
    begin
      i := 1;
      j := 1;
      TGenerator<integer>.Yield(1);
      TGenerator<integer>.Yield(1);
      while true do
      begin
        t := i + j;
        TGenerator<integer>.Yield(t);
        i := j;
        j := t;
      end;
    end;
  LSeq := [1, 1, 2, 3, 5, 8, 13, 21, 34, 55];
  for i := 0 to 9 do
  begin
    j := LGenerator.Value;
    assert.AreEqual(LSeq, j);
  end;
end;

So what is going on here.... 

A producer is a procedure that must call Yield(value). So we know that with Fibonacci, f(0) =1, f(1) =1, f(n) = f(n-1) + f(n-2). Rather than having a recursive routine, the producer method simply Yields the values appropriately. Yes, it looks like the method will never stop. That doesn't matter. Once the method has a value to yield, the context is switched back to the main bit of code that is requesting the value ... that where we have:

Quote

j := LGenerator.Value;

As mentioned, this is just a simple demo of using fibers. Another project I had that has a bit more complex utilisation of fibers was for async IO on sockets - the principle was to make async calls with call back functions appear synchronous, which is what most people are more comfortable with. Even though most people will never need to use the fiber methods directly as I illustrated above with the TGenerator example, it is quite a powerful tool in the toolbelt.

Hi all,

I'm pleased to announce the new release (v1.7.2) of the Sempare Template Engine. 

This release includes:

• NEW
  • manage/unmanage methods for managing objects created while the template is being evaluated.
  • Template Engine Playground enhancements (now using the Carbon theme, shows template evalution time in ms, extract variables form a template)
  • json/map like variable support 
• FIX
  • removed caching of TRttiMethod to ensure virtual methods are supported correctly
  • fixed regression of TValue type coercion

Older release notes can be viewed on the releases page.

Have fun.

That looks like your file system may be corrupted if a .pas is opening that way. Worth looking at a backup (hopefully you have external version control like github). Worth running a chkdsk on your drive and check other sources too that you may rely on.

Hi all,

I'm pleased to announce the new release (v1.7.0) of the Sempare Template Engine. It has been a while, and thought I'd just remind any users here about the project. 

Also included are some demos... for apps in general, there is a new helper class TTemplateRegistry, which should make things much easier to just get going and supports template reloading without restarting the app and can load templates from file, resources, or custom a loader. This takes some of the grunt work out of working with templates, besides just using them. More information is available in the docs on the repo. Some simple examples are provided to illustrate server side scripting. If you have any suggestions, or see any gaps, please feel free to provide comments on the issue tracker.

- Demo using the Horse framework

- Demo using WebBroker standalone console app

New features:

- improved support for comments

- extends/block template support

- improved whitespace removal

- renamed demo app to Sempare Template Engine Playground - a useful tool for testing language features / templates.

- added the Sempare.Template.RCGenerator project to help with creating resource rc files by scanning directories.

- support multiple statements in a script block. e.g. <% print('a'); print('b') %>

- additional context configuration 

- improved docs (improved navigation and added railroad diagrams to illustrate the grammar)

Older release notes can be viewed on the releases page.

Have fun.

Maybe better to have a callback function that can be used for debugging. That would be controllable by the application, UI or console. 

e.g. in the DLL, have a

proceudure SetDebugCallback(const ACallback:TDebugCallback);

where

   TDebugCallback = procedure (const AMessage: PChar); stdcall;

Note: use PChar and not a string (maybe be even more explicit like PWideChar, PAnsiChar).

That way, in the application, console app could do writeln, where the UI app could show a dialog, add to a list, whatever...

Also, I wouldn't trust sending a Delphi managed type across a DLL boundary. It limits what type of apps can consume and use the function as they would need to emulate the string interface, knowing the internals of the type, where using zero terminated strings is supported by the Delphi compiler.

 I could assume that if the structure in question is more complex and the snippet we see is a sample, the use of withValue() would be part of a bigger set of stuff where the structure is set using method chaining...

e.g. myrec.withValue('abc').withNum(123)...

returning a pointer would be slightly more optimal as the delphi syntax should still allow the chaining with the '.', and the original structure would be referenced rather than copies being made if the result of each withXyz() was an actual record.

if part of the question is about record vs class, well records in Delphi can be heap or stack based, where classes in Delphi are heap based. Assignment of a record in general, even in a method chain would result in copies, vs using a class would simply be passing a pointer around.

As David mentioned however, it is a trivial scenario, just using a constructor would do. Using method chaining is sometimes an opinion piece, commonly used when using builder type patterns.

Using fork() is the Linux way, but you don't have to worry about that in a docker environment. In Docker, you can simply rely on creating a simple console app. You then just rely on Docker Swarm or Kubernetes to be running on the host machine running the container as required.

You simply need to provide a CMD or ENTRYPOINT in the Dockerfile referencing your command line application.

I forgot you mentioned NGINX... So it depends how 'pure' you want to be... the 'pure' way is you have 2 containers: 1) nginx and 2) your app....

However, there is nothing stopping you from creating one container with NGINX and your application in it, but it is a container anti-pattern.   

2 minutes ago, Anders Melander said:

Shouldn't that be "that can be consumed and used in a tool instantly"...?  or do you know of something other than ANTLR that consumes the ANTLR grammar?

My point was less about how many tools can consume it, but about having something that could actually be used at the end of the day. It could be anything, I just happen to like ANTLR more compared to lex/yacc and flex/bison, which I used to use in the old days.  

I'm not sure 100% of our objective - do we just want to have a text document (which is valuable in its own right), or do we want to have a grammar that can be consumed by a tool as well?

I mentioned ANTLR before (https://www.antlr.org/). I'm just wondering if it would not be worth adopting a syntax like that that can be consumed and used in tooling instantly?

e.g. https://github.com/fabriciocolombo/sonar-delphi/blob/master/src/main/antlr3/org/sonar/plugins/delphi/antlr/Delphi.g (a version 3 grammar)

      https://github.com/gotthardsen/Delphi-ANTRL4-Grammar/blob/master/Delphi.g4

Unfortunately, there isn't a 'runtime' for delphi on the latest version which is very mature, but using another language run time, like java, c#, etc, we could easily make a simple parser for delphi, until someone has time to work on one for delphi.

Re BNF vs diagrams…

as the format should be formal, it would allow for diagrams to be created easily using tools like graphgviz/dot and sexier/modern versions thereof.

Re benefits of a grammar

it can be used as a basis for all kinds of things… once a model is created using a parser generator or manually crafted, you can create things that might be out of scope of the actual compiler. E.g. static code analysis, documentation generation, dependency analysis (between units), search tools, creating diagrams, etc.

Writing parsers can take a bit of effort, and to do it correctly, the grammar needs to be right. There is a lot of boiler plate (repeated patterns) when it comes to parsing, so over time people have written parser generation tools to assist with this process more optimally (and avoid bugs). Their input would normally be a bnf type config file. Examples of such tools is yacc (yet another compiler compiler), etc. I personally like the ANTLR framework. Unfortunately, the latest version 4 doesn’t have a Delphi runtime, but version 3 had one. 
 

it would be convenient to update this to fit in with one of those tools as it also makes it easier to instantly validate and create test cases on the grammar.

The issue is that for anything 'virtual' to work, a VMT needs to be available. Virtual methods are accessible on instances of a class through the class VMT. 

The only way to accomplish something like what you wanted would be to introduce some helper variables/procedures in delphi/pascal to accomplish this - something that is overly verbose in pascal compared to python. I've given an example below.

From a python perspective, you would maybe have something like:

class X:
    @staticmethod
    def do_something():
        return '42'

print(X.do_something()) # 42

# illustrating overriding with a lambda
X.do_something = lambda: '43'
print(X.do_something()) # 43

In Delphi, you could do the following:

type
   TMyGetter = function : string;
   TTest = class
    private
       class var FValueGetter: TMyGetter;     
       class function GetValue:string;
    public
       class constructor Create;
       class property ValueGetter: TMyGetter read FValueGetter write FValueGetter;
       class property Value : string read GetValue;
   end;

function MyGetValue:string;
begin
   exit('42');
end;
function MyGetValue2:string;
begin
   exit('43');
end;
class constructor TTest.Create;
begin
    TTest.FValueGetter := MyGetValue;
end;
class function TTest.GetValue:string;
begin
    if assigned(TTest.FValueGetter) then begin
       result := TTest.FValueGetter();
       exit;
     end;
     raise Exception.Create('no getter set');
end;

begin
    writeln(TTest.Value); // 42
    
    TTest.ValueGetter := MyGetValue2;
    writeln(TTest.Value); // 43 
end. 

In the above the the 'ValueGetter' sort of provides the same behaviour of the VMT in allowing it to be overridden. However, this is not a polymorphic property - so inheritance does not work... child classes would essentially modify the same static variable - but this principle applies the same way for my python example above as well I think.

You may know this, or for other readers... As Lars mentioned, 'waiting' correctly is important. Ideally we should not be sleep()ing before querying a queue.

Be mindful not to create 1 thread per request. If there are too many threads, then your scheduler ends up spending more time spinning, which doesn't help.

Ideally, it should be utilising thread synchronisation primitives such as TEvent SetEvent() and WaitFor() when adding requests to the queue and waiting for events in the queue respectively.

Further, we need to ensure we have some locking around whatever structure is used for the queue so that concurrent access doesn't destroy the structure's consistency.

I'll write a little example later.

I agree with Anders.

using threads can be for different reasons though… backgrounding a blocking io bound tasks is different to backgrounding a cpu bound tasks. you may want to consider thread pools, as they ensure threads are recycled efficiently, and don’t normally exceed the number of cores.

On multi core systems, I’ve normally used threadpools usually helps in recycling threads. The Delphi tasks library essentially captures this concept at a high level. 
 

there is also thread affinity that can be assigned using SetThreadAffinityMask, (on windows) but not sure how effective it is. The docs also recommend letting the system do the scheduling.

If you use TTask.Run(), it returns an ITask which resembles the 'future' pattern - the interface provides a getStatus() to check if it is complete or not, as well as other helper methods. 

From OmniThreadLibrary does seem like the Parallel.Async() could be extended to return something similar.

However, using ITask / IFuture requires 'polling' type behavior compared to callbacks ... but I think what Dalija has just suggested is closer to what you probably want.

I've added installers for both Delphi 10.4.2 and Delphi 11.

I've uploaded an update for interested parties to try. http://downloads.sempare.ltd/SempareDebugConfigurationManager/

I wish RTTI embedded the generic type parameters and the TRttiType included a way to extract  them.

I've often had to rely on specific use cases and knowing that I can extract the types easily from known method return types as highlighted by the enumerator / pair example above...

I've had times where I needed some really generic stuff and had to split the TWhatever<T> as a string and resolve the generic parameter however it is defined recursively -  which is a bit annoying.

I'm not sure if think you might be thinking that 'uses' works similarly to the 'import' statement in Java. In Java, you normally import the type from the fully qualified package as you have attempted.

'System.Generics.Collections' is the unit, with many types/functions defined within. if you just needed to use it, you simply do 'uses System.Generics.Collections;'

From the online help, if you see something like 'System.Generic.Collections.TArray' -  it normally means that 'TArray' will be defined in 'System.Generics.Collections', so you 'uses' as illustrated above.

Hi delphi fans,

I'm never sure how much attention is paid to the 'upvotes' on issues in quality portal... 

Anyways, as there has been considerable effort in getting LSP working and improving that by the Emb team, one feature I'd really like to see the IDE provide is a 'rad' experience from the code editor with auto completion of anonymous method signatures, etc.

https://quality.embarcadero.com/browse/RSP-27293 and https://quality.embarcadero.com/browse/RSP-24310

to summarize, as an example:

type
    TMyfunction = reference to procedure (const AArg:string);
begin
    var func : TMyfunction := /* press ctrl+space which completes the following: */ procedure (const AArg:string)
          begin
          end;
end;

There can be many different ways in which the context of the type can be known based on the cursor position when ctrl+space is pressed...

If you have some time and think it would also be valuable, please upvote...

I don't know if the moderators should consider a separate forum section for these types of requests. 

regards,

Conrad

Hi @Edwin Yip

I'm not sure why 

'<% getdata("somevalue", sharedstate) %>';

would be difficult? from a user perspective (someone you may have creating a template), they may not have to know about the parameter types - they just need to know what parameters are available to be passed in...

I do understand however that you may have non technical users from which you may want to hide this detail. At this stage, the only way is to use class variables - which is what would be available to the class functions.

Maybe a potential enhancement would be to allow the template context to be param 1:

     class function dosomething(const ACtx:ITemplateContext;const AFirstArg:string):string;

From a usage perspective, the template engine could expect the following usage:

    <% dosomething('hello') %>

where it auto inserts the context.

The reason I mention the context is because it allows you to store data in there as well which can generally be accessed by the template.

Hi @Edwin Yip

As functions are static, it means that you must pass all required data to them. This will further ensure they are in a way thread safe, unless you are referencing some data that is 'shared' across threads... So normal solutions apply:

1. get a copy before hand of whatever is required 

2. locking while the template is potentially referencing that shared data

doing 2. is probably the most expensive, but if you are referencing shared state, there are some as to the shared state changing or not. if it doesn't, well, then it shouldn't be a problem. This may be the case say with static config loaded at startup that may be referenced during rendering. if the shared state does change - say some service methods mutate the state, then you may need some locking to ensure the underlying structures are not corrupted. 

with the shared state,you  probably have a method that is used generally in anycase with locking for the multi threaded scenario. so exposing that function to the template is possible as well... so you may do the following:

1. pass in a reference to the static data when rendering the template

2. register a method with the template so that it can extract relevant information from the shared state

e.g.

type
   TMyProg = class
   private
      class var FLock:TCriticalSection;
      class constructor Create; // init flock
      class destructor Destroy;  // free flock
   public    
        class function getdata(const AKey:string; const AState:TDictionary<string, string>):string; static;
   end;

  TMyData = record
     SharedState : TDictionary<string,string>;
  end;

// -------------------

class function TMyProg.getdata(const AKey:string; const AState:TDictionary<string, string>):string;
begin
   FLock.Acquire;
   try
       if not AState.TryGetValue(akey, result) then
           exit('-');
   finally
      FLock.Release;
   end;
end;

// -------------------

var GState:=TDictionary<string, string>.Create();

var GTemplate := '<% getdata("somevalue", sharedstate) %>';
var GCtx := template.context; // context could be global as well, so you don't have to keep on creating it
GTtx.AddFunctions(TMyUtilities);

var GTpl := Template.Parse(GCtx, GTemplate); // template could be global as well

// local state
var LData : TMyData;
LData.SharedState := GState;

writeln(Template.Eval(GCtx, GTpl, LData); // this should be localised

I havn't thrown the above into an IDE, but it should work in principle... (code should be placed in relevant procedures and GState should be seen as global, etc...)

I hope the above makes sense...

I did a quick stab at option 2 approach (delphi first)... with this working, could now apply the logic to the template engine (another post may follow):

type
  TMenuPos = (mpFirst, mpLast);
  TMenuPosInfo = set of TMenuPos;

  TMenuItem = class(TMenuItemCollection)
  private
    FName: string;
    FPosition: TMenuPosInfo;

  public
    constructor Create(const AName: string = '');
    property Name: string read FName;
    property Position: TMenuPosInfo read FPosition write FPosition;
  end;

  TMenu = TMenuItem;

function GetHtmlStack(const AMenu: TMenu): string;
var
  lStack: TStack<TMenuItem>;
  lParentStack: TStack<TMenuItem>;
  LItem: TMenuItem;
  LChildItem: TMenuItem;
  LSB: TStringBuilder;
  i: integer;
begin
  if AMenu.Children.Count = 0 then
    exit('');
  lStack := TStack<TMenuItem>.Create;
  lParentStack := TStack<TMenuItem>.Create;

  try
    LSB := TStringBuilder.Create;

    try
      lStack.Push(AMenu);

      while (lStack.Count > 0) do
      begin

        LItem := lStack.Pop;

        if AMenu <> LItem then
        begin
          if mpFirst in LItem.Position then
            LSB.Append('<ol>');

          LSB.Append('<li>');
          LSB.Append(LItem.Name);
        end;

        if LItem.Children.Count > 0 then
        begin
          lParentStack.Push(LItem);
          for i := LItem.Children.Count - 1 downto 0 do
          begin
            lStack.Push(LItem.Children[i]);
          end;
          continue;
        end;

        LSB.Append('</li>');

        if mpLast in LItem.Position then
        begin
          LSB.Append('</ol>').Append('</li>');
          lParentStack.Pop;
        end;
      end;

      while lParentStack.Count > 0 do
      begin
        LItem := lParentStack.Pop;
        lsb.Append('</ol>');
        if lParentStack.Count > 0 then
          LSB.Append('</li>');
      end;

      exit(LSB.ToString);
    finally
      LSB.free;
    end;
  finally
    lParentStack.free;
    lStack.free;
  end;
end;

this seems to produce the same as the functionally reclusive function.

Ah, yes. I see what you mean. 😉