Playing with Windows Fibers by emulating Python Generators

[darnocian 84]

Hi

 

I created a ticket a while back on Quality Portal for some missing Windows SDK functions for Fibers. I created a small test project to illustrate them working in https://github.com/sempare/sempare-delphi-fiber-generator.

 

For those that don't know, fibers essentially are light weight threads, where context switching between fibers is managed by the developer and not by the OS, as is done with threads. It can be complex to get your head around them if you are not familiar with them. For a more detailed discussion, here is a nice article: https://nullprogram.com/blog/2019/03/28/ and https://learn.microsoft.com/en-us/windows/win32/procthread/fibers. I think the reason they were omitted in the Delphi Windows unit is because the tool that did the main conversion of the Windows SDK functions was working with normal C functions, where the missing functions I've included were defined as C macros.

 

From a Delphi perspective, GetFiberData annd GetCurrentFiber are available in the unit https://github.com/sempare/sempare-delphi-fiber-generator/blob/develop/src/Sempare.Win32.Fiber.pas for win32 and win64.

In the project, the unit https://github.com/sempare/sempare-delphi-fiber-generator/blob/develop/src/Sempare.FiberGenerator.pas contains a TGenerator<T> which switches between the procedure producing values and the main method that is consuming them. Here is an example of it in use with a fibonacci generator.

 

Quote

procedure TGeneratorTest.TestFibonacci;
var
  LGenerator: TGenerator<integer>;
  LSeq: TArray<integer>;
  i, j: integer;
begin
  LGenerator.Producer := procedure
    var
      i, j, t: integer;
    begin
      i := 1;
      j := 1;
      TGenerator<integer>.Yield(1);
      TGenerator<integer>.Yield(1);
      while true do
      begin
        t := i + j;
        TGenerator<integer>.Yield(t);
        i := j;
        j := t;
      end;
    end;
  LSeq := [1, 1, 2, 3, 5, 8, 13, 21, 34, 55];
  for i := 0 to 9 do
  begin
    j := LGenerator.Value;
    assert.AreEqual(LSeq, j);
  end;
end;

 

So what is going on here.... 

 

A producer is a procedure that must call Yield(value). So we know that with Fibonacci, f(0) =1, f(1) =1, f(n) = f(n-1) + f(n-2). Rather than having a recursive routine, the producer method simply Yields the values appropriately. Yes, it looks like the method will never stop. That doesn't matter. Once the method has a value to yield, the context is switched back to the main bit of code that is requesting the value ... that where we have:

Quote

j := LGenerator.Value;

 

As mentioned, this is just a simple demo of using fibers. Another project I had that has a bit more complex utilisation of fibers was for async IO on sockets - the principle was to make async calls with call back functions appear synchronous, which is what most people are more comfortable with. Even though most people will never need to use the fiber methods directly as I illustrated above with the TGenerator example, it is quite a powerful tool in the toolbelt.

 

Edited August 12, 2023 by darnocian

[Remy Lebeau 1394]

2 hours ago, darnocian said:

I think the reason they were omitted in the Delphi Windows unit is because the tool that did the main conversion of the Windows SDK functions was working with normal C functions, where the missing functions I've included were defined as C macros.

There are plenty of C macros that are translated into Delphi's Window SDK units. More likely, they were not translated simply because fibers are just not used by most developers, not even Microsoft: Fibers aren’t useful for much any more; there’s just one corner of it that remains useful for a reason unrelated to fibers

Edited August 13, 2023 by Remy Lebeau

[darnocian 84]

@Remy LebeauYes, I've seen Raymond Chen's blog. 

 

However, in many languages, coroutines are popular. Fibers is one way to implement them.

 

Generally, I would say the argument would be that most people would not use them directly, but possibly a wrapper or language feature would utilise fibers to provide some interesting behaviour. I would say, it is pointless including fiber functions in the windows unit without these as they provide context around whatever needs to be done. Agreed - nobody but me has missed probably them as they have been excluded for so long, or nobody bothered raised a QP ticket for the omission. Unfortunately, Delphi is not in the list of languages supporting coroutines either. ;(

Edited August 13, 2023 by darnocian

[David Heffernan 2345]

I thought they'd been removed because they weren't useful. They were added for SQL server and in the end it turned out they weren't useful for that application. 

[Stefan Glienke 2002]

https://bitbucket.org/sglienke/spring4d/branch/feature/coroutines - the reasons it never made it into the main branch are what has been mentioned before and the fact that on POSIX I emulate them using a thread and an event (which is deadly slow) - still until anyone convinces me otherwise (by showing some actual working code!) this is the closest we get to some conveniently usable coroutines in Delphi.

[Brian Evans 105]

I think coroutines are a dead end just like fibers in regards to performance. With pre-emptive multitasking the amount of work done before switching can be adjusted by the OS scheduler for the current hardware and execution environment. A system with a lot of CPUs with deep pipelines will perform better overall switching tasks less often than one with fewer CPUs and very shallow pipelines for example. 

 

With coroutines/fibers the work division is basically fixed in the source code and even if optimal for one hardware and execution environment will likely be suboptimal in others. It is similar to the fixed instruction parallelism attempted with EPIC but in source code form as a language feature. Looks good at first but runs into problems. 

 

 

Edited August 14, 2023 by Brian Evans

[Stefan Glienke 2002]

Coroutines are not a dead end at all - in fact, there are more and more languages making them first-class citizens because you can write asynchronous code just like you would write sequentially.

Google for "project loom" for example.

Edited August 14, 2023 by Stefan Glienke

[Vincent Parrett 750]

Project Loom looks really interesting

https://developer.okta.com/blog/2022/08/26/state-of-java-project-loom

They are calling them Virtual Threads - but the effect is the same - I actually quite like the design they have come up with.

Coroutines/fibers/virtualthreads etc are really useful in high concurrency server applications, or under the hood in other libraries - if you are still dropping db components on forms then they probably would not impact you (directly at least). 
 

[darnocian 84]

For reference, I raised https://quality.embarcadero.com/browse/RSP-40056 in Nov 22 to include the missing fiber functions.

[Fr0sT.Brutal 900]

On 8/15/2023 at 2:38 AM, Vincent Parrett said:

Coroutines/fibers/virtualthreads etc are really useful in high concurrency server applications, or under the hood in other libraries

Regular DB client app but loading a data takes some time... While Delphi coder starts inventing a triangle-wheeled bicycle, anyone using other languages just write "await query.ExecSQL()" and chill.

Or try to implement a protocol with complex req-resp like FTP in async (event-driven) mode. This quickly becomes a hard task.

Edited August 21, 2023 by Fr0sT.Brutal

[darnocian 84]

I do find the mixed response in this thread really interesting. When I raised the RSP, Brian Long, an long time Delphi guy, asked if I had tested the routines and I said I'd provide an example test project. Most missed the intent to facilitate completeness as I did not labour that point too much. I thought providing the missing routines and tests is something that could be useful to to anyone else that was curious/explorers/prototypers, etc. I agree, 99.9% of coders will never need to go near this stuff.

 

Open question - should there be a blanket ban on experiments or using stuff that nobody else does because it is deemed obsolete (g*d, how many times have I heard that about Delphi/Pascal). Just putting it out there.... 

 

Edited August 22, 2023 by darnocian

[pmcgee 10]

Fibers under the magnifying glass
Gor Nishanov (gorn@microsoft.com) - author? of c++ coroutines
(Advocating coroutines over fibers.)
Abstract
"Fibers (sometimes called stackful coroutines or user mode cooperatively scheduled threads) and stackless coroutines (compiler synthesized state machines) represent two distinct programming facilities with vast performance and functionality differences. This paper highlights efficiency, scalability and usability problems of fibers and reaches the conclusion that they are not an appropriate solution for writing scalable concurrent software."
https://open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1364r0.pdf

[Stefan Glienke 2002]

People like Gor would call the majority of the Delphi RTL and alike "not an appropriate solution for writing scalable concurrent software" so we have to be a bit reasonable when putting this statement into context.

[darnocian 84]

I appreciate the conceptual difference between stackfull vs stackless coroutines. Fibers are stackful, where for stackless, we would need the compiler to weave in state management support for the yield/await type behaviour efficiently as done in many other languages that have the feature. Stackful has to save the stack when context switching, where the stackless has minimal state to save. I think without having language features, and trying to emulate the behaviour is probably only achievable with fiber like apis, be it less efficiently.

 

The Gor article does provide good references / motivations as to why use is not encouraged. The scalability, performance and maintainability aspects ultimately dictate how useful something is in the long run. 

 

Needless, I still find it an interesting space and something to play with.

[pyscripter 689]

This project deserves some attention (greenlets-coroutines, generators, channels and more). It is based on fibers.

Edited September 14, 2023 by pyscripter

[Vincent Parrett 750]

I came across this interesting comparison yesterday between Goroutines and C# async/await (part of a group of articles comparing the languages). 

 

https://alexyakunin.medium.com/go-vs-c-part-1-goroutines-vs-async-await-ac909c651c11

 

Well worth a read. 
TLDR - Goroutines are faster, easier to use than async await (async everywhere is a pain), stack sizes matter and Go runs out of memory long before C# - in the benchmarks at least.  

[darnocian 84]

Another related project, although C + Linux based, https://swtch.com/libtask/ has related concepts for event driven apps. Some concepts overlap with Golang.