Cross-platform messaging system

[vfbb 285]

Making it available to everyone who want a thread safe, asynchronous, cross-platform and simplistic messaging system for communication between classes / layers in delphi applications.

 

https://github.com/viniciusfbb/ipub-messaging

 

[Andrea Raimondi 13]

Hi!

 

Personally, I like System.Messaging because it allows you to do things like this:

 

unit Unit3;

interface

uses System.Messaging;

Type

  TNofification = class( TMessage )
  public
    class procedure Notify;
  end;

  TNotification<T> = class( TMessage<T> )
  public
    class procedure Notify(Value: T);
  end;

var MsgMgr : TMessageManager;

implementation

{ TNofification }

class procedure TNofification.Notify;
begin
  MsgMgr.SendMessage( nil, Self.Create );
end;

{ TNotification<T> }

class procedure TNotification<T>.Notify(Value: T);
begin
  MsgMgr.SendMessage( nil, Self.Create( Value ) );
end;

initialization
begin
  MsgMgr := TMessageManager.DefaultManager;
end;

end.

Now what you can do here is subclass the right notification and simply use it: 

 

unit Unit3;

interface

uses System.Messaging;

Type

  TNofification = class( TMessage )
  public
    class procedure Notify;
  end;

  TNotification<T> = class( TMessage<T> )
  public
    class procedure Notify(Value: T);
  end;

  TNotifyLoggedUsers = class( TNotification<integer> )
  public
    class procedure NotifyNumberOfLoggedUsers( Value : Integer );
  end;

var MsgMgr : TMessageManager;

implementation

{ TNofification }

class procedure TNofification.Notify;
begin
  MsgMgr.SendMessage( nil, Self.Create );
end;

{ TNotification<T> }

class procedure TNotification<T>.Notify(Value: T);
begin
  MsgMgr.SendMessage( nil, Self.Create( Value ) );
end;

{ TNotifyLoggedUsers }

class procedure TNotifyLoggedUsers.NotifyNumberOfLoggedUsers(Value: Integer);
begin
  Notify( Value );
end;

initialization
begin
  MsgMgr := TMessageManager.DefaultManager;
end;

end.

Now what you can do is to call NotifyLoggedUsers directly in a way such as:

begin
  TNotifyLoggedUsers.NotifyNumberOfLoggedUsers(Users.Count);
end;

This is *very* interesting, powerful and readable. 

The other big plus is that being synchronous it allows you to reason easily about message flows and you can build up programming documentation using diagrams that can be used and understood by sales and other parties as well. 

[vfbb 285]

@Andrea Raimondi

The full explanation is on the github page. See this part:

 

"The problem

Delphi has its own messaging system (System.Messaging.pas) that works well but is totally synchronous and thread unsafe. In multithreaded systems we always need to communicate with other classes, sometimes synchronously, sometimes asynchronously, sometimes synchronizing with the mainthread (in the case of the UI), and doing this without a message system (communicating directly) makes the code large and complex, prone to many bugs."

 

Edited January 15, 2021 by vfbb

[Lars Fosdal 1793]

I use OmniThreadLibrary queues for this.

[Anders Melander 1815]

53 minutes ago, Lars Fosdal said:

I use OmniThreadLibrary queues for this.

How is that relevant to the topic?

[vfbb 285]

51 minutes ago, Lars Fosdal said:

I use OmniThreadLibrary queues for this.

This is just a simplistic messaging system. Don't compare a simple, lightweight 1k line system with a complete 50k line system (although messages work just as well).

[Attila Kovacs 631]

@Andrea Raimondi

Quote

This is *very* interesting, powerful and readable. 

At first sight this hocus-pocus has not much to do with System.Messaging at all and could be be easily create the same with this lib.

 

By the way, what do you mean with

Quote

"allows you to reason easily about message flows and you can build up programming documentation using diagrams that can be used and understood by sales and other parties as well"

Are you writing message driven apps?

 

By the way 2

initialization
begin
  MsgMgr := TMessageManager.DefaultManager;
end;

What is your purpose with begin/end? Is this also some readability thing?

 

[Bob Devine 12]

@vfbb Thanks for this looks interesting. Would be worth putting a link to DEB on the readme https://github.com/spinettaro/delphi-event-bus

 

There's another one here: https://github.com/THSoftPoland/DelphiMessageBus

 

I've been mulling over using an event bus rather than the observer pattern in some of my code, but I need something that works with pas2js (in TMS Webcore). The latest pas2js has generics etc. so I'm aiming to try some testing next month - might be worth you taking a look though.

[vfbb 285]

29 minutes ago, Bob Devine said:

@vfbb Thanks for this looks interesting. Would be worth putting a link to DEB on the readme https://github.com/spinettaro/delphi-event-bus

Made.

33 minutes ago, Bob Devine said:

There's another one here: https://github.com/THSoftPoland/DelphiMessageBus

It's not worth, it's a very limited system, synchronous, windows support only ...

[Andrea Raimondi 13]

11 hours ago, Attila Kovacs said:

@Andrea Raimondi

At first sight this hocus-pocus has not much to do with System.Messaging at all and could be be easily create the same with this lib.

 

By the way, what do you mean with

Are you writing message driven apps?

 

By the way 2

initialization
begin
  MsgMgr := TMessageManager.DefaultManager;
end;

What is your purpose with begin/end? Is this also some readability thing?

 

Hi,

 

Yes it may be redone, probably, but it would go against the "current" of what that library is trying to accomplish and it would be unnaturally convoluted to do it. 

And yes, whenever I can, I write message driven apps. I find reasoning about message flows much easier than using events and such. I can also share code much more easily and because one of my rules is that "Messages often are features" that allows other people to reason about the flow and the behaviour in much easier to understand ways. The final advantage is that such architecture fosters statelessness which means you have much broader reach and need to pay attention to what you do 😄 

 

The purpose there is not so much readability (which is part of it) but also consistency. I find lack of begin/end pairs in initialization/finalization very bad because you don't get the lines and you can't easily collapse them (please do give it a try: both work when you use begin/end). Moving things is also a lot easier for you could - for example - create a new InitYaddaYaddaYadda and just cut and paste the whole block. Finally, keep in mind that this is an example and chances are thart in my actual prologue/epilogue there mayt be a lot more stuff. 

[Andrea Raimondi 13]

The other big advantage of this approach is maintenance: by doing this you only need to search for the message class and you will not miss any code and it's easier to pick up for newcomers to the project.

 

This approach has all kinds of advantages that become clear as soon as you start using it. 

 

You can also split the code initiating the message from that which receives it so you can have different people working on the same feature from two different angles and if a message cuts across features everyone can just add a handler and be done with it. What happens to the handlers becomes a problem you don't have to deal with. 

Edited January 16, 2021 by Andrea Raimondi

[Andrea Raimondi 13]

16 hours ago, vfbb said:

@Andrea Raimondi

The full explanation is on the github page. See this part:

 

"The problem

Delphi has its own messaging system (System.Messaging.pas) that works well but is totally synchronous and thread unsafe. In multithreaded systems we always need to communicate with other classes, sometimes synchronously, sometimes asynchronously, sometimes synchronizing with the mainthread (in the case of the UI), and doing this without a message system (communicating directly) makes the code large and complex, prone to many bugs."

 

I read that and I still disagree 😄 

 

I like System.Messaging because it's synchronous and you need to plan the message flow. I am not saying that yours is a bad solution, all I am saying is that I like synchronous solutions such as Systerm.Messaging better on philosophical  grounds. Async stuff is much harder to debug and will often not return enough on your investment to make up for the added difficulty in debugging and reasoning about things. 

 

I want to be able to follow things along so that I know that from Step A we go into Step B and not that Step XYZ gets in the middle without any apparent reason 😄 

[Dalija Prasnikar 1404]

4 hours ago, Andrea Raimondi said:

I read that and I still disagree 😄 

 

I like System.Messaging because it's synchronous and you need to plan the message flow. I am not saying that yours is a bad solution, all I am saying is that I like synchronous solutions such as Systerm.Messaging better on philosophical  grounds. Async stuff is much harder to debug and will often not return enough on your investment to make up for the added difficulty in debugging and reasoning about things. 

 

I want to be able to follow things along so that I know that from Step A we go into Step B and not that Step XYZ gets in the middle without any apparent reason 😄 

That is fine. 

 

Point is that System.Messaging is not thread safe. If you are writing multi-threaded application and you need to communicate using some messaging system, then System.Messaging will simply not work in such conditions. You can still use it to send messages to the main thread, but you need to synchronize messaging code, which is not always viable solution.

 

In multithreaded applications, you will need thread safe messaging system that is also capable of posting messages to some message queue.

Edited January 16, 2021 by Dalija Prasnikar

[Attila Kovacs 631]

@Andrea Raimondi Interesting. One would think it would be a debugging horror and one could easily break a lot of things without compiler errors, but if you and your team are successful with that, then it must be ok.

 

Anyway, I was really happy when I saw this post. Thread-safe and async, I'll add an UDP gateway to it and will try to set up some communication between client apps / services and between brokers on the LAN. Sounds fun!

[Lars Fosdal 1793]

On 1/15/2021 at 3:12 PM, vfbb said:

This is just a simplistic messaging system. Don't compare a simple, lightweight 1k line system with a complete 50k line system (although messages work just as well).

I like the KISS rule - what can I say.

[Dalija Prasnikar 1404]

1 hour ago, Lars Fosdal said:

I like the KISS rule - what can I say.

Me, too. 

 

But complexities often come with additional features. And often simple systems do not have all the needed features, so when you need them, you can either write your own thingy that only has features you need, or you can use existing more complex one that has features you need plus some more.

[Andrea Raimondi 13]

On 1/16/2021 at 9:27 AM, Dalija Prasnikar said:

That is fine. 

 

Point is that System.Messaging is not thread safe. If you are writing multi-threaded application and you need to communicate using some messaging system, then System.Messaging will simply not work in such conditions. You can still use it to send messages to the main thread, but you need to synchronize messaging code, which is not always viable solution.

 

In multithreaded applications, you will need thread safe messaging system that is also capable of posting messages to some message queue.

Hi,

 

Multithreaded applications - in my view - should be made with appropriate libraries such as OmniThread, because you reap all the benefits of multithreading while keeping (most of) your sanity 😄 

The problem remains that if you send async messages in a multi-threaded application you very easily end up not knowing how that message came about in the first place because, you know, thread debugging sucks and not just in Delphi.

Given that a message may be coming from a different threading context, things get wild very quickly. 

 

[Rollo62 539]

I like to speak a word for System.Messaging, which I extended to some kind of thread safety too.

This seems a good base for me, and works very stable in main UI and threads also, if you took the right measures to secure it.

To use System.Messaging  as a base makes sense to me, because its a System unit and also the FMX messaging system is build on top of this,

thats why I assume is has to be well tested and maintained.

 

Nonetheless I like to see new libraries, which can do it much better too.

 

Edited January 22, 2021 by Rollo62

[Dalija Prasnikar 1404]

2 hours ago, Andrea Raimondi said:

Multithreaded applications - in my view - should be made with appropriate libraries such as OmniThread, because you reap all the benefits of multithreading while keeping (most of) your sanity 😄 

The problem remains that if you send async messages in a multi-threaded application you very easily end up not knowing how that message came about in the first place because, you know, thread debugging sucks and not just in Delphi.

Given that a message may be coming from a different threading context, things get wild very quickly. 

Yes, it can be harder to track from where message originated, but synchronous executing will make your thread wait twiddling thumbs while code runs in the context of the main thread. That is why posting messages is preferred. Of course, everything depends on the particular application and code and whether you can afford performance drop or not.

[Vincent Parrett 763]

3 hours ago, Dalija Prasnikar said:

Yes, it can be harder to track from where message originated, but synchronous executing will make your thread wait twiddling thumbs while code runs in the context of the main thread. That is why posting messages is preferred. Of course, everything depends on the particular application and code and whether you can afford performance drop or not.

I would suggest that if you want everything synchronous then don't use messaging, there's no point and you are just adding overhead for very little gain.

 

We use asynchronous messaging extensively in FinalBuilder, yes it takes a bit of thought but then that's what programming is about. We use our own messaging library which is thread safe. Under the hood we also use omnithread library for some of the more complex threading. 

[Dalija Prasnikar 1404]

20 minutes ago, Vincent Parrett said:

I would suggest that if you want everything synchronous then don't use messaging, there's no point and you are just adding overhead for very little gain.

Gain with any messaging system is decoupling, so it makes sense to use it even if you make synchronous calls.

 

But in multithreading scenarios, you need full fledged thread safe system capable of posting messages to a queue. In other words, asynchronous messaging. Anything else is just a crutch that can be used only in most simple scenarios.