Calling an "application" from windows service

[Clément 148]

Hi,

 

I wrote a Windows Service (REST HTTP Server) does a lot of stuff 😎. Several users (hopefully) will be connected and each might need to call this "app" from their own thread.

Well, I would like to avoid compiling those modules into the main Service, because they will get changed a lot more often than the service itself.  Some modules are specific to some users.
A few details about this app: No form, no console no user interaction are required. I just need to pass a command line with some parameters ( 4 or 5 actually) and the "app" should run, do the thing,  and return to the service.
My options are:

1) Small executable with the required modules: ... just a log file to guide me (or my user). Will be called from the service with the required parameters (command line)
2) Write a DLL and load and unload it when there's an update (don't like the idea, but it works)

3) Write another Windows Service with the required modules. As both services share the same machine, communication between them should not be a problem.

4) Mix options 3 and 1 to build a powerfull architecture ( not required right now ).

 

Do you guys have another option? A preferred one?

 

TIA,

Clément

 

 

 

[Anders Melander 1782]

From your description this looks like you need some kind of plug-in system.

My personal preferences would be (in this order):

1. In process COM server
2. A run-time package
3. DLL

But if an external EXE can solve the problem for you, without any compromises, then that's clearly the easiest solution.

[Arnaud Bouchez 407]

 

1 hour ago, Clément said:

I wrote a Windows Service (REST HTTP Server) does a lot of stuff 😎. Several users (hopefully) will be connected and each might need to call this "app" from their own thread.

Well, I would like to avoid compiling those modules into the main Service, because they will get changed a lot more often than the service itself.  Some modules are specific to some users.

I am confused by this description. What do you call a "user"? Is it a client app? Then why this client logic should be compiled within the main module?

Please refine your system description.

 

If you meant having your main service call some sub-functions for custom process, then you may rather call separated services.
An embedded dll (or com server, whatever) is the right way to make your system instable. If the dll has some memory leaks or write somewhere in memory, you may corrupt the main process....
So I would isolate the custom logic into one or several dedicated services - so point 3.

 

Perhaps you need to refine the design of your SOA solution. You don't need a monolithic REST server. The best practice today for several services is to create MicroServices.

When a Windows Service "does a lot of stuff", from my point of view it sounds like not a very maintainable design.

The first step would be to split the main service into smaller services, then put an orchestrator/application service as frontend, calling third-party services if necessary.

Perhaps some design part of our framework documentation may help. Check http://mormot.net

Edited August 10, 2020 by Arnaud Bouchez

[Guest]

In those cases I prefer DLL (2)

Advantages

1) You can do the job with command line this means you need one-only-function DLL, but unlike the EXE you can add more features like self test, simulate, check versions....

2) Fewer exe means less exposure to security software running on the OS, this goes to COM too.

3) One EXE means those DLL adds extra valuable protection against been stolen and used alone, of course if that is necessary or critical, they can't run without the main EXE(service) means more you have central protection on the main EXE/service.

4) the flexibility of the DLL (as plugins) will enforce you to keep tracking them and their versions, easily you can leave the older versions available to users, many client appreciate such things for backwards testing, and it just amplify your project in their eyes, they all love that such things.

5) Your service is REST/HTTP means you can update/add DLL's remotely, no worries about registering a COM or installing service permissions.

6) after developing and testing a Module, the easiest way is to convert it to your template DLL/plugins, and you can compile them and leave some of them in the main EXE as defaults.

 

Disadvantages

1) As Arnaud pointed, corrupt memory in DLL may spill over the Service, but this may be manageable with the same logic as separated EXE with DLL have own memory manager.

2) My recommendation is to have check overflow enabled on those DLL's, and this will affect the performance in variable degree, if the performance is an issue, then isolate the affected part of the code and disable those checks like compressing data, here you don't need the checks, or parsing JSON with well tested and mature library...

 

Now if you only will need 2-3 (less than 5) of those modules then may be Arnaud is right here and different services may be the best approach, but the more you will add the clearer you will see how hard (not easy) managing services on the recent Windows OS, specially if the system put them to sleep.

 

In all cases, Log then log after that is done, try to log more.

[FPiette 382]

2 hours ago, Clément said:

I wrote a Windows Service (REST HTTP Server) does a lot of stuff 😎. Several users (hopefully) will be connected and each might need to call this "app" from their own thread. 

This is not very clear to me. Are the users HTTP clients connected to the HTTP server? Are those client local or remote?

 

Your "app" look like a standard console mode executable. Your service can call it calling CreateProcess but be aware that a service run with a special account, even when no user is logged in Windows and certainly not the user that has logged in Windows. The account for the service has different permissions from the logged in user. You must grant all required permissions and it is not easy to "impersonate" the user that a HTTP client is representing.

 

[Clément 148]

2 minutes ago, Arnaud Bouchez said:

 

I am confused by this description. What do you call a "user"? Is it a client app? Then why this client logic should be compiled within the main module?

Please refine your system description.

 

Perhaps you need to refine the design of your SOA solution. You don't need a monolithic REST server. The best practice today for several services is to create MicroServices.

When a Windows Service "does a lot of stuff", from my point of view it sounds like not a very maintainable design.

The first step would be to split the main service into smaller services, then put an orchestrator/application service as frontend, calling third-party services if necessary.

Hi Arnaud.
The HTTP server is compiled with some functionalities that are common for the customers that will access that server. Customers can be are either small companies or individuals.  Each company can have many employee and each will have to a "user" to log in the client app and make requests.
The "common" functionalities can group Companies in the same segment for example. Those companies will access the same HTTP Server compiled with modules, so if you wish, monolithic.
What I need is to address specific needs from some customers. And for those I wish to execute an "app". And this app will be called from this HTTP server.
 

 

[Guest]

I did something similar by "simply" routing some request to a different port. The lib i use has some nice stuff for routing/balance loading efficiently but perhaps just calling a local server from your process using indy would suffice.

 

So you recompile your (other) processe(s) that listen to 90, 91, 92... and just switch them. You can do it while in production if you design REST-ful.

 

Like your serevr are calling a service out there and blocking while until it responds.

 

HTH

 

Edited August 10, 2020 by Guest

[Anders Melander 1782]

If one didn't know any better one might get the impression that DLLs and COM are inherently insecure, unstable and "dangerous".

Yet all of Windows and every single application running on it are based on these technologies...

 

Anyway. One alternative I forgot to mention is to implement custom functionality using a scripting system. I've successfully applied this approach in one of the applications I've worked on to replace a DLL-based plugin system.

[Guest]

Off topic.

 

@Arnaud Bouchez 15 years ago i was looking for a library to use for very simple HTTP stuff and may be will need it to share DB over Internet, so i landed on RTC website and fall in love directly with its multithreaded design and its functionality, now i opened mormot.net then followed the documentation and it is intimidating by the large number of of files, this means to grasp its functionality i will need hours or even days to browse the source, on other hand the documentation is really impressive and covering everything, yet it need hours to read, and specially need long time to connect two aspect in one page, and please don't take this the wrong way, it is fantastic but not for the new comers !, unlike RTC where documentation is very small, mORMot have it all, but RTC does have samples cover almost most of its functionality and flexibility, while with mORMot, one will not know how and where to start unless waste hours between that page and the source.

Thinking now about the reason i went with RTC back then, it was few pages titled build your own client/server DB in 5 minutes.

 

So i would suggest another page above documentation and installation and below the main page on GitHub, to show samples, the shortest samples for specific functionality starting with HTTP GET ( secure and unsecure), to the most advanced topic like microservices and IPC.

The samples should be working and no more than copy and paste, this will help a lot in showing the real power using mORMot library.

[Fr0sT.Brutal 900]

Calling apps from server is OK. This is how CGI works.

[Clément 148]

1 hour ago, FPiette said:

This is not very clear to me. Are the users HTTP clients connected to the HTTP server? Are those client local or remote?

 

Your "app" look like a standard console mode executable. Your service can call it calling CreateProcess but be aware that a service run with a special account, even when no user is logged in Windows and certainly not the user that has logged in Windows. The account for the service has different permissions from the logged in user. You must grant all required permissions and it is not easy to "impersonate" the user that a HTTP client is representing.

 

The users are remote. Most request (99.5% 🙄 )  will be handled by the HTTP Server and it's modules. But there's always that customer that requires some special attention. In this case I must complement the request with a call to a "microservice"
Everyone will request the same action, but, for this customer, I must execute an extra call. This extra call is very customer specific, so I don't want to include it in the main service, I want to execute it either by simple EXE, or by DLL, or by another service....

But from what I saw, I'll have to write another server to call those microservices. That arquitecture gave me some ideas

 

I will use CreateProcess. Since this process will not required any interaction, I see no reason it can't run in the same service session.

 

[FPiette 382]

47 minutes ago, Clément said:

Everyone will request the same action, but, for this customer, I must execute an extra call. This extra call is very customer specific, so I don't want to include it in the main service, I want to execute it either by simple EXE, or by DLL, or by another service....

Be aware that running an executable is a slow process. If you put the same code in a DLL, this can stay loaded as long as you decide to. You may unload the DLL for example when the last request occured 5 minutes ago.

 

If you prefer the EXE, then you may manage to let it run as long as you need, avoiding a new start for each request. The only difference is that you'll pass the arguments not thru the command line but thru some IPC (named pipe is quite fast). And why not a secondary HTTP server?

 

[Clément 148]

2 hours ago, FPiette said:

Be aware that running an executable is a slow process. If you put the same code in a DLL, this can stay loaded as long as you decide to. You may unload the DLL for example when the last request occured 5 minutes ago.

 

If you prefer the EXE, then you may manage to let it run as long as you need, avoiding a new start for each request. The only difference is that you'll pass the arguments not thru the command line but thru some IPC (named pipe is quite fast). And why not a secondary HTTP server?

 

Well, my last options option is Windows Service with microservice, but it is so close to another HTTP server that it might be better to just get it done.

[Arnaud Bouchez 407]

22 hours ago, Kas Ob. said:

So i would suggest another page above documentation and installation and below the main page on GitHub, to show samples, the shortest samples for specific functionality starting with HTTP GET ( secure and unsecure), to the most advanced topic like microservices and IPC.

Fair remark.

This was the point of the FAQ:

https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITL_123

 

Also check https://tamingthemormot.wordpress.com/ blog entries.

[Guest]

On 8/10/2020 at 6:51 PM, Kas Ob. said:

so i landed on RTC website and fall in love directly

Me too! RTC rocks.

[c0d3r 17]

The top 2 must have 3rd party components in my list:  kbmMW and RTC

Edited August 11, 2020 by c0d3r

[Guest]

@c0d3r, i use kbmMemTable, RTC and other some of mORMot.

Isn't kbmMW goal the same as mORMot with ORM?

Or to put my question another way; if you have kbmMW, why do you need RTC. Or... is kbmWM written is such way that you can utilise RTC for it's socket-layer?

[c0d3r 17]

17 hours ago, Dany Marmur said:

@c0d3r, i use kbmMemTable, RTC and other some of mORMot.

Isn't kbmMW goal the same as mORMot with ORM?

Or to put my question another way; if you have kbmMW, why do you need RTC. Or... is kbmWM written is such way that you can utilise RTC for it's socket-layer?

We bought RTC first, to serve all the HTTP/HTTPS API calls,  then we bought kbmMW for Desktops.  We built a window sevice which includes micro services.  I knew kbmMW can do all these HTTP/HTTPS stuff, but we don't want to rewrite all the codes.

[Arnaud Bouchez 407]

RTC is a great set of communication classes.

kbmMW and mORMot are not only communication layers, but full toolboxes, with plenty of other features. They have very diverse philosophy/design.

You may also take a look at the TMS components.

 

One big difference is that mORMot is fully Open Source, is used in several critical projects so is highly maintained, and works fine with FPC on a variety of platforms - we use it on production Linux x86_64 servers to handle TB of data from thousands of clients.
The fact that mORMot SOA can use interfaces to define the services, and WebSockets for real-time callbacks from server to client, make it unique.
There is a full refactoring called mORMot2 which should be finished next october (I hope).

[Kim Madsen 4]

First thank you for mentioning kbmMW.
Just FTR, kbmMW is also highly maintained and used in many critical projects, including the ones both bringing lives and protecting them, which goes thru serious scrutiny.

[Anders Melander 1782]

Welcome @Kim Madsen

[Kim Madsen 4]

Anders Melander
Than you :)