borni69
-
Content Count
54 -
Joined
-
Last visited
Posts posted by borni69
-
-
Not sure if you figured it out, but the code below is working for me.
And for me I see a diff from your code in
RESTClient1.HandleRedirects:=true;
RESTRequest1.Method :=TRESTRequestMethod.rmPOST;endpoint := 'https://something/'; content_type := 'application/json'; // ++ some toher params RESTClient1 := TRESTClient.Create(nil); RESTRequest1 := TRESTRequest.Create(nil); RESTResponse1 := TRESTResponse.Create(nil); try RESTRequest1.Client := RESTClient1; RESTRequest1.Response := RESTResponse1; RESTClient1.BaseURL:=endpoint; RESTClient1.HandleRedirects:=true; RESTRequest1.Method :=TRESTRequestMethod.rmPOST; RestRequest1.Params.Clear; RestRequest1.Params.AddItem('content-type', content_type ,pkHTTPHEADER,[poDoNotEncode] ); RestRequest1.Params.AddItem('x-amz-Date', amzdate ,pkHTTPHEADER ); RestRequest1.Params.AddItem('Authorization',authorization_header ,pkHTTPHEADER ,[poDoNotEncode]); RestRequest1.Params.AddItem('x-amz-content-sha256', payload_hash ,pkHTTPHEADER ,[poDoNotEncode] ); RestRequest1.AddBody(request_parameters , TRESTContentType.ctAPPLICATION_JSON ); try RESTRequest1.Execute; writeln('respons'); writeln( RESTResponse1.Content); except On E: Exception do writeln( e.Message); end; finally RESTClient1.Free; RESTRequest1.Free; RESTResponse1.Free; end;
But I also set the body Json using system.json unit
Jrequest_parameters := TjsonObject.Create; try Jrequest_parameters.AddPair(TJSONString.Create('fromhost'),TJSONString.Create('10.211.55.2')); Jrequest_parameters.AddPair(TJSONString.Create('fromport'),TJSONString.Create('3306')); Jrequest_parameters.AddPair(TJSONString.Create('fromschema'),TJSONString.Create('atest')); Jrequest_parameters.AddPair(TJSONString.Create('frompass'),TJSONString.Create('xxxxx')); Jrequest_parameters.AddPair(TJSONString.Create('tohost'),TJSONString.Create('10.211.55.2')); Jrequest_parameters.AddPair(TJSONString.Create('toport'),TJSONString.Create('3306')); Jrequest_parameters.AddPair(TJSONString.Create('toschema'),TJSONString.Create('atest2')); Jrequest_parameters.AddPair(TJSONString.Create('topass'),TJSONString.Create('xxxxx')); Jrequest_parameters.AddPair(TJSONString.Create('table'),TJSONString.Create('hseq_crm')); Jrequest_parameters.AddPair(TJSONString.Create('copylogs'),TJSONString.Create('0')); request_parameters := Jrequest_parameters.ToString; finally Jrequest_parameters.Free; end;
-
Not sure I understand you, but I make a lot of curl command from my web app running on linux..
Example converting heic to jpg I do this with curl command from Delphi on Linux ubuntu..
process
1) image uploadet from web in .heic format
2) save to disc
3) curl convert heic to jpg using converter installed on linux
4) load .jpg from disc
5) delete on disc
6) upload to s3
If you need something like this I can give you an example.. but maybe I am misunderstanding what you are looking for...
-
On 3/8/2023 at 2:06 PM, tgbs said:Sorry, but could you give an example of how you organize a loop in a linux console program and an Indy httpserver. Is there an option without a sleep() or while?
I need this program run forever or until I stop via route to webmodules action. ThanksNot sure I understand you but in Delphi you can.
1) File - New - other - Web - Web Server Application -
-Wizzard
2 ) Then select Windows and Linux
3) Stand alone Console app
4) port 8080
5) Complete
Add Linux and compile.. then it should work.
Maybe I misunderstood your question..
B
-
2 hours ago, Vincent Gsell said:Hi !
Idem !
We have Indy based tenant : Prod logs 60 connections "average" (so, 60 connections threads), with no problem at all.
As a detail, we have Nginx on front, fine configured, with geoip and ip2fail and few other tools. (Delphi servers have not to deal with server side ssh currently)
Architecture detail : Server only manage connections and context : it communicates throught our bus with "services" (which is standalone delphi linux console app) which perform workers duty. The Servers do not perfoms business stuffs directly.
Thanks for this update 🙂 We will test it ...
-
30 minutes ago, esegece said:Hello,
I don't use ISAPI, but I've been running Indy servers 24/7 for several years now without any problems, so yes, Indy's fine for production. Just keep in mind that each connection runs in it's own thread, so if there are 80 concurrent connections, the server will be using 80 threads.
Kind Regards,
Sergio
Thanks, we also use 80 threads today so this is the same 🙂.
-
Hi today we run our backend API system on Linux Ubuntu using ISAPI mod files on Linux With apache in front.
We run this system in docker containers.
We have autoscale when the system reaches 80 simultane requests, it increases a new docker instance.
Normal traffic 1 to 2 instances, peak 3.
This means the Delphi app will never need to handle more than 80 requests at the same time.
We are considering to do this with a new setup using an Nginx proxy in docker, and then use standalone console APP to handle the requests.
Is this a good idea ?
Is a standalone console APP “Indy http” ment for production ? or only test.
Hope to get some advice
Thanks
-
Just an idea.
you could upload all images to AWS s3, then use the event on s3 to trigger a lambda function running a delphi linux console app in docker.
This example is not using Delphi, but still explain the consept using s3 as a pool.
https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
We have several Delphi apps running smootly on Lambda..
We use this Lambda runtime API for our Delphi apps
https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html
-
2 hours ago, Daniele Teti said:Can you fill an issue about this? Also did you tried the setcustomlogger? Are you sure the problem is the instant where the logger is created?
In the create
FMVC := TMVCEngine.Create
I send in a custom logger like below GetLogger
end,GetLogger); //GetLogger // Added the logger at this stage
unit CustomLoggerConfigU; interface uses LoggerPro; // loggerpro core function GetLogger: ILogWriter; implementation uses System.IOUtils , LoggerPro.FileAppender ; function GetLogger: ILogWriter; begin Result := BuildLogWriter([ {$IFDEF MSWINDOWS} TLoggerProFileAppender.Create ( 10, 1000 , 'C:\hseqsetting\logs' ) {$ENDIF} {$IFDEF LINUX} TLoggerProFileAppender.Create ( 10, 1000 , '/etc/hseq/logs' ) {$ENDIF} ], nil,TLogType.Info); end; end.
is this above what you mean with setcustomlogger ?
This is an ISAPI app .so for Linux Apache.
It was not able to create the file in the location I set in GetLogger
But if I added a file manually and set chmod 0777 libmod_hseq2.00.dmvcframework.log
to the file, it worked.
But after the file logging worked, I was not able to use /dev/stdout
My plan was to send the log to AWS cloud watch together with the apache log.
So I have for now instead created a custom logger for MYSQL.
I can of course be my little knowledge of the framework, and this is not an issue,
but I was not able to find any examples for ISAPI Linux using log, and based on my reading I could not make it work.
Do you still think I should make it an issue?
Also the linux docker is Ubuntu 18.04
-
Also if someone needs it later I added the logger in the create part of TMVCEngine
procedure TMyWebModule.WebModuleCreate(Sender: TObject); begin FMVC := TMVCEngine.Create(Self, procedure(Config: TMVCConfig) begin // session timeout (0 means session cookie) Config[TMVCConfigKey.SessionTimeout] := '0'; // default content-type Config[TMVCConfigKey.DefaultContentType] := TMVCConstants.DEFAULT_CONTENT_TYPE; // default content charset Config[TMVCConfigKey.DefaultContentCharset] := TMVCConstants.DEFAULT_CONTENT_CHARSET; // unhandled actions are permitted? Config[TMVCConfigKey.AllowUnhandledAction] := 'false'; // default view file extension Config[TMVCConfigKey.DefaultViewFileExtension] := 'html'; // view path Config[TMVCConfigKey.ViewPath] := 'templates'; // Enable Server Signature in response Config[TMVCConfigKey.ExposeServerSignature] := 'true'; end,GetLogger); //GetLogger // Added the logger at this stage FMVC.AddController(TMyController);
- 1
-
I will also have a look at Restappender, maybe it is a better solution for me...
-
The problem was access
in my docker file I added the file libmod_hseq2.00.dmvcframework.log and
chmod 0777 libmod_hseq2.00.dmvcframework.log
then it works
so now it seems to log fine to this file
/var/log/hseq/libmod_hseq2.00.dmvcframework.log
But I got a new problem I would like to send this file to "stdout"
Tried using this command
RUN ln -sf /dev/stdout /var/log/hseq/libmod_hseq2.00.dmvcframework.log
It gives me no error but no logging is showing using docker logs <dockername>
I did find something about it maybe not working because it's not in PID1?
Any idea
-
Hi ,
I am trying to make a custom logg work in Linux , but it make the ISAPI stop running
It works great in windows.
I have this CustomLoggerConfigU.pas
unit CustomLoggerConfigU; interface uses LoggerPro; // loggerpro core function GetLogger: ILogWriter; implementation uses System.IOUtils , LoggerPro.FileAppender // loggerpro file appender (logs to file) {$IFDEF MSWINDOWS} , LoggerPro.OutputdebugStringAppender {$ENDIF} // loggerpro outputdebugstring appender (logs to the debugger) ; function GetLogger: ILogWriter; begin Result := BuildLogWriter([ {$IFDEF MSWINDOWS} TLoggerProFileAppender.Create(10, 1000, 'C:\hseqsetting\logs') {$ENDIF} {$IFDEF LINUX} TLoggerProFileAppender.Create(10, 1000, '/etc/hseq/logs') {$ENDIF} ], nil,TLogType.info); end; end.
and in my webmodule i have added MVCFramework.Logger to uses
and use the following code to trigger SetDefaultLogger
........ procedure TMyWebModule.WebModuleDestroy(Sender: TObject); begin FMVC.Free; end; initialization DataModuleDict:= TDataModuleDict.Create(nil); DataModuleDict.initDict; SetDefaultLogger(GetLogger); finalization DataModuleDict.Free; end.
System run ok if I remove SetDefaultLogger(GetLogger); , but if it is present i get this line in terminal, but after it hang
root@5e5f9a598ac5:/etc/hseq/logs# cat libmod_hseq2.00.dmvcframework.log 2022-03-22 18:57:54:100 [TID 274930554752][INFO ] Custom Logger initialized [dmvcframework] root@5e5f9a598ac5:/etc/hseq/logs#
So its initialized but then hole system hang... not able to run before i comment out line below
//SetDefaultLogger(GetLogger);
Is this not the correct way to do this on Linux ?
thanks in advance
-
On 2/21/2022 at 7:50 PM, João Antônio Duarte said:By default DMVC registers a serializer only for the application/json content-type.
But you can define a serializer for other types. In your WebModule, after creating the TMVCEngine add the following code:
FEngine.Serializers.Add('application/sim+json', TMVCJSONDataObjectsSerializer.Create);
Great Thanks 🙂
-
I have this code
procedure THseqScimController.createUser; var aUser : TUser; begin aUser := Context.Request.BodyAs<TUser>; try if UserScimHandleClass.CreateUser(aUser) then begin Render(aUser,false); Context.Response.StatusCode:=201; end else raise EMVCException.Create(HTTP_STATUS.Conflict,'Conflict'); finally aUser.Free; end; End;
It works great from postman with this content type
application/json
But when request come from Microsoft Azure it have this content type
application/sim+json
And it gives me an error on
aUser := Context.Request.BodyAs<TUser>;
Body content type not supported…
Is there a way to change this application/sim+json to application/json or to add application/sim+json as a supported type ?
I guess other option is to parse json and build the object myself
Thanks
-
Hi,
I am trying to make a SAML request. Anyone now how to deflate a string in Delphi ?
AuthNRequest := '<samlp:AuthnRequest xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"'+ 'xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" ID="ONELOGIN_809707f0030a5d00620c9d9df97f627afe9dcc24" Version="2.0" '+ 'ProviderName="SP test" IssueInstant="2014-07-16T23:52:45Z" Destination="http://idp.example.com/SSOService.php" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"'+ ' AssertionConsumerServiceURL="http://sp.example.com/demo1/index.php?acs">'+ '<saml:Issuer>http://sp.example.com/demo1/metadata.php</saml:Issuer>'+ '<samlp:NameIDPolicy Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress" AllowCreate="true"/>'+ '<samlp:RequestedAuthnContext Comparison="exact">'+ '<saml:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport</saml:AuthnContextClassRef>'+ '</samlp:RequestedAuthnContext>'+ '</samlp:AuthnRequest>';
https://en.wikipedia.org/wiki/Deflate
I am using Delphi 11 Enterprise
B
-
21 hours ago, borni69 said:OKI thanks alot 🙂
This is my result from test.. and it seems to work..
unit HseqDmDictU; interface uses System.SysUtils, System.Classes, Generics.collections; type TDataModuleDict = class(TDataModule) private FString_Dict: TDictionary<string, String>; Fstring_Dict_Sync: TMultiReadExclusiveWriteSynchronizer; procedure SetString_Dict(const Value: TDictionary<string, String>); procedure Setstring_Dict_Sync( const Value: TMultiReadExclusiveWriteSynchronizer); { Private declarations } public { Public declarations } destructor Destroy; override; procedure initDict; property String_Dict : TDictionary<string,String > read FString_Dict write SetString_Dict; property string_Dict_Sync : TMultiReadExclusiveWriteSynchronizer read Fstring_Dict_Sync write Setstring_Dict_Sync; end; var DataModuleDict: TDataModuleDict; implementation {%CLASSGROUP 'System.Classes.TPersistent'} {$R *.dfm} { TDataModuleDict } procedure TDataModuleDict.initDict; begin FString_Dict := TDictionary<string,String >.create ; Fstring_Dict_Sync := TMultiReadExclusiveWriteSynchronizer.Create; end; destructor TDataModuleDict.Destroy; begin FString_Dict.Free; Fstring_Dict_Sync.Free; inherited; end; procedure TDataModuleDict.SetString_Dict( const Value: TDictionary<string, String>); begin FString_Dict := Value; end; procedure TDataModuleDict.Setstring_Dict_Sync( const Value: TMultiReadExclusiveWriteSynchronizer); begin Fstring_Dict_Sync := Value; end; end.
On my console app startup init the Datamodule (not sure for Linux where I will put it, but will try later)
begin ReportMemoryLeaksOnShutdown := True; IsMultiThread := True; // Prepare Dict DataModuleDict:= TDataModuleDict.Create(nil); try DataModuleDict.initDict; // Important try if WebRequestHandler <> nil then WebRequestHandler.WebModuleClass := WebModuleClass; WebRequestHandlerProc.MaxConnections := 1024; RunServer(8080); except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; finally DataModuleDict.Free; end; end.
Using dict in code from seperate unit
unit DictU; interface uses system.sysutils , system.classes ,DataConU, Generics.collections , TemplateRecordU; type TDictClassV2 = Class ( Tobject ) private public procedure saveString ( akey , avalue : String ) ; function loadString ( akey : String ) :String; End; implementation uses HseqDmDictU; { TDictClassV2 } function TDictClassV2.loadString(akey: String): String; begin result:=''; DataModuleDict.string_Dict_Sync.BeginRead; try DataModuleDict.string_Dict.TryGetValue(akey,result); finally DataModuleDict.string_Dict_Sync.EndRead; end; end; procedure TDictClassV2.saveString(akey, avalue: String); begin DataModuleDict.string_Dict_Sync.BeginWrite; try try if not DataModuleDict.String_Dict.ContainsKey(akey) then DataModuleDict.String_Dict.Add(akey,avalue); Except on E : Exception do begin // logg error end; end; finally DataModuleDict.string_Dict_Sync.EndWrite; end; end; end.
If you would do it differently please let me know Thanks again...
-
12 minutes ago, Alberto Fornés said:No , this datamodule should be created at start, and always alive, you reference it from your controller (add datamodule unit to controller), and access it through TMultiReadExclusiveWriteSynchronizer.
OKI thanks alot 🙂
-
3 hours ago, Alberto Fornés said:Yes, you can use this in delphimvcframework. I declare such dictionary in the service unit (it's a windows service), or you can put it in a datamodule and create it at program start and destroy at the end. Webmodule can be created and destroyed several times during program execution.
Thanks, I can create it in a datamodule, but I still do not understand howto get this datamodule all the way down to the controller..
FMVC.AddController(TCustomersController);
Tcustomercontroller is a class created on request ,and the dict is created at startup.
I undestand that I can use/create a datamodule in the controller, but not howto use one created at startup, holding all the data,, normally I would send the dict to my class using a variable.
procedure TMyController.GetCustomers ; var lCustomersModule : TMyDataModule ; begin lCustomersModule := TMyDataModule . Create ( nil ) ; try lCustomersModule . QryCustomers . Open () ;
again thanks
I will also try a few options myself...
-
Hi ,
I have downloaded the delphimvcframework and it look’s great 🙂 thanks to @Daniele Teti , I have also bought the book 🙂 great reading, recommended.
I love this system , but will play more with it next couple of weeks.
A small question.
We have a small webbroker app today running on Apache AWS, we use a globale Dict as a keystore in all requests, and we use FMX data pooling.
FMX data pooling seems to be well documented in your book, and I guess I can almost use like before.
If you look at below code we use dict String_Dict as a global variable handled by
TMultiReadExclusiveWriteSynchronizer, any suggestion where to put his variable in delphimvcframework to be able to use it in the different controller / actions
like FMVC.AddController(TCustomersController);
In our webmodule we have a global variable and we use it with TMultiReadExclusiveWriteSynchronizer from system.sysutil
Example data from from our Webmodule
… public { Public declarations } SettingTemplateClass : TSettingTemplateClass; end: var WebModuleClass: TComponentClass = TWebModule1; String_Dict : TDictionary<string,String > ; string_Dict_Sync : TMultiReadExclusiveWriteSynchronizer; ….. // connecting String_Dict to a setting class unique for each request procedure TWebModule1.WebModuleCreate(Sender: TObject); begin SettingTemplateClass := TSettingTemplateClass.Create SettingTemplateClass.string_Dict_Sync := string_Dict_Sync; SettingTemplateClass.String_Dict := String_Dict; …. end; // a typical request sending string_dict further using settingTemplateClass to the class generating the response via usersession class procedure TWebModule1.WebModule1getlogondataAction(Sender: TObject; Request: TWebRequest; Response: TWebResponse; var Handled: Boolean); var userSession : TuserSession; ssoInternalClass : TssoInternalClass; companyid : Integer; lsession_data : String; begin companyid := strtointdef( getRequestParamValue (Request , 'companyid' ),0); userSession := TuserSession.create(SettingTemplateClass,getRequestParamValue(request,'token')); try if isSessionValid(Response,userSession) then begin lsession_data :=getRequestParamValue (Request , 'session_data' ); userSession.userSessionData.setSessionData(lsession_data); ssoInternalClass := TssoInternalClass.Create(userSession); try Response.content := ssoInternalClass.getLogonData( companyid ); finally ssoInternalClass.Free; end; end; finally userSession.Free; end; end; ………… initialization String_Dict := TDictionary<string,String >.create ; string_Dict_Sync := TMultiReadExclusiveWriteSynchronizer.Create; finalization String_Dict.Free; string_Dict_Sync.Free;
Example using string_dict in class generating the response data below .
string_Dict_Sync.BeginRead; try string_Dict.TryGetValue('severDayNumberString', severDayNumberString ); finally string_Dict_Sync.EndRead; end;
It it possible o do something similar with delphimvcframework, ?
For me it seems like difficult to share data to a controller ..
Thanks in advance
-
Thanks I will increase it
-
Hi,
we are using delphi webbroker backend as backend running on Apache docker AWS ECS
Been running smoothly for long time
I read this article today
https://community.embarcadero.com/blogs/entry/optimize-your-isapi-dlls--increase-maxconnections-1612
In my server I have ThreadsPerChild 50
in mpm_event_module
ServerLimit 4 MaxRequestWorkers 200 ThreadsPerChild 50
From my understanding in this article I should increase Application.MaxConnections to 50
like below
Web.ApacheApp.InitApplication(@GModuleData); Application.Initialize; Application.MaxConnections:=50; Application.WebModuleClass := WebModuleClass; Application.Run;
Is this correct understood, I have never had an error for this, but my understanding is that this number is 32 today and can raise an error ??
Bernt -
23 hours ago, mvanrijnen said:Don't know your "memcache" class, but why not use a simple dictionairy ?
Thanks for the Idea, and we are considering this.
But we have some consideration.
Our Delphi app is an API server , ISAPI webbroker module running on Apache linux docker several container instances behind a load balancer.
We can share a dictionary between the threads using
DataSync := TMultiReadExclusiveWriteSynchronizer.Create;
We use MPM_event on apache with this setup per production server
<IfDefine TSprod> ServerLimit 8 MaxRequestWorkers 200 ThreadsPerChild 25 </IfDefine>
And for setup above this will give 8 dictionary per server holding almost the same information, and 25 threads will share same dictionary this will be super fast but consume more memory than memcache.
We could consider reducing ServerLimit and increase ThreadsPerChild for this setup to not use so much memory per server this will increase threads using same dict.
We will test a little more on this.
B
-
OK thanks all for your help, I will give it a try...
-
Thanks,
Yes you are rigth it look like Ansistring 8 bit , and it also looks like my encoding send with it is not working for memcache.. 😞
IndyTextEncoding_UTF8
tcp.Socket.Writeln(Value,IndyTextEncoding_UTF8);
So I guess I Either I need to convert this delphi unicode UTF-16 string TemplateLines
TemplateLines:= TFile.ReadAllText(afilepath,Tencoding.UTF8 );
to Ansistringor
continue with base64encode
Are there an easy way in Delphi to convert a Delphi UTF-16 string to Ansistring ? Sorry if this is a dumb question...
WEB Standalone console APP “Indy http” in production ?
in Network, Cloud and Web
Posted · Edited by borni69
deleted...