Search the Community

When using the `TSslWebSocketCli.WSConnect` method an expected Connection Refused error is thrown (See Attached). I want to hide this. How can I do so? I have tried wrapping a `try..except` block around it without any luck. I have also assigned the Error and Exception events to empty procedures.

Hi Some protocols require Sec-WebSocket-Protocol header to be set when connecting, I made a patch to allow it in OverbyteIcsWebSocketCli.pas OverbyteIcsWebSocketCli.pas.patch

Hello, I'm trying to configure the most simple websocket client (with TSslWebSocketCli) and server (with TSslHttpAppSrv) in order to get them communicate with each other locally. Worth noting that I'm a beginner in the internet communication protocol world, this is why I'm trying to get the minimum working and then I will build from that. I'm using the ICS 9.0 C++ package, I believe I have a configuration problem more than a C++ problem. For the client part, I configured the following (no TLS for now) : client->SocketFamily = sfAnyIPv4; client->DebugLevel = DebugHdr; client->CertVerMethod = CertVerNone; client->SslCliSecurity = sslCliSecDefault; client->URL = "ws://127.0.0.1/"; client->WSPingSecs = 10; event handlers to get something printed in the terminal : client->OnHttpRestProg = HttpRestProg; client->OnWSFrameRcvd = FrameRcvd; client->OnWSConnected = Connected; client->OnWSDisconnected = Disconnected; For the server part, based on the Delphi samples OverbyteIcsBasicWebServer1.pas and OverbyteIcsSslMultiWebServ.pas, I configured : server->AuthRealm = "ics"; server->SocketErrs = wsErrFriendly; server->ExclusiveAddr = true; server->SessionTimeout = 300; server->MaxSessions = 100; server->OnDisplay = Display; server->SslEnable = false; server->SslCliCertMethod = sslCliCertNone; server->SslCertAutoOrder = false; server->CertExpireDays = 30; server->OcspSrvStapling = false; server->AddGetAllowedPath("/", afBeginBy, "WEB-APP"); server->OnClientConnect = ClientConnect; I added an host : server->IcsHosts->Clear(); TIcsHost *host = new TIcsHost(server->IcsHosts); host->HostEnabled = true; host->HostNames->Text = "localhost"; host->HostTag = "WEB-APP"; host->Descr = "WebApp Server"; host->BindIpAddr = "127.0.0.1"; host->BindNonPort = 80; host->BindSslPort = 0; host->WebDefDoc = "index.html"; host->WebDocDir = dir + "data"; When the client is connecting (TSslWebSocketCli.WSConnect), OnClientConnect is triggered server side, then : THttpWSSrvConn *conn = (THttpWSSrvConn *)Client; conn->WSessionCookie = "OverbyteIcsWebAppServer" + server->Port; conn->OnWSLogEvent = ClientLogEvent; conn->OnWSHandshake = ClientHandShake; conn->OnWSDisconnected = ClientDisconnected; conn->OnWSFrameRcvd = ClientFrameRcvd; conn->OnWSFrameSent = ClientFrameSent; conn->OnWSReady = ClientReady; conn->OnWSPingTimer = ClientPingTimer; conn->OnBgException = ClientBgException; Then the server handle the request (these are called : THttpConnection::ProcessRequest(), ::ProcessGet(), ::ProcessGetHeadDel(), ::SendDocument()...) and returns an "HTTP/1.1 200 OK" response to the client. From TSslWebSocketCli.WSConnect I get "WebSocket: Failed to Connect:" but the LastResponse string is empty. It looks like the server treats my websocket upgrade request as a "classic" request, so it returns an "HTTP/1.1 200 OK" response instead of the expected "HTTP/1.1 101 Switching Protocols". To make sure my client part was ok, I quickly tried it on a node.js websocket server, no problem, I get the "101 Switching Protocols" response and the websocket connection is established. If I do not use TSslWebSocketCli.WSConnect but do a "manual" GET with an upgrade request header and on server side add an url handler (TUrlHandlerIndexHtml) and send a response header through AnswerStream, the connection is established. But if I send frames from the client, OnWSFrameRcvd is not triggred server side (as expected I believe since server has no clue I handled the upgrade "myself"). Hopefully this post makes sense, in short, I don't understand why my upgrade request on url: "ws://127.0.0.1/" fails on the ICS server, any help would be appreciated. Thanks in advance.

I've added a new event OnWSFramesDone to the Websocket client, called when the queue of frames has been sent, for flow control when sending a lot of data. Previously, the OnWSFreameSent was called after each frame, but it was not easy to tell if this was the last queued frame. I've only tested it with a few frames of ASCII, the HttpRest sample really needs a new button to send a queue of files, which is on my list, but also needs a server able to accept those files. But I'm sure you'll tell me if it does not work. In SVN now, will be zipped overnight. Angus

I believe the Internet Component Suite is amazing and a godsend for Delphi. Thank you for everything you do to support this software. I wish to use WebSockets with ICS Delphi components. I wish to use this in a very large and important system (cannot name it). I am challenged to prototype this because the Overbyte demo OverbyteIcsSslMultiWebServ.dpr has so many features it is impossible to extract a simple webSocket server. I wish to demonstrate ssl/tls and non ssl/tls connections. I am not interested in echo servers/clients. I want servers/clients that can handshake and exchange messages with each either other. How can I demo this. The client side I have been able and is fairly straight forward. The server side is too complex. Please provide a simple Websocket Server application that supports ssl/tls and non ssl/tls connections. Thank you Richard Maley

ICS V9.4 has been released at: https://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10, 10.1, 10.2, 10.3, 10.4, 11 and 12 and C++ Builder 10.4, 11 and 12. ICS supports VCL and FMX, Win32 and Win64 targets. The distribution zip includes the latest OpenSSL 3.0.16, 3.2.4, 3.3.3 and 3.4.1 for Win32 and Win64. Changes in ICS V9.4 include: 1 - Completed the ICS Application Monitoring system added in V9.3, designed to locally and remotely monitor ICS servers and applications, and to locally restart applications on demand or if they crash. It comprises a small TIcsAppMonCli client monitoring component that is added to ICS applications, usually Windows Servers, but also client applications. This client component communicates with a TIcsAppMonSrv server component, usually running as a Windows Service on the same server so it is able to restart applications, but can also support clients on a LAN. The monitoring server has web and Websocket servers, allowing remote browsers to view the state of all applications being monitored by the server with a continually updated web page. There is also ICS Application Monitor - Remote Manager application that provides remote monitoring of multiple ICS Application Monitor servers using Json web and Websocket requests on a single screen. 2 - The SMTP client component fixes a bug introduced in V9.3 which could corrupt the Content-Transfer-Encoding header line. 3 - In TWSocket, fixed a potential problem using multiple threads where a new connection opened very quickly (ie localhost) and then stalled due to an unexpected connection state. Made DataToString Unicode compatible, only used for diagnostic dump logs. 4 - In the HTTP client, fixed a check for an overflowing buffer when receiving very long headers that could cause failure detecting headers end. Made several URL validation functions public: GetProtocolPort, IsSSLProtocol, IsKnownProtocol and IsKnownProtocolURL. 5 - When creating PKCS12/PFX certificates, change the 3DES cipher to AES256 if the legacy provider is not loaded. 6 - In the TIcsFtpMulti component, skip download of zero length file by creating an empty file, previously this got SSL handshake error. Don't report directories as being downloadable, they are not. If extended passive mode allowed, send EPSV ALL at start so firewalls and NAT routers can handle sessions more efficiently. Added CheckBadUnicode property defaults to false, so that checks for bad Unicode to Ansi conversions with ? are skipped, allowing more complex paths without errors. 7 - The FTP server FEAT request now returns EPRT and EPSV which have been supported for IPv6 for years, but were not advertised for IPv4. 8 - The OverbyteIcsSnippets sample adds two new simple REST snippets to Get/Post Parameters that send them to an ICS server, and the server echoes back those params so you can check what was actually sent. 9 - When loading the OpenSSL DLLs, no longer check they are digitally signed for Windows XP, 2003, Vista and 2008, they don't recognise SHA-256 code signing, never tested since no longer have those old versions available. 10 - The TRestParams method AddItemSO to add a SuperObject now has an Escape parameter defaulting to True, so non-ASCII characters are escaped by default. 11 - The Proxy component TProxyTarget now has a SocketFamily property so target connections can be restricted to TSocketFamily values. Added property SrvTotSess count of server session connections for logging. 12 - In the HTTP Application Web Server, fixed a memory leak with multiple virtual PUT and POST documents. 13 - The OverbyteIcsJoseTst sample 'Sign/Verify Data' tests now support hashes other than SHA-256, selected from the Key and Signing Hash Algorithm drop down list. Also, a private key matching that selection is created automatically, including X25519. 14 - There is a new unit OverbyteIcsWinUtils that contains Windows API functions, built from selected Magenta Systems libraries, with functions needed to build and control Windows Service applications, accessing the Windows registry, Windows firewall, Windows tasks, hardware, and with simple encryption for passwords. Most of these functions are used by the TIcsAppMonSrv server component and IcsAppMon sample, but should have much wider use for Windows Service server applications in particular, like allowing firewall access. It's planned to move most other Windows specific function here for ICS V10. 15 - In the ICMP component, fixed a problem setting property PingMsg to the text to ping. 16 - In the TIcsIpStrmLog component, added method ListenStates which for logprotUdpServer and logprotTcpServer returns a multiline string listing the IP, port, SSL and state of all socket listeners. The CurSockets property now reflects actual TCP Server clients. 17 - In the TIcsMailQueue component, don't keep retrying email that is too short to send with no body or with no sender headers. Added more error handling if the SMTP component fails to build the EML spool file. 18 - In OverbyteIcsUtils, finished the cleanup of old Base64 functions by adding new IcsBase64 functions using TBytes internally to replace old Base64 functions that used AnsiChars, with no overloaded versions for simplicity. Old Base64 versions retained as deprecated for user applications, please update to the IcsBase versions. Added IcsTBytesCompare to compare two TBytes. Added IcsOutputDebugStr for Posix and Windows. Added IcsDateToAStr and IcsDateTimeToAStr with alpha month (Jan/Feb). 19 - The ICS C++ packages for C++ 10.4 and later have been updated with the correct paths for the three supported platforms, and all build and install correctly for Win32. Win64 should also build, but not Win64x Modern which needs fixes in a future release of C++ 12. The release notes for V9.4 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.4 All ICS active samples are available as prebuilt executables, to allow ease of testing without needing to install ICS and build them all. There are four separate zip files split into clients, servers, tools and miscellaneous samples which can be downloaded from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples Angus

I'd make the general point that ICS is an async library, generally you should never use Sleep(), but events. If you want to delay something, use triggers within a timer. Having multiple ProcessMeesages everywhere is also bad design. Your code should be packaged into an object with events, called before a single message loop. In fact, this design would mean you can test and debug your WS code in a simple GUI before using it in a DLL. From your various comments, I gather you are writing a Websocket client DLL that sends large binary blocks of data to a server, All my testing and the ICS samples are server to client communication, although in theory the code is two-way and similar in client and server. But I simply don't have a way to easily test your requirement. Angus

Using OverbyteIcsSnippets as an example. Websocket CLIent section, after wsconnect is successful it falls in to this loop. while NOT AbortFlag do begin Application.ProcessMessages; if Application.Terminated then Break; if IcsTestTrgTick64(TrgTimerEnd) then Break; end; Mine is a console app and I do something almost identicle to this on startup. I connect then fall in to the main thread loop waiting for a signal to end. My application runs at 48% cpu usage just like the example. I realize this is a tight loop. Running as-is the OnWSFrameRecv will receive a 50M file in a few seconds. However when I try to address the high cpu issue and add a simple sleep(5), which drops the CPU down to less than 5 percent. The 50M file receive via the OnWSFrameRecv event now slows down to 9 minutes. Its like the receive is blocking. I'm not sure why a simple sleep would do this. Is the sleep cascading in the tight loop and causing the overall main thread to sleep much longer than the 5ms maybe? I know there is missing info here. The application is quite large but the websocket connect and receive is very basic. I rebuilt excluding throttling in the library but it didn't make any difference. Is there a better way this not CPU intensive to loop and wait for the termination? The goal is to receive large files via WS and not have the appl;ication idle at this high usage. Any help would be appreciated. Regards

Are you using an old version of ICS? ICS added full Websocket server and client support two years ago, and handles the upgrade negotiation process automatically. The main unit is OverbyteIcsWebSocketSrv.pas which contains a derived connection class THttpWSSrvConn that handles the Websocket protocol. There are two samples OverbyteIcsSslMultiWebServ.dpr and IcsAppMon.dpr that show how it all works. Angus

var ARequestInfo := THttpConnection(Client); if SameText(ARequestInfo.RequestConnection,'Upgrade') then ARequestInfo.Answerstring(Flags, '101', '', 'Upgrade: websocket' + #13#10 + 'Connection: Upgrade' + #13#10 + 'Sec-WebSocket-Accept: ' + WebSocketAccept, '');//WebSocketAccept is working good with idhttpserver what ever i do i can't make this work and its just the first step maybe i am trying wrong way maybe i need to use ARequestInfo.Send(TWSocketdata,len) or ARequestInfo.StartConnection; idk what i have to do to make handshake then send client alert from server

I offer clients a REST API service to look-up telecommunication information, using the ICS web application server and MS SQL server. It works well, for low volumes of queries, but most users start a new SSL/TLS session for each query, which becomes a limiting factor with performance. So I want to offer a Websocket API as well, so one SSL/TLS session stays open, with just simple request/response packets sent. But how to adapt the REST HTTP request/response to Websocket? My queries are simple URL parameters, ie codelookapi.htm?numhistory=118118. Should the Websocket message just be the arguments or include the full or partial URL as well? Or something else, like a command? Should the initial Websocket request allow arguments, or just open the connection? Should be Websocket response message be just a JSON block, or include a wrapper of some sort, like the HTTP response header? Has anyone done a similar design? I just want to make it easy for clients using standard Websocket client libraries to integrate the new API. Angus

I'd first make the general comment that it is always best to develop and test the two parts of client/server applications separately, against known working versions. In this case, with the ICS OverbyteIcsSslMultiWebServ and OverbyteIcsHttpRestTst samples. Don't know if C++ allows you to build them, but the wiki site allows you to download prebuilt executable files. Your settings are missing a websocket path or page, just ws://127.0.0.1/ so you are assuming the web server default HTML page is actually a Websocket request, this was never testing with the ICS web server, perhaps my fault for not expecting anyone to try that. Since you set default page to index.html, I assume that is the websocket URL you are checking for, but you don;t show any of that code. I find it best to use a virtual path /websocket/ to clarify that such requests from HTML. Angus

There is a new ICS sample OverbyteIcsBasicWebServer.dpr in SVN, will be zipped overnight, which is a simplified version of OverbyteIcsSslMultiWebServ ignoring configuration INI files, security features, session data, most demo pages and most logging, and settings for localhost set in code, search for IcsHosts to change IP addresses, etc. But this sample will be a much easier starting point for those developing web or Websocket servers with ICS. I'm going to copy most of the code into another new ICS sample that needs a Websocket server. Angus

I am using the ICS TWSocketServer to communicate between a delphi application and a collection of React applications embedded in Edge browsers. The websocket server is running in the delphi application as are the React web applications. When one of the React applications attempts to connect with the websocket server it sends a json frame with some information needed for authentication. I am able to extract the header information but I cannot figure out how to get the json. Could someone help me? Richard Maley

If you really mean the websocket protocol and not HTTP, there are no header or bodies or protocol, once a websocket is open there is simply a two way TCP stream, and you can send what you like. That stream arrives at the server in the ClientWSFrameRcvdEvent as a string packet and TWebSocketReceivedFrame to tell what typer of data is arriving (text, binary, or closing), it's up to the developer to decide how to interpret that data. Angus

The OverbyteIcsSslMultiWebServ sample serves both web and websocket pages, since all websocket requests start with an HTTP request to port 80/443. But you can remove all the code relating to creating web pages from the sample, the OverbyteIcsSslMultiWebxx units, etc, leaving just the default page and websocket handlers ClientWSxx and WSxx which do the websocket events. I will look at a new simple web server sample, similar to the client snippet samples, but unlikely to be ready for the next release. Angus

Angus Using TSslWebSocketCli, after setting CertifyMethod to CertVerNone, verifypeer no longer gets executed. But the Websocket connects and is ready. If it completely bypasses all internal chain verification AND does not fire verifypeer (which it seems to behave the way you described), how do I do a manual cert verification? These are self signed certs and we validate using hashes, serialnum, pubkey(Mod N) etc... whatever the cert provides that is unique. Setting the CertifyMethod to CertVerBundle or CertVerdWinstore fires verifypeer but the certificates fail regardless of the _OK_ flag. I'm looking for something that may be described as CertVerSelf. This would fire the verifypeer to be used without other verification and the _OK_ would signify pass/fail of the cert. Is there a method in place to accomplish this already?

CertVerifyMethod := CertVerNone is how you bypass internal chain verification. TSslWebSocketCli is derived from TSslHttpRest which has an internal TSslContext, and it's that component that does all the SSL stuff. You can use an external SslContext, see the notes in the rest component, for instance if you are using a hundred rest component instances to download lots of stuff in parallel. If you are using the real WebSocket protocol, it exchanges frames between client and server, and vice versa, and there are several methods WSSendText, WSSendBinary, WSSendFrame, etc, with the OnWSFrameRcvd event being received frames, which you need to check the data type and then process. IsWSConnected does as it says. If you want to send raw data without using WS frames, you are using the wrong component. Angus ,

The purpose of the Websocket is to exchange data with a server, so it opens a connection and does a GET to the path you specify, just as HTTP does, if you don't want that it's not Websockets. I suggest you try using the OverbyteIcsHttpRestTst sample I mentioned, if that does not work you would appear to be using a custom protocol. In a console application, you need to call the message pump after every line that accesses IP functions, ICS is fully event driven. The sample I mentioned makes a single HTTP command, waitis for it to finish and exits. If you are making a series of commands, the console applications gets more complicated. Angus

My apologies for the confusion. Yes Websocket as a protocol. I understand that. I was originally using OverByteSimpleSslCli as an example reference and it uses TSslWSocket and TSslContext and it seemed like a great starting point. So I just now switched to TSslWebSocketCli and can do a little refactoring. But in regards to a console example and implementing events would be very helpful. I'll for sure checkout the Snippets. Thanks Angus.

There is a terminology issue here, Websocket is a specific protocol based on HTTP. TWSocket is the name of the base ICS component for all low level socket operations, and does not handle high level protocols like Websocket. ICS has a TSslWebSocketCli component, there is a simple function doWebSocketClick in the OverbyteIcsSnippets sample you can build and run which accesses one of my servers, or a full example in the OverbyteIcsHttpRestTst sample that should allow you to access your site. ICS has several console samples, but they are no longer in the main distribution not being needed very often, I'll have to look one out. Angus

I want to implement a Delphi WebSocket server to send binary data (Stream and byte array) to javascript client. This could be done after receive a client message OR EVEN as unidirectional message so no client message need to be received. Below the pseudocode. var WSServer: TWSocketServer; Case A: I want to send back the binary data after receive a message WSServer.OnClientConnect := procedure (Sender: TObject; Client: TWSocketClient; Error: Word) begin TWebSockSrvClient(Client).OnWebSocketMessage := WebSocketMessage; end; procedure WebSocketMessage(Sender: TObject; Msg: string); var MemStream: TMemoryStream; SocketConn: IWebSocketConnection; begin //create and populate MemStream if (Msg = 'A') then begin SocketConn := TWebSocketSocket(Sender).getConnection; //I need to send back the MemStream but I found only these method: //SocketConn.sendString //SocketConn.sendFrame //SocketConn.sendMessage end; end; Case B: send binary data to some client from button click procedure TForm1.btnSendBufferClick(Sender: TObject); var ArrB: TBytes; begin ArrB := [1, 2, 3]; //I tried WSServer.Client[0].SendTB(ArrB) but is doesn't work end; I check OverbyteIcsWebSocketSrv.dpr demo but only strings are send. Any advice?

There is a clever improvement in the GetIt ICS installation process, thanks Embarcadero. After the packages have been built and installed, the ICS demos-delphi-vcl project group opens in the IDE, allowing all the samples to be viewed and built, recommend saving the group as a favourite so it can be easily found. The group opens with the OverbyteIcsSnippets project, that provides one button examples of HTTP and FTP multiple file downloading and uploading, HTTP REST requests, Websocket client, TCP socket traffic, and sending email using Mail Queue. Angus

Hi, Here may be a bug,would you please have a look?, unit: OverbyteIcsWebSocketCli.pas position: line 1494 ServerKey := Base64Encode(String(SHA1ofStr(s))); ------------ test: x3JJHMbDL1EzLkh9GBhXDw==258EAFA5-E914-47DA-95CA-C5AB0DC85B11 expect: HSmrc0sMlYUkAGmm5OPpG2HaGWk= output:HSmrc0sMlYUkAGmm5OPpG2E/aQ== ------------ test code: procedure TForm1.Button3Click(Sender: TObject); var s : AnsiString; ServerKey:string; begin s := AnsiString(edit1.Text); ServerKey := Base64Encode(String(SHA1ofStr(s))); memolog.Lines.Add(ServerKey); end; node.js - Base64 encoding for Sec-WebSocket-Accept value - Stack Overflow

This Websocket client problem should be fixed in SVN and the overnight zip, assuming it was an ANSI/Unicode casting problem with non-English character set conversions, which I'm unable to reproduce. It did not affect Websocket server which had no casting, and yet always worked against our client. The real problem is ICS has overloaded versions of many functions for ANSI and Unicode, and the compiler does not always choose the correct version if input and output parameters don't match, fixed by using a specific ANSI function. I'm slowly adding TBytes versions for binary data to avoid such problems. but it's a long job. Angus