Jump to content

Angus Robertson

Members
  • Content Count

    2063
  • Joined

  • Last visited

  • Days Won

    38

Posts posted by Angus Robertson


  1. On the general issue of using threads, remember that the ICS web and FTP servers support hundreds of simultaneous clients without using threads, I've never understood why so many people assume that threads are necessary to do two or more client connections at once.  Unless they are migrating from using blocking components... when the code should be rewritten properly.

     

    At some point, there will be a new ICS threaded server component, since threads are necessary to use more than one CPU, but the server will generally only need one thread per CPU, with multiple clients in each thread.

     

    Angus

     

     

     

     


  2. Using threads to handle multiple components in parallel is unnecessary, provided you are using async methods, so fully event driven.  I've done tests with hundreds of components running in parallel, the SSL negotiation takes a while so eventually limits the number in parallel.

     

    The FTP component should usually be created and destroyed within the thread. 

     

    There is a specific component for your purpose, TIcsFtpMultiThread in OverbyteIcsFtpMult.pas with a sample OverbyteIcsXferTst.

     

    Angus

     

     


  3. I've added a new event OnWSFramesDone to the Websocket client, called when the queue of frames has been sent, for flow control when sending a lot of data. Previously, the OnWSFreameSent was called after each frame, but it was not easy to tell if this was the last queued frame.  

     

    I've only tested it with a few frames of ASCII, the HttpRest sample really needs a new button to send a queue of files, which is on my list, but also needs a server able to accept those files.  But I'm sure you'll tell me if it does not work.  In SVN now, will be zipped overnight.

     

    Angus

     


  4. ICS V9.4 is ready for RAD Studio 12 Update 3, aka RAD Studio 12.3, and specifically the new 64-bit IDE.  The ICS Delphi V9.4 design packages already have Windows 64-bit as a target, allowing them to be installed into the 64-bit IDE.  The 32-bit IDE allows design packages to be built with target Windows 64-bit, but not installed.

     

    Note the 64-bit IDE does not support Windows 32-bit targets, only Windows 64-bit.  If you use the IcsInstallVclFmx.groupproj Build Group, all design packages will be built for Windows 64-bit.

     

    All the main samples build OK in the 64-bit IDE, with a Windows 64-bit target being automatically added, unfortunately even when deliberately missing such as the OverbyteIcsNetMon sample that does work in Windows 64-bit.

     

    Currently, GetIt only works in read-only mode in the 64-bit IDE, so you will need to install ICS using GetIt from the 32-bit IDE, then start the 64-bit IDE and manually install the three design packages.

     

    The ICS V9.4 C++ Builder packages can be currently be installed successfully into C++ Builder 12.3 32-bit IDE for the Windows 32-bit target only.  The 32-bit IDE should be able to build for Windows 64-bit and Win64x (Modern) as well, all the package paths are correct, but both targets currently give compiler errors.

     

    The C++ Windows 64-bit errors are four unresolved external, and the same error happens in Delphi 11, so should be fixable by someone with C++ knowledge.  Building for Win64x (Modern) gives lots of undefined symbol errors due to RAD Studio being unable to import other packages compiled built with C++ Win64x, this error was reported six months ago for C++ 12.2 but is not fixed in 12.3.  So ICS can not be used in the C++ 64-bit IDE that only supports Win64x (Modern), not Windows 64-bit.

     

    Separately, there is a new version of the DD Service Application Framework used by two of the ICS samples, adding support for 64-bit IDE Wizards.  Available shortly from https://www.magsys.co.uk/delphi/ddservice.asp

     

     

    Angus

     

    • Like 1
    • Thanks 3

  5. I've installed the Mosquitto Broker server locally, and it works OK against the ICS MQTT client.  

     

    Our sample currently always adds a default username/password, and this was upsetting test.mosquitto.org on the unauthenticated port, so the sample now allows them to be set specifically, and leaving them blank connects OK to test.mosquitto.org, which is also now on a ComboBox dropdown to make testing easier.

     

    But this does not explain why PicoMQTT fails (unless it's also authentication).  Mosquitto supports 3.1, 3.1.1 and 5, I'll see if I can disable 3.1.  There is a comment in the conf file about clients connecting with 3.1.1 and a zero length client Id so the server allocates one, perhaps this explains the difference. 

     

    Angus

     


  6. Sorry, we ceased support for Windows XP several years ago, the OpenSSL DLLs do not support it.  The oldest OS supported is Windows 7, and that is rarely tested. 

     

    ICS does not use ConvertThreadToFiberEx, perhaps OpenSSL does.  So you may be able to disable all SSL functionality to keep XP working, but you can not use very old OpenSSL DLLs. 

     

    Or stick with old unsupported code for an unsupported OS. 

     

    Angus

     

     

     


  7. Sorry, no simple answer, the MQTT component was contributed, and only tested against itself.  I don't have any other local MQTT servers to test against, although it was tested against some public servers.  I'm aware there are various versions of MQTT and some servers are non-standard, questionable whether ICS should break the protocol to work with devices that don't follow it, please search back in this forum, I think it's been discussed before.

     

    Otherwise, you'll need to use Wireshark or something to see how the other clients are communicating with the server, and change the component to match that.

     

    Angus

     


  8. I'd make the general point that ICS is an async library, generally you should never use Sleep(), but events.  If you want to delay something, use triggers within a timer.  Having multiple ProcessMeesages everywhere is also bad design.  

     

    Your code should be packaged into an object with events, called before a single message loop.   In fact, this design would mean you can test and debug your WS code in a simple GUI before using it in a DLL. 

     

    From your various comments, I gather you are writing a Websocket client DLL that sends large binary blocks of data to a server, All my testing and the ICS samples are server to client communication, although in theory the code is two-way and similar in client and server.  But I simply don't have a way to easily test your requirement.

     

    Angus

     


  9. Sorry, no idea off hand, not used Websockets for sending large binary blocks, only simple ANSI/HTML packets. 

     

    Reproducing your scenario is not trivial and would take some effort, I'll put it on my list, but it might take a while.  Are both server and client ICS WS apps?

     

    Angus

     


  10. ICS has a specific component for sending email, TIcsMailQueue, it uses a thread with a TSslSmtpCli component to send email previously saved in EML files, using multiple servers and retries over many hours to ensure email is sent.  It handles all the SSL stuff for you.  

     

    The main sample is OverbyteIcsMailQu which has OAuth2, but the component is also used in several server samples.

     

    Haven't written an ISAPI for many years, but the component should run in a DLL easily.

     

    Angus

     

     

     

     


  11. CloudFlare is the obvious solution for most commercial web sites, although I find my link site checker app being blocked from some sites CF 'protects'. 

     

    But this is an ICS web server, and developers have vastly more control over checking and blocking connections than sites using Apache, etc, that need extras to protect them. 

     

    Although I get the usual general hackers, they are normally easy to block, anyone accessing the SSL site using an IP address immediately goes on the blocked list, or trying to access CGI script, etc.  

     

    Anagus

     


  12. Thanks for the thoughts.  

     

    The user agent strings are partly randomised, lots of different Chrome/xx versions, the Safari version seems to be the same, but is probably legitimate.  The SSL HELO packet has some unknown EC groups, but Chrome often has test groups.  The ALPN is always blank, and the requests use a URL without www, but blocking either of those would also hit legitimate API users. 

     

    The server does not currently log any request headers, not sure if VPNs would add anything to identify themselves, as proxies normally do.  

     

    One possible solution would be counting IP accesses within a /24 or larger block, although that might include some corporates with outgoing blocks, I'd need to update my white lists as well. 

     

    Don't want to spend too much time on a rare problem...

     

    Angus

     


  13. My Chinese hackers have changed strategy to get around my IP address blocks and access my web site database, that restricts free access to 50 requests a day, paying for unlimited access seems beyond them. 

     

    So now they are using VPNs, making two requests at a time from thousands of different IP addresses around the world, 3,500 over the last 48 hours, with requests now repeating after 24 hours, |previously I cleared the block list after six hours of no repeat access. 

     

    I've not yet managed to define an automated strategy to block relatively random IPs, a  CAPTCHA would work, but don't want to annoy my users, likewise giving them a free login.   Has anyone got a better strategy for blocking unwanted access by IP?

     

    Meanwhile, I'll add /24 level IP blocks manually for a few dozen VPN ranges, to means the server will immediately close any connections from those ranges.  Last time I did this to block TOR  nodes, I accidentally blocked some large corporates resulting in some interesting telephone calls. 

     

    Angus

     


  14. That sort of error usually happens if you don't open the form in the IDE, and the new properties are not saved to the DFM, so fail to be read when executed.  That property was added six months ago, and no-one else has reported a problem in that time.

     

    Angus

     

×