Remy Lebeau

Once upon a time, Indy had a larger dev team, but I suppose no-one ever had the time or opportunity to implement SSH. Not that it is a trivial protocol to implement, what with encryption and all. Now I'm the only dev left, and I definitely don't have the time to do something that large-scale on my own. I don't even have a working SChannel IOHandler working yet, and that is just using a few (albeit complicated) system APIs. I suppose someone could write an IOHandler wrapper for libssh or similar library, though.

The response's "Content-Type" header tells you the exact format of the data, such as " application/json", "application/xml", etc.

That is still the case. And that is my only complaint I have about TService - lack of access to newer SCM features introduced since Win2K (and I have complained about it for years, and posted several QC tickets about it, which are now lost and I don't care to repost them). Most things I can just work around in my own code by calling various APIs directly. But TService's use of an old Handler() callback instead of a newer HandlerEx() callback is still an issue for some features to work correctly.

Most of my services are written using the standard TService, with separate apps for their UIs. But I do have one service that runs in both service and app mode, and I use SvCom for that project.

A TListBox or TListView would be far more appropriate for that than a TMemo.

Are you trying to leave TIdFTP connected to the server during idle periods when no uploads are being performed? That is not likely to work, unless you send commands periodically, even if just a simple NOOP, so the server knows you still want the connection to stay open. Like many Internet protocols, FTP is a command+response protocol. The server can't send unsolicited messages whenever it wants, it can only send responses to commands. So no, there is usually no message sent when the server disconnects, unless it is sent as a response to a command (the FTP protocol has a 421 error code defined for this exact purpose). However, Indy will raise an exception when it receives an error response to a command, or tries to read from/write to the socket after it has been disconnected. Usually no. Just have your thread perform the TIdFTP.Put() normally and handle any exception it raises, and also have it either 1) connect TIdFTP only when it has transfers to perform and then disconnect, or else 2) call TIdFTP.Noop() at fixed intervals in between periods of active transfers. You don't need to connect/disconnect for each individual file. You can connect once, do all of the transfers you need, and then disconnect. Just don't leave the connection idle for a long time, as the server (or a network firewall/router) is likely to close the connection. Well, then you need to add throttling to your code logic. Don't upload more than a few files at a time. Many servers have such a restriction. Either have 1 thread that sleeps between transfers, or make a few threads with their own connections to the server and then queue the files and have idle threads pulling from the queue as needed.

That will only work in non-Unicode environments (Delphi/C++Builder 2007 and earlier, and FreePascal when not in UNICODESTRINGS mode), where string is AnsiString. In Unicode environments, string is UnicodeString instead, and so using UTF8Encode() is deprecated and unnecessary.

In Unicode environments (Delphi/C++Builder 2009+, and FreePascal in UNICODESTRINGS mode), Indy encodes the Subject (and other headers) using UTF-8 by default (which can be overridden using the TIdMessage.OnInitializeISO event). But for the message body, you need to manually set the TIdMessage.CharSet and/or TIdText.CharSet property to 'utf-8', as they default to 'us-ascii' if left blank and the corresponding ContentType property is set to a 'text/...' media type, like 'text/html' (I am open to consider changing this default to UTF-8 instead). Note that the TIdMessageBuilder... classes already default the CharSet properties to UTF-8 in Unicode environments. No, it doesn't matter. That has no effect on how Indy encodes the email. And in fact, since the TIdMessage.Body and TIdText.Body properties are TStrings objects, you can't store UTF-8 text in the them anyway, only UTF-16 text. So if you want the text encoded as UTF-8 during transmission, make sure the corresponding CharSet properties are set to 'utf-8'. But, are you loading that file as UTF-8? Does the file have a UTF-8 BOM? If not, you must explicitly make use of TEncoding.UTF8 or equivalent when loading the file. What does your loading code look like? That is not true at all. TIdMessageBuilderPlain has a PlainText property for the body, just as TIdMessageBuilderHtml has an Html property for the body. I think you are just confused because the PlainText property is not declared in the TIdMessageBuilderPlain class, it is actually inherited from the TIdCustomMessageBuilder class (which means TIdMessageBuilderHtml also has the PlainText property, so that you can provide alternative plain text for HTML-unaware readers). You should read my articles on this topic on Indy's website: HTML Messages New HTML Message Builder class I have posted an update to my answer on that question. As of 2 months ago, the order in which ContentType and CharSet proerties is set is no longer important. No, you do not. However, if you do need to specify a CharSet explicitly, you should do so on the MessageBuilder classes themselves. TIdMessageBuilderPlain has a PlainTextCharSet property, and TIdMessageBuilderHtml has PlainTextCharSet and HtmlCharSet properties. With that said, try something more like this: procedure SendEmailIndy( const SMTPServer: string; const SMTPPort: integer; const SMTPUserName : string; const SMTPPassword : string; const FromName, FromAddress: string; const ToAddresses: string; //comma "," separated list of e-mail addresses const CCAddresses: string; //comma "," separated list of e-mail addresses const BCCAddresses: string; //comma "," separated list of e-mail addresses const Subject: string; const EmailBody: string; const IsBodyHtml: Boolean; //verses Plain Text const Attachments: TStrings; UseTLS : Boolean); var smtp: TIdSMTP; // IdSmtp.pas TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support msg: TidMessage; // IdMessage.pas builder: TIdCustomMessageBuilder; //IdMessageBuilder.pas s: string; emailAddress: string; begin msg := TIdMessage.Create(nil); try if IsBodyHtml then begin builder := TIdMessageBuilderHtml.Create; end else begin builder := TIdMessageBuilderPlain.Create; end; try try if IsBodyHtml then begin TIdMessageBuilderHtml(builder).Html.Text := EmailBody; TIdMessageBuilderHtml(builder).HtmlCharSet := 'utf-8'; TIdMessageBuilderHtml(builder).HtmlContentTransfer := '8bit'; end else begin TIdMessageBuilderPlain(builder).PlainText.Text := EmailBody; TIdMessageBuilderPlain(builder).PlainTextCharSet := 'utf-8'; TIdMessageBuilderPlain(builder).PlainTextContentTransfer := '8bit'; end; if Attachments <> nil then begin for s in Attachments do builder.Attachments.Add(s); end; builder.FillMessage(msg); except {$IFDEF TRACEDEBUG} on E : Exception do begin AddDebugEntry('Email Builder Exception : ' + E.Message); Exit; end; {$ELSE} Exit; {$ENDIF} end; finally builder.Free; end; msg.From.Name := FromName; msg.From.Address := FromAddress; msg.Subject := Subject; for s in ToAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.Recipients.Add.Address := emailAddress; end; for s in CCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.CCList.Add.Address := emailAddress; end; for s in BCCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.BccList.Add.Address := emailAddress; end; smtp := TIdSMTP.Create(nil); try smtp.Host := SMTPServer; smtp.Port := SMTPPort; smtp.Username := SMTPUserName; smtp.Password := SMTPPassword; if UseTLS then begin TLSHandler := TIdSSLIOHandlerSocketOpenSSL.Create(smtp); smtp.IOHandler := TLSHandler; if SMTPPort = 465 then begin smtp.UseTLS := TIdUseTLS.utUseImplicitTLS; end else begin smtp.UseTLS := TIdUseTLS.utUseExplicitTLS; end; end; try smtp.Connect; except {$IFDEF TRACEDEBUG} on E : Exception do begin AddDebugEntry('SMTP Connect Exception : '+E.Message); Exit; end; {$ELSE} Exit; {$ENDIF} end; try try smtp.Send(msg) except {$IFDEF TRACEDEBUG} on E : Exception do AddDebugEntry('SMTP Send Exception : '+E.Message); {$ENDIF} end; finally smtp.Disconnect; end; finally smtp.Free; end; finally msg.Free; end; end; Note that TIdMessageBuilderHtml CAN create plain-text emails as well, if no HTML is assigned, so you can alternatively use this instead: procedure SendEmailIndy( const SMTPServer: string; const SMTPPort: integer; const SMTPUserName : string; const SMTPPassword : string; const FromName, FromAddress: string; const ToAddresses: string; //comma "," separated list of e-mail addresses const CCAddresses: string; //comma "," separated list of e-mail addresses const BCCAddresses: string; //comma "," separated list of e-mail addresses const Subject: string; const EmailBody: string; const IsBodyHtml: Boolean; //verses Plain Text const Attachments: TStrings; UseTLS : Boolean); var smtp: TIdSMTP; // IdSmtp.pas TLSHandler : TIdSSLIOHandlerSocketOpenSSL; // TLS support msg: TidMessage; // IdMessage.pas builder: TIdMessageBuilderHtml; //IdMessageBuilder.pas s: string; emailAddress: string; begin msg := TIdMessage.Create(nil); try builder := TIdMessageBuilderHtml.Create; try try if IsBodyHtml then begin builder.Html.Text := EmailBody; builder.HtmlCharSet := 'utf-8'; builder.HtmlContentTransfer := '8bit'; end else begin builder.PlainText.Text := EmailBody; builder.PlainTextCharSet := 'utf-8'; builder.PlainTextContentTransfer := '8bit'; end; if Attachments <> nil then begin for s in Attachments do builder.Attachments.Add(s); end; builder.FillMessage(msg); except {$IFDEF TRACEDEBUG} on E : Exception do begin AddDebugEntry('Email Builder Exception : ' + E.Message); Exit; end; {$ELSE} Exit; {$ENDIF} end; finally builder.Free; end; msg.From.Name := FromName; msg.From.Address := FromAddress; msg.Subject := Subject; for s in ToAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.Recipients.Add.Address := emailAddress; end; for s in CCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.CCList.Add.Address := emailAddress; end; for s in BCCAddresses.Split([',']) do begin emailAddress := Trim(s); if emailAddress <> '' then msg.BccList.Add.Address := emailAddress; end; smtp := TIdSMTP.Create(nil); try smtp.Host := SMTPServer; smtp.Port := SMTPPort; smtp.Username := SMTPUserName; smtp.Password := SMTPPassword; if UseTLS then begin TLSHandler := TIdSSLIOHandlerSocketOpenSSL.Create(smtp); smtp.IOHandler := TLSHandler; if SMTPPort = 465 then begin smtp.UseTLS := TIdUseTLS.utUseImplicitTLS; end else begin smtp.UseTLS := TIdUseTLS.utUseExplicitTLS; end; end; try smtp.Connect; except {$IFDEF TRACEDEBUG} on E : Exception do begin AddDebugEntry('SMTP Connect Exception : '+E.Message); Exit; end; {$ELSE} Exit; {$ENDIF} end; try try smtp.Send(msg) except {$IFDEF TRACEDEBUG} on E : Exception do AddDebugEntry('SMTP Send Exception : '+E.Message); {$ENDIF} end; finally smtp.Disconnect; end; finally smtp.Free; end; finally msg.Free; end; end;

Did you, by chance, set the grid to Enabled=false, or place it on a parent container that is set to Enabled=false?

New values are not actually saved in the Registry until the key is closed, which you are not doing until AFTER sending the broadcast. And the TRegistry.LazyWrite property is true by default, making TRegistry.CloseKey() essentially asynchronous, so new values are saved in the background while control returns to your app. It may take a few seconds for the new values to actually get written, flushed to disk, and cached, thus other apps are likely to read old values while processing the broadcast. You should set LazyWrite=false to ensure the new value is fully saved when closing the key, and you should close the key before sending the broadcast. Try this: function SetGlobalEnvironment(const Name, Value: string; const User: Boolean = True): Boolean; const REG_MACHINE_LOCATION = 'System\CurrentControlSet\Control\Session Manager\Environment'; REG_USER_LOCATION = 'Environment'; var dwReturnValue: DWORD_PTR; begin with TRegistry.Create do try LazyWrite := false; if User then begin { User Environment Variable } RootKey := HKEY_CURRENT_USER; Result := OpenKey(REG_USER_LOCATION, True); end else begin { System Environment Variable } RootKey := HKEY_LOCAL_MACHINE; Result := OpenKey(REG_MACHINE_LOCATION, True); end; if not Result then Exit; WriteString(Name, Value); { Write Registry for Global Environment } finally Free; end; { Update Current Process Environment Variable } SetEnvironmentVariable(PChar(Name), PChar(Value)); { Send Message To All Top Windows for Refresh } //SendMessage(HWND_BROADCAST, WM_SETTINGCHANGE, 0, LPARAM(PChar('Environment'))); SendMessageTimeout(HWND_BROADCAST, WM_SETTINGCHANGE, 0, LPARAM(PChar('Environment')), SMTO_ABORTIFHUNG, 5000, @dwReturnValue); end;

No. Well, now I'm confused. You asked about SSH, then said you don't need SFTP (FTP over SSH). So which do you actually need - SFTP (which Indy does not support) or FTPS (which Indy does support)?

Is "HKCU\DbiW\DBiW\TJGrid" the actual path to your subkey? Your screenshot does not show the whole tree. It is very unusual to see a user-defined key that is not located under one of the system-provided keys, such as "HKCU\Software" (where they usually belong). TRegistry.DeleteKey() returns a boolean to indicate success, which you are not checking. If it returns False, check the TRegistry.LastError and/or TRegistry.LastErrorMsg properties, they tell you why if failed. Despite what the documentation claims, TRegistry.DeleteKey() actually handles this internally for you. It runs a recursive loop to delete all subkeys before then deleting the requested key. You don't have to delete them manually (though you probably should so you can perform error handling on each one). Otherwise, use the Win32 RegDeleteTree() or SHDeleteKey() function instead and let the OS delete all of the subkeys for you.

Which provider are you using? What does your code look like? Without seeing the actual HTTP requests, we can only speculate what the problem is, but most likely either the requests are malformed in some way the server does not like, or are not carrying required info the server needs, or the server is simply encountering an error on its end. Do you mean 500 instead of 5000? HTTP status code 500 is a server-side error, likely caused by you sending bad data to the server and making it crash. Did you ask them to do any debugging on their end, to look at the requests you are sending, and watch how their server reacts? Maybe the provider found a problem on their end and fixed it? You will have to ask them. There is not enough info to diagnose the root problem, especially if the error is not happening anymore.

Because it has a visual UI: How does Task Manager categorize processes as App, Background Process, or Windows Process? It is exactly because it is a VCL Win32 project that it gets categorized as an App. Because it is doing something internally that is preventing it from processing window messages in a timely manner. You need to debug your app and figure out what that something is. Task Manager is merely reacting to the fact that your app is no longer processing messages. It is not the one suspending your app, a bug in your app is. You need to find the bug and fix it. Exactly. So figure out why the message queue is getting blocked in your code.

Yup, see A Key’s Odyssey for more details about that.

I often wonder that myself. Apparently saving a JSON entity to a string in memory and not do anything else with it is a different operation then saving it to its actual JSON formatted form for storage/transmission.

I do, but if I told you then I'd have to <expletive> you

Random crashes like this usually tend to imply that there is possible memory corruption at play. What does Processor_analyze() actually do internally (if you have its source code), and are you sure you have declared it correctly (if importing it from a DLL)?

A wizard index is not a package index, though. AFAIK, there is no way to enumerate the installed wizards and query them for their package info.

That's fine if you control both apps. But what if you want to use a hotkey that is used by another app that you don't control? Then you are SOL. This is why apps are supposed to pick unique hotkeys for themselves, or allow the user to configure the hotkeys (see the VCL's THotKey component and TextToShortCut() function).

If you are referring to the Win32 RegisterHotKey() function and WM_HOTKEY window message, then such hotkeys are system-wide and any given hotkey can only be registered for one window at a time. To use the same hotkey with multiple processes at the same time, you will have to resort to using a global keyboard hook and handle the keystrokes manually.

Yes, it does, and should. The logic to detect when data arrives should not block, but once data is detected then it does block until the complete message has been received. If you don't want it to block at all, then you need to maintain your own cache for the raw bytes, and then parse those bytes yourself after each read. var Cache: TMemoryStream; procedure TForm2.FormCreate(Sender: TObject); begin Cache := TMemoryStream.Create; end; procedure TForm2.FormDestroy(Sender: TObject); begin Cache.Free; end; procedure TForm2.Timer1Timer(Sender: TObject); begin Timer1.Enabled := False; try with IdTCPClient1 do begin if not Connected then Exit(); // read any data in if IOHandler.InputBufferIsEmpty then begin IOHandler.CheckForDataOnSource(0); IOHandler.CheckForDisconnect; if IOHandler.InputBufferIsEmpty then Exit(); end; Cache.Position := Cache.Size; IOHandler.InputBuffer.ExtractToStream(Cache); // parse the Cache.Memory looking for complete JSON messages as needed... end; finally if IdTCPClient1.Connected then Timer1.Enabled := True; end; end; Or, simply move the reading logic into a worker thread and let that thread block as needed, as you said.

Your timer code is not reading the data correctly. The Connected() method performs a read operation, so it is likely to receive bytes you are expecting, thus InputBufferIsEmpty() does not return False and you then skip calling AllData(). Which is also the wrong reading method to use, as it reads until the connection is disconnected and then returns what was read. That is not what you want in this situation. The data you are looking for is JSON formatted data, so you should read based on the JSON format. In this particular case, the greeting and WATCH replies are both JSON objects with no nested sub-objects, so it would be good enough to simply read from the starting curly brace to the ending curly brace once you detect data arriving, eg: procedure TForm2.Timer1Timer(Sender: TObject); var ReceivedText: string; begin Timer1.Enabled := False; try with IdTCPClient1 do begin if not Connected then Exit(); // read any data in if IOHandler.InputBufferIsEmpty then begin IOHandler.CheckForDataOnSource(0); IOHandler.CheckForDisconnect; if IOHandler.InputBufferIsEmpty then Exit(); end; IOHandler.WaitFor('{', False); ReceivedText := IOHandler.WaitFor('}', True, True, IndyTextEncoding_UTF8); Memo1.Lines.Add(ReceivedText); // if not already, send streaming command if not SentStreamCommand then begin IdTCPClient1.IOHandler.WriteLn('?WATCH={"enable":true,"json":true}'); SentStreamCommand := True; end; end; finally if IdTCPClient1.Connected then Timer1.Enabled := True; end; end; Though, this really isn't the best way to handle this situation. Knowing that the server always sends a greeting banner, and each request sends a reply, I would simply get rid of the timer altogether and do a blocking read immediately after connecting, and after sending each request. And move the logic into a worker thread so the UI thread is not blocked. But, if you must use the UI thread, a better option would be to find a JSON parser that supports a push model, then you can push the raw bytes you read from the socket connection into the parser and let it notify you via callbacks whenever it has parsed complete values for you to process. Not all replies in the GPSD protocol are simple objects. Some replies can be quite complex, more than the code above can handle. For instance, the reply to a POLL request contains an array of sub-objects. An even better option is to use a pre-existing GPSD library (there are C based libraries available for GPSD, and C libraries can be used in Delphi) and let the library handle these kind of details for you.

They are aware of it. Just as they have already fixed other access issues related to helpers, it is likely only a matter of time before they do fix this one too.

Have you tried toJSON() instead of toString()? procedure TForm1.Button1Click(Sender: TObject); const KEK = '"value\with' + sLineBreak + '{strange}chars/"'; Var s: TJSONString; begin s := TJSONString.Create(KEK); Try Edit1.Text := s.ToJSON; Finally s.Free; End; end; Also, which version of Delphi are you using? Recent versions have made updates to the JSON framework to make it more compliant with the JSON spec.