TIdTCPClient - gpsd communication

[ertank 28]

Hello,

 

I am using Delphi 10.3.2, Indy version: 10.6.2.5366 (default coming with Delphi).

 

There is a gpsd daemon running on a Raspberry Pi. It is broadcasting some json strings over TCP:2947

- I can successfully establish a connection using TIdTCPClient. There is no data incoming after connection.

- Send command to stream my client these json strings after connection. There is no data incoming after sending command.

Memo output:

Memo1
Connecting to 192.168.1.90.
Connected.

 

On the other hand;

- Using Putty, I instantly get initial greeting json string right after connection without sending anything.

- if I send command to stream I instantly get json string replies.

 

Putty terminal output:

{"class":"VERSION","release":"3.17","rev":"3.17","proto_major":3,"proto_minor":12}
?WATCH={"enable":true,"json":true}
{"class":"DEVICES","devices":[{"class":"DEVICE","path":"/dev/ttyS0","activated":"2019-08-31T15:56:43.607Z","native":0,"bps":9600,"parity":"N","stopbits":1,"cycle":1.00}]}
{"class":"WATCH","enable":true,"json":true,"nmea":false,"raw":0,"scaled":false,"timing":false,"split24":false,"pps":false}

 

My current test code:

unit uMain;

interface

uses
  Winapi.Windows,
  Winapi.Messages,
  System.SysUtils,
  System.Variants,
  System.Classes,
  Vcl.Graphics,
  Vcl.Controls,
  Vcl.Forms,
  Vcl.Dialogs,
  IdBaseComponent,
  IdComponent,
  IdTCPConnection,
  IdTCPClient,
  Vcl.ExtCtrls,
  Vcl.StdCtrls;

type
  TForm2 = class(TForm)
    IdTCPClient1: TIdTCPClient;
    Memo1: TMemo;
    Timer1: TTimer;
    Button1: TButton;
    procedure IdTCPClient1Connected(Sender: TObject);
    procedure IdTCPClient1Disconnected(Sender: TObject);
    procedure IdTCPClient1Status(ASender: TObject; const AStatus: TIdStatus;
      const AStatusText: string);
    procedure Timer1Timer(Sender: TObject);
    procedure Button1Click(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
  private
    { Private declarations }
    SentStreamCommand: Boolean;
  public
    { Public declarations }
  end;

var
  Form2: TForm2;

implementation

{$R *.dfm}

procedure TForm2.Button1Click(Sender: TObject);
begin
  if Button1.Tag <> 0 then
  begin
    IdTCPClient1.Disconnect();
    Button1.Caption := 'Connect';
    Button1.Tag := 0;
  end
  else
  begin
    IdTCPClient1.Connect('192.168.1.90', 2947);
    Button1.Caption := 'Disconnect';
    Button1.Tag := 1;
  end;
end;

procedure TForm2.FormCreate(Sender: TObject);
begin
  Timer1.Enabled := False;
  SentStreamCommand := False;
end;

procedure TForm2.FormDestroy(Sender: TObject);
begin
  if IdTCPClient1.Connected then IdTCPClient1.Disconnect(False);
end;

procedure TForm2.IdTCPClient1Connected(Sender: TObject);
begin
  Timer1.Enabled := True;
end;

procedure TForm2.IdTCPClient1Disconnected(Sender: TObject);
begin
  Timer1.Enabled := False;
end;

procedure TForm2.IdTCPClient1Status(ASender: TObject; const AStatus: TIdStatus;
  const AStatusText: string);
begin
  Memo1.Lines.Add(AStatusText);
end;

procedure TForm2.Timer1Timer(Sender: TObject);
var
  ReceivedText: string;
begin
  Timer1.Enabled := False;
  try
    with IdTCPClient1 do
    begin
      if not Connected then Exit();

      // read any data in
      if IOHandler.InputBufferIsEmpty then
      begin
        IOHandler.CheckForDataOnSource(0);
        IOHandler.CheckForDisconnect;
        if IOHandler.InputBufferIsEmpty then Exit();
        ReceivedText := IOHandler.AllData();
        if ReceivedText <> EmptyStr then Memo1.Lines.Add(ReceivedText);
      end;

      // if not already, send streaming command
      if not SentStreamCommand then
      begin
        IdTCPClient1.IOHandler.WriteLn('?WATCH={"enable":true,"json":true}');
        SentStreamCommand := True;
        Exit();
      end;
    end;
  finally
    if IdTCPClient1.Connected then Timer1.Enabled := True;
  end;
end;

end.

 

I would like to understand what I am doing wrong. My main purpose is to read each json string separately as they are incoming one per line.

 

I appreciate any help, please.

 

Thanks & regards,

Ertan

[Remy Lebeau 1428]

Your timer code is not reading the data correctly. The Connected() method performs a read operation, so it is likely to receive bytes you are expecting, thus InputBufferIsEmpty() does not return False and you then skip calling AllData(). Which is also the wrong reading method to use, as it reads until the connection is disconnected and then returns what was read. That is not what you want in this situation.

 

The data you are looking for is JSON formatted data, so you should read based on the JSON format. In this particular case, the greeting and WATCH replies are both JSON objects with no nested sub-objects, so it would be good enough to simply read from the starting curly brace to the ending curly brace once you detect data arriving, eg:

procedure TForm2.Timer1Timer(Sender: TObject);
var
  ReceivedText: string;
begin
  Timer1.Enabled := False;
  try
    with IdTCPClient1 do
    begin
      if not Connected then Exit();
      // read any data in
      if IOHandler.InputBufferIsEmpty then
      begin
        IOHandler.CheckForDataOnSource(0);
        IOHandler.CheckForDisconnect;
        if IOHandler.InputBufferIsEmpty then Exit();
      end;
      IOHandler.WaitFor('{', False);
      ReceivedText := IOHandler.WaitFor('}', True, True, IndyTextEncoding_UTF8);
      Memo1.Lines.Add(ReceivedText);
      // if not already, send streaming command
      if not SentStreamCommand then
      begin
        IdTCPClient1.IOHandler.WriteLn('?WATCH={"enable":true,"json":true}');
        SentStreamCommand := True;
      end;
    end;
  finally
    if IdTCPClient1.Connected then
      Timer1.Enabled := True;
  end;
end;

Though, this really isn't the best way to handle this situation.  Knowing that the server always sends a greeting banner, and each request sends a reply, I would simply get rid of the timer altogether and do a blocking read immediately after connecting, and after sending each request.  And move the logic into a worker thread so the UI thread is not blocked.

 

But, if you must use the UI thread, a better option would be to find a JSON parser that supports a push model, then you can push the raw bytes you read from the socket connection into the parser and let it notify you via callbacks whenever it has parsed complete values for you to process. Not all replies in the GPSD protocol are simple objects. Some replies can be quite complex, more than the code above can handle.  For instance, the reply to a POLL request contains an array of sub-objects.

 

An even better option is to use a pre-existing GPSD library (there are C based libraries available for GPSD, and C libraries can be used in Delphi) and let the library handle these kind of details for you.

Edited September 2, 2019 by Remy Lebeau

[ertank 28]

That code did work. Though, it block my main thread. I just need to modify my logic slightly and put it in a thread and call a procedure once some json is in.

 

I will definitely use some json parser. I simply failed to get data from server to start parsing it.

 

Thank you.

[Remy Lebeau 1428]

12 hours ago, ertank said:

That code did work. Though, it block my main thread.

Yes, it does, and should.  The logic to detect when data arrives should not block, but once data is detected then it does block until the complete message has been received.  If you don't want it to block at all, then you need to maintain your own cache for the raw bytes, and then parse those bytes yourself after each read.

var
  Cache: TMemoryStream;

procedure TForm2.FormCreate(Sender: TObject);
begin
  Cache := TMemoryStream.Create;
end;

procedure TForm2.FormDestroy(Sender: TObject);
begin
  Cache.Free;
end;

procedure TForm2.Timer1Timer(Sender: TObject);
begin
  Timer1.Enabled := False;
  try
    with IdTCPClient1 do
    begin
      if not Connected then Exit();
      // read any data in
      if IOHandler.InputBufferIsEmpty then
      begin
        IOHandler.CheckForDataOnSource(0);
        IOHandler.CheckForDisconnect;
        if IOHandler.InputBufferIsEmpty then Exit();
      end;
      Cache.Position := Cache.Size;
      IOHandler.InputBuffer.ExtractToStream(Cache);
      // parse the Cache.Memory looking for complete JSON messages as needed...
    end;
  finally
    if IdTCPClient1.Connected then
      Timer1.Enabled := True;
  end;
end;

Or, simply move the reading logic  into a worker thread and let that thread block as needed, as you said.

Edited September 2, 2019 by Remy Lebeau