thread-safe ways to call REST APIs in parallel

[David Schwartz 426]

I have a list of REST API queries going to the same service and I want to be able to run them all in parallel. I've been told there shouldn't be any problem running up to 100 or so in parallel on this service.

 

Are the Delphi TRESTClient / TRESTRequest / TRESTResponse components thread-safe?

 

If so, can I create just one TRESTClient and then multiple Request/Response pairs linked to it? Or do I need duplicate Client objects as well?

 

If not, is there something better to use?

 

I'm guessing I'd run each synchronously in its own thread and wait until they all returned to continue, because the TRESTRequest does not appear to have an async option, right?

 

If the data collected in each thread is in a TStringlist, then is it safe to save it to disk using a unique name with sl.SaveToFile(...)? (There's no contention for the files until after they've all completed.)

[Uwe Raabe 2057]

53 minutes ago, David Schwartz said:

can I create just one TRESTClient and then multiple Request/Response pairs linked to it? Or do I need duplicate Client objects as well?

You have to create an isolated set of all components for each thread. 

[Dalija Prasnikar 1396]

56 minutes ago, David Schwartz said:

Are the Delphi TRESTClient / TRESTRequest / TRESTResponse components thread-safe?

 

If so, can I create just one TRESTClient and then multiple Request/Response pairs linked to it? Or do I need duplicate Client objects as well?

 

None of those components are not thread-safe. They can be used in background threads, but each thread needs to have their own instances. Yes, you need separate client objects for each thread, too. 

 

56 minutes ago, David Schwartz said:

If the data collected in each thread is in a TStringlist, then is it safe to save it to disk using a unique name with sl.SaveToFile(...)? (There's no contention for the files until after they've all completed.)

If the StringList is not shared and you generating unique file name is coordinated between threads, so there cannot be any overlap, then it is safe to save inside thread.

[Geoffrey Smith 45]

Check out the TRESTRequest.ExecuteAsync function.

[David Schwartz 426]

3 hours ago, Geoffrey Smith said:

Check out the TRESTRequest.ExecuteAsync function.

Thanks, I had not noticed that for some reason.

[Uwe Raabe 2057]

Be aware that this also needs a dedicated TRestClient for exclusive use in the internal thread. Also make sure that the TRestRequest events are either thread-safe or the SynchronizeEvents property is set to true.

 

For create the TRestRequest dynamically for each case and create the TRestClient with the request as owner. That way the client is automatically freed with the request.

[David Schwartz 426]

On 8/22/2021 at 12:33 AM, Uwe Raabe said:

Be aware that this also needs a dedicated TRestClient for exclusive use in the internal thread. Also make sure that the TRestRequest events are either thread-safe or the SynchronizeEvents property is set to true.

 

For create the TRestRequest dynamically for each case and create the TRestClient with the request as owner. That way the client is automatically freed with the request.

Everybody is describing eating a different piece of this elephant, making different assumptions about what's being used.

 

Doesn't anybody have a more or less complete code example available?

 

I like the ExecuteAsync method, which seems to solve a lot of the sync issues.

I've got a few dozen API calls to the same endpoint with different parameters (similar to the same SQL query with different parameters). I want to run them all in parallel, then extract some data from each of the returned JSON packets, put that data into a fairly simple (newly-created) object, and stick the object onto the Data property of a TListItem in a ListView while updating the line to show its status.

 

There are virtually no examples of this sort of thing I can find anywhere (with or without the UI updates).  Just stuff that increments an integer 100k times.

 

Doesn't anybody do anything useful with multi-threading REST API calls besides incrementing integers?

 

 

[Dalija Prasnikar 1396]

3 hours ago, David Schwartz said:

Doesn't anybody do anything useful with multi-threading REST API calls besides incrementing integers?

I don't understand what is your problem. You asked whether those classes are safe, we answered they are not. Every thread needs to have dedicated object instance for each one - client, request, response. 

 

Even if you use ExecuteAsync, you cannot share that instance. 

 

So basic example would be 

 

for i := 0 to Data.Count-1 do
  begin
    RunRest(Data[i]);
  end;

procedure RunRest(const s: string);
begin
  TTask.Run(
    procedure
    var Client, Response, Request... 
    begin
      Client := TRESTClient.Create(s);
      try
        Response := TRESTResponse.Create(Client);
        Request := TRESTRequest.Create(Client);
        // other rest setup...

        Request.Execute;
        // process Response, if needed you can do that in Synchronize call - if you are interacting with GUI then you definitely need to Synchronize here
     finally
       Client.Free;
     end;
    end); 
end;

 

 

[Uwe Raabe 2057]

4 hours ago, David Schwartz said:

Doesn't anybody do anything useful with multi-threading REST API calls besides incrementing integers?

Of course I do, but that cannot be fully described in a few sentences of an answer to your question.

 

F.i. it took more than two hours to explain only the basics to a co-worker and we had to make a cut as it was too much information to grasp all at once. After all I developed that in several weeks (if not months) to get all things running fast and stable.

 

I also don't know if what is working for us will work for your use case.

 

My plans are to do a more detailed blog post on this, but as that is going to target a broader audience the examples given will probably fall in the incrementing integers category, too. 

[Guest]

I am going to suggest something completely different here. Even though you may have "corporate decisions" or some such to use certain libraries, some 3rd party libraries has these thing pinned down with examples and demos and stuff. In order to learn the "advanced basics" of concepts a demo or even a one-dev-one-year licence can take you a long way IMHO.

 

Two tricky concepts and two 3rd parties with the "umpf" to educate properly (not just with "Hello world" demos):

 

http and multithreading: real thin client, https://rtc.teppi.net/

Firebird/full blown RDBMS advanced transaction handling: http://www.ibobjects.com/.

 

HTH

[David Schwartz 426]

3 hours ago, Dalija Prasnikar said:

I don't understand what is your problem. You asked whether those classes are safe, we answered they are not. Every thread needs to have dedicated object instance for each one - client, request, response. 

 

I understand from the first reply that the three objects need to be created dynamically inside the task. That's fine.

 

A different post about this general topic suggested a fork/join approach using a counter object to know when it's finished. That's fine, too, although it may be more complicated than I need.

 

Another reply here pointed out ExecuteAsync which I had not noticed (although I looked for Async...something), and it has some very interesting properties that also suggest the fork/join approach might have a simpler solution with it. In particular, I like the options that the anonymous methods offer in conjunction with the parameters that say whether the code should run in the main form's thread or the current thread.

 

But every other post here added a warning about this or that, and it got confusing with the Execute vs. ExecuteAsync. The entire discussion seems like it could have been reduced to one or two short code examples like the above, so thank you Dalija.

 

(The whole discussion on fork/join has quite a bit of code in the referenced article, although it seems to me it shouldn't be that complicated. Be that as it may, adding the logic needed to perform the REST queries in the task shouldn't add very much complexity.)

 

I looked on SO for examples as well. It's hard to search for help there (or anywhere) on obscure topics. The few I found had lots of comments back and forth debating this and that, and not much code.

 

(I'd post something on SO, but it seems a lot of my questions get downvoted or locked for being "off-topic". They're Delphi questions -- I don't get what's "off-topic" about Delphi questions in a forum intended for Delphi questions. Now I get warnings from SO saying my account is close to being locked because of the "low-quality" of my questions. I used to have nearly 2000 points and 45 badges before something short-circuited and SO's support people denied there was a problem. All of that "bling" just disappeared. So I don't spend much time there any more. I'm also very hesitant to post any more questions. The last one I posted got lots of opinions, and no definitive answers.)

 

OT:

 

There have been times in my life when I worked with a team of other developers. For the past 12 years or so, I've been either the only Delphi dev or had a junior person working with me with only 3-5 years of experience. Also, I'm a fairly high-level abstract thinker and most of the folks I work with are heavily left-brained highly detail-oriented people who can't handle abstract discussions. They want concrete examples for everything, which I find challenging.

 

My only resources for help have been this place, SO, and Google. I truly do appreciate all of the help that's available here, and it's mostly because I value the level of mastery everybody here has. I know I can be a PITA at times because I often have very clear pictures in my head of what I'm looking for, I just have trouble expressing things clearly. We programmers are often no better at describing things in our heads than our clients, eh? 🙂

 

Unfortuntely, my memory recall is getting a little flakey, and complex topics I used to have no problem dealing with seem to show up in my head with pieces missing. But I still do love the process of programming -- it has always been like playing word games to me, and it's something I can do for hours and hours and never really get tired of doing. I'm sure most of you can relate ... how many people can say that about the things they do to earn a living? We're blessed in that respect.

 

I only have one compliaint about what we do: we spend way too much time describing and explaining, in text, what it is we're talking about. I mean ... that's all programming is at its core: we have an idea in our head, we conjur up a recipe in some arcane language (Delphi in this case) and we use that to make the computer sing, dance, and jump through hoops. If we want to see a circle on the screen, we describe a circle. If we want to launch a rocket, we describe all of the steps that all of the different parts need to accomplish that. As annoying as I find reverse-engineering code to understand what it's doing (since most code is undocumented and what's there is usually outdated), sometimes the answer to a question is better given as code (or pseudo-code) than words describing what code needs to be written. But overall, I wish there was a more abstract way of dealing with things other than, say, English words explaining what's needed in a translation.

Edited August 26, 2021 by David Schwartz

[ertank 27]

Not a direct reply to the question. But, if I ever need to run parallel requests, I should be really thinking about building a custom thread class or two with main thread synchronization where needed.

[David Schwartz 426]

2 hours ago, Dany Marmur said:

I am going to suggest something completely different here. Even though you may have "corporate decisions" or some such to use certain libraries, some 3rd party libraries has these thing pinned down with examples and demos and stuff. In order to learn the "advanced basics" of concepts a demo or even a one-dev-one-year licence can take you a long way IMHO.

 

I have no problem paying for libraries. I am aware that some come with better examples than others. I'm also aware that there are plenty of things in the Delphi help system that are completely undocumented, even though they have placeholder pages for them on the wiki or the help system. We all continue to pray that our steadily increasing annual "maintenance" fees will eventually be used to pay someone to fill in these gaps instead of just creating more. That IS what "maintenance" is all about, right?

It's also annoying when there is documentation that only explains WHAT parameters are needed and leaves it to the reader to figure out HOW this mess of stuff is supposed to be used. I've found that to be fairly common with the different classes and whatnot involved with this particular effort.

 

Why should one need to buy a 3rd-party library to figure out how to code something that Delphi has plenty of available mechanisms to work with, but they're just not documented anywhere?

 

There are no examples shown in Delphi's help for TRESTRequest.ExecuteAsync for example, even though there are help pages there. 

[Fr0sT.Brutal 900]

May I conclude to move this floody topic to more precise rails?

1. You have to use one client object per simult request.

2. You can either use worker threads with blocking requests or async requests in main thread.

3. Ehm... that's all?

Edited August 26, 2021 by Fr0sT.Brutal

[Dalija Prasnikar 1396]

1 hour ago, David Schwartz said:

I understand from the first reply that the three objects need to be created dynamically inside the task. That's fine.

 

A different post about this general topic suggested a fork/join approach using a counter object to know when it's finished. That's fine, too, although it may be more complicated than I need.

 

Creating separate objects would be my preferred approach, they are not that heavy to have significant impact on performance.

 

1 hour ago, David Schwartz said:

Another reply here pointed out ExecuteAsync which I had not noticed (although I looked for Async...something), and it has some very interesting properties that also suggest the fork/join approach might have a simpler solution with it. In particular, I like the options that the anonymous methods offer in conjunction with the parameters that say whether the code should run in the main form's thread or the current thread.

 

ExecuteAsync is .... I have no words for describing it... maybe some people would like it, but for me it is useless addition. If you want to run rest it in the background thread simplest code is to just run the whole thing in background thread and then synchronize only if you need to synchronize some piece of code. With Execute you get clean code under your control where it is obvious what runs where and how you need to handle things, with ExecuteAsync you first need to learn how to use it properly. I would forget about it as it does not offer any advantages.

 

1 hour ago, David Schwartz said:

But every other post here added a warning about this or that, and it got confusing with the Execute vs. ExecuteAsync. The entire discussion seems like it could have been reduced to one or two short code examples like the above, so thank you Dalija.

 

I am glad if it helped to give you some  insight, because I was confused about what you are aiming at.

 

1 hour ago, David Schwartz said:

(I'd post something on SO, but it seems a lot of my questions get downvoted or locked for being "off-topic". They're Delphi questions -- I don't get what's "off-topic" about Delphi questions in a forum intended for Delphi questions. Now I get warnings from SO saying my account is close to being locked because of the "low-quality" of my questions. I used to have nearly 2000 points and 45 badges before something short-circuited and SO's support people denied there was a problem. All of that "bling" just disappeared. So I don't spend much time there any more. I'm also very hesitant to post any more questions. The last one I posted got lots of opinions, and no definitive answers.)

 

You can ask only very specific questions on Stack Overflow. For instance your initial question here would be definitely off topic on Stack Overflow. If you can create some minimal code example something in line what I have posted here as example, then you can aske whether you are doing it correctly and whether it is thread safe.  It is also good to point to specific parts of the code you are not sure about. When it comes to thread safety it depends on the actual code and sometimes people don't add enough code. On the other hand adding too much code is also a problem because it is harder to get clear picture id there is plenty of code. For instance, if you need to populate UI, you don't need to show 50+ lines of UI related code. One is enough. 

 

The quality of answers also very much depends on the quality of the question. If answering requires a lot of guessing you will less likely get good answers. 

1 hour ago, David Schwartz said:

My only resources for help have been this place, SO, and Google. I truly do appreciate all of the help that's available here, and it's mostly because I value the level of mastery everybody here has. I know I can be a PITA at times because I often have very clear pictures in my head of what I'm looking for, I just have trouble expressing things clearly. We programmers are often no better at describing things in our heads than our clients, eh? 🙂

 

Few lines of code speak better than words 🙂 Don't describe your code, write it - even if it is just a rough concept. Then build the rest of the explanation on top of that.

[Vincent Parrett 750]

This sort of task looks like something that OmniThreadLibrary would handle - but does require some study to get right (worthwhile imho). I created a wrapper over it's async/await feature that might be useful

 

https://github.com/VSoftTechnologies/VSoft.Awaitable (depends on https://github.com/VSoftTechnologies/VSoft.CancellationToken and omnithread) 

 

An extremly naive example (with no queueing or thread limiting)  - will probably not compile as mostly hand typed/copypasta 😉 with some details omitted for brevity.

 

var
  RequestsInFlight : integer = 0; 
  CancelTokenSource : ICancellationTokenSource;


procedure DoRequest(const cancelToken : ICancellationToken; const param1 : string)
var
  LParam1 : string;
begin
  LParam1 := param1; //local for capture
  Inc(RequestsInFlight);
  TAsync.Configure<string>(
        function (const cancelToken : ICancellationToken) : string
		var
	      restclient : TWhateverclient;
        begin
		  //run rest request here
		  //check cancelToken.IsCancelled or use the cancelToken.Handle with waitformultipleobjects etc
		  restclient := TWhateverclient.Create;
		  try
			result := restclient.Execute();
		  finally
			restclient.free;
		  end
		  
        end, token);
    )
    .OnException(
        procedure (const e : Exception)
        begin
        //log error
        end)
    .OnCancellation(
        procedure
        begin
            //clean up
        end)
    .Await(
        procedure (const value : string)
        begin
		  Dec(RequestsInFlight);
          //use result - runs in the calling thread.
        end);
end;


procedure StartRequests(const cancelToken : ICancellationToken)
begin
//Start the requests.
 for i := 0 to RequestCount - do
   ExecuteRequest(cancelToken, RequestParams[i]);
  //monitor requests inflight to know when it's done. 
end;

procedure StopRequests;
begin
  CancelTokenSource.Cancel;
end;

 

The cancellation part requires that your rest client supports cancelling requests somehow.

 

Hope that helps. 

 

Edited August 26, 2021 by Vincent Parrett
typo

[David Schwartz 426]

On 8/26/2021 at 3:50 PM, Vincent Parrett said:

This sort of task looks like something that OmniThreadLibrary would handle - but does require some study to get right (worthwhile imho). I created a wrapper over it's async/await feature that might be useful

 

https://github.com/VSoftTechnologies/VSoft.Awaitable (depends on https://github.com/VSoftTechnologies/VSoft.CancellationToken and omnithread) 

 

I notice that OmniThreadLibrary 3.07.9 (10/26/2021) has added this:

 

[HHasenack] Added Cancel and IsCancelled to IOmniParallelTask.