Having trouble with OnRestRequestDone

[w0wbagger 0]

I've been using ICS for more than 15 years but have never used the RESTful or HTTP components (always just FTP), but finally have to use it. I'm trying not to use Indy, but installing and learning how to use ICSin C++ is tough (I don't know Pascal). So apologies if I ask some stupid questions here. There were a *lot* of problems trying to use the v8.7 components (had to comment out a lot of lines from OverbyteIcsWinCrypt.hpp, because the compiler generated pretty bad .hpp files, I guess), but I couldn't get 9.1 to install in C++ Builder, so sticking with the older version for now.

 

I've written code to send an XML stream to an API site, and having issues with the status returned. If I understand how this all works, then when I send my POST, my program should continue doing things until the REST server responds with some message, either "Bad Request", or "Unauthorized", or "200 - everything is good with what you sent" or other documented response.

 

But it seems that what actually happens is that once the data is sent, whether good or bad, RestClientRequestDone is fired immediately with ErrCode = 0, and says it was completed successfully.

 

However, if there was an error, it does the above (says it was successful), but then the catch at the bottom traps the error returned by the API server ("Not Authorized", or "Bad Request"). Do I misunderstand when RestClientRequestDone fires? Does it fire before hearing anything back from the API Server?

If I want to wait until that happens, what event should I be trapping? 

 

Thanks in advance for your help (and your patience!)

 

Here's my pretty simple code:

 

// I believe this should fire after the file/apikey have been sent, and the API server responds

void __fastcall TfrmCAD::RestClientRequestDone(TObject *Sender, THttpRequest RqType,
          WORD ErrCode)
{
    GRequestCompleted = true;
    if (ErrCode != 0) {
        ShowMessage("Error during HTTP request: " + IntToStr(ErrCode));
    } else {
        ShowMessage("Request completed successfully.\nResponse: " + GResponseData);
    }
}

 

//---------------------------------------------------------------------------

// Function to send an XML file to an API using ICS TSslHttpRest
void TfrmCAD::SendXMLFileToAPI(const String &xmlFileName) {
    if (!FileExists(xmlFileName)) {
        ShowMessage("File does not exist: " + xmlFileName);
        return;
    }

    // Prepare the HTTP client
    TSslHttpRest *RESTClient = new TSslHttpRest( this );
    RESTClient->OnRestRequestDone = &RestClientRequestDone;

    RESTClient->URL = "APIURLHIDDEN";

    // Load the XML file into a stream
    TFileStream *fileStream = new TFileStream(xmlFileName, fmOpenRead | fmShareDenyWrite);
    try {
        // Simplified approach, adjust accordingly
        RESTClient->SendStream = fileStream;
        RESTClient->ExtraHeaders->Add("x-api-key: APIKEYREDACTED");
        GRequestCompleted = false;
        RESTClient->Post();

        // Wait for the request to complete
        while (!GRequestCompleted) {
            Application->ProcessMessages();
        }
    } catch (const Exception &e) {
        ShowMessage("Error sending file: " + e.Message);
    }

    // Clean up
    delete RESTClient;
    delete fileStream;
}

 

 

[Angus Robertson 572]

To get started with the ICS REST component, I suggest you build and run the OverbyteIcsSnippets sample, click the 'HTTP REST Json Request' button and see what it does.  You can then copy the doHttpRestReqClick function which is heavily documented for all the hard coded properties.

 

In your code, you say you are using the TSslHttpRest component, but you are not using the RestRequest method which is how all requests are started, you've not looked at our REST samples.

 

BTW, one of the arguments in RestRequest specifies if the component makes sync or async requests, so make a sync request and your processmessages loop is no longer required, nor the RequestDone method.

 

Angus

 

Edited March 22 by Angus Robertson

[w0wbagger 0]

Thank you, Angus. I find it difficult to understand Pascal code, but I did what you said and I now have something that seems to be working. I pretty much directly translated, as best I could, the code from OverbyteIcsHttpRestTst.PAS. I don't think OverbyteIcsSnippets is part of what is shipped with 8.70? (Spent hours last night trying to install 9.1 on C++ builder, but was unable to get it to work).

 

HOWEVER, 

I  see in the OverbyteIcsHttpRestTst.PAS file that you have defined an Async Flag in the call to RestRequest. I have defined a function RESTClientRequestDone, and assigned it to RESTClient->OnRestRequestDone..

If I set Async to false, I get an immediate response to my RestRequest as 202 (file accepted), and then OnRestRequestDone fires and I can get a ton of good information about the request. This is true if I deliberately send the wrong API or an incorrect URL. The ErrCode is correct in that case and it catches the raw error data if something went awry.

If I set Async to true, however, it fails with a Status code of 0, which as I understand it, is indeterminate. OnRestRequestDone still fires, but I get an HTTP 0. In both cases (Async true OR false), RESTClientRequestDone fires, but only in the non-Async version does it give me proper values.

The only difference from "it works!" to "it doesn't work!" is the Async flag that I set in the call to RestRequest (see below). I just can't understand from the Pascal source what I might be doing incorrectly. It seems to work perfectly with Async = false, so I guess I can just run with that, but I would like to understand why Async mode blows up.

 

Thanks for any light you can shed on this.

 

Code follows:

 

void __fastcall TfrmCAD::RESTClientRequestDone(TObject *Sender, THttpRequest RqType, WORD ErrCode)
{
    TSslHttpRest *restClient = dynamic_cast<TSslHttpRest*>(Sender);
    if (restClient == nullptr) return;

    if (ErrCode != 0) {
        memoStatus->Lines->Add("Request failed: Error: " + restClient->RequestDoneErrorStr +
               " - " + IntToStr(restClient->StatusCode) + " " + restClient->ReasonPhrase);
        return;
        }

    memoStatus->Lines->Add("Request done, StatusCode " + IntToStr(restClient->StatusCode));

    if (restClient->ContentType.Pos("text/") > 0 || restClient->ContentType.Pos("xml") > 0 ||
        restClient->ContentType.Pos("json") > 0 || restClient->ContentType.Pos("java") > 0) {
        memoStatus->Lines->Add(String("Raw Response:"+restClient->ResponseRaw));
    } else {
        memoStatus->Lines->Add("<Non-textual content received: " + restClient->ContentType + ">");
    }

    // Clean up
    delete restClient;
}

//---------------------------------------------------------------------------

// Function to send an XML file to an API using ICS TSSlHttpRest
void TfrmCAD::SendXMLFileToAPI(const String &xmlFileName, String& APIURL, String& APIKey) {

    if (!FileExists(xmlFileName)) {
        ShowMessage("File does not exist: " + xmlFileName);
        return;
    }

    // Set up the URL and SSL options
    TSslHttpRest *RESTClient = new TSslHttpRest( this );

    RESTClient->OnRestRequestDone = &RESTClientRequestDone;
    RESTClient->URL = APIURL;
    RESTClient->SslCliSecurity = sslCliSecTls12;
    RESTClient->ContentTypePost = "application/xml";

    // Add the API key to the request headers
    if (!chkSendEmptyHeader->Checked) { // I do this to test that errors are detected and reported correctly - this removes the api-key to test "not authorized".
        RESTClient->ExtraHeaders->Add("x-api-key: "+APIKey);
    }

    // Load the XML file into a string and start the asynchronous POST request

    int StatusCode;

    TFileStream *fileStream = new TFileStream(xmlFileName, fmOpenRead | fmShareDenyWrite);
    try {
        // Simplified approach, adjust accordingly
        RESTClient->SendStream = fileStream;
        StatusCode = RESTClient->RestRequest(httpPOST, APIURL, true, ""); // Works perfectly if Async = false, fails if Async = true;
        memoStatus->Lines->Add("ASync REST Request Started, StatusCode = "+IntToStr(StatusCode));

    } catch (const Exception &e) {
        ShowMessage("Error preparing request: " + e.Message);
    }

    delete fileStream;
}

//---------------------------------------------------------------------------

void __fastcall TfrmCAD::btnSendXMLClick(TObject *Sender)
{
    String xmlFileName = btnLoadXML->Text;
    if (!xmlFileName.IsEmpty()) {
        String apiKey = txtAPIKey->Text;
        String apiURL = txtAPIURL->Text;
        SendXMLFileToAPI(xmlFileName, apiURL, apiKey);
    } else {
        ShowMessage("No file selected.");
    }
}
 

 

 

[Angus Robertson 572]

Although the snippets sample was added after your release, you should be able to built it with V8.70 after removing some new units added with later releases.  Or you download a compiled version of snippets and httpresttst from https://wiki.overbyte.eu/arch/icsdemos-clients.zip to test them.

 

As I said, for sync mode you don't RequestDone event, just check the status code returned by the RestRequest method, the error code in the event is irrelevant. 

 

The one event you should use is OnHttpRestProg and set DebugLevel := DebugHdr, that is the logging event that shows exactly what the component is doing with commands, hesders and error messages, you will see that running either of the samples I mentioned.

 

Angus

 

 

 

[w0wbagger 0]

Thank you, Angus. I'll unzip the 9.1 distribution and take a look at the ICSSnippets there. I thought the preferred method was to use non-blocking Async mode, so I've been trying to do that everywhere I use ICS components. Is there any downside to using Sync mode for a low-volume application?

 

[Angus Robertson 572]

Async components are certainly better for servers and applications making parallel requests, but sync are easier to write not needing to keep track of state, and where only one request is made at a time.  

 

The main issue here is knowing when RequestDone is called, what published component properties need to be checked and for what, and what other requests need to be made, a sync request hides all that from you so is easier to use.  

 

Your original use of the async component was actually sync since you had a wait loop afterwards, which is exactly how the sync component works,

 

Angus

 

[w0wbagger 0]

Understood. I'll take a look at ICSSnippets, which should get me to where I need to be. Thanks for your help. I'll try not to bother you again. FWIW, Chat-GPT is quite helpful in converting Pascal code to C++, for any other C++ programmers trying to make sense of Pascal code. I may even use it to convert some of the new sample code to C++ to include in a future release, if you think that would be helpful (after testing it fully, of course).