Automatic fill & submit web forms using TWebBrowser on Android

[Yaron 53]

I wrote a web-facing application that uses a combination of javascript/css/html for the front-end and Delphi for the back-end.

I decided to create a simple Android app that would wrap the web application using a TWebBrowser control that automatically navigates to the web application's URL.

 

The web application's requires a login and once logged in, a token is generated sever-side and then embedded into the html of subsequent server requests for future authentication.

 

For security reasons, I do not want to store the code used to create the token within the Android app, so the easiest way for me to perform the login would be to automatically fill in the web form fields and simulate a "submit" action.

 

Searching the web, I only found examples of how to do this using the WinAPI version of TWebBrowser, but I'm trying to write something that will work on Android and possibly iOS, any ideas?

[Remy Lebeau 1396]

1 hour ago, Yaron said:

the easiest way for me to perform the login would be to automatically fill in the web form fields and simulate a "submit" action.

A better solution would be to forgo the TWebBrowser frontend altogether and redesign your web app to accept REST requests on the backend server to do the work you need.  Then use a standard UI on the client side to display the results as needed.

1 hour ago, Yaron said:

Searching the web, I only found examples of how to do this using the WinAPI version of TWebBrowser, but I'm trying to write something that will work on Android and possibly iOS, any ideas?

You would have to check if the underlying platform browsers provide any APIs to automate them in code.  VCL's TWebBrowser is a wrapper for Internet Explorer and exposes access to its DOM API.  FMX's TWebBrowser does not expose similar functionality.

[Yaron 53]

21 minutes ago, Remy Lebeau said:

A better solution would be to forgo the TWebBrowser frontend altogether and redesign your web app to accept REST requests on the backend server to do the work you need.  Then use a standard UI on the client side to display the results as needed.

You would have to check if the underlying platform browsers provide any APIs to automate them in code.  VCL's TWebBrowser is a wrapper for Internet Explorer and exposes access to its DOM API.  FMX's TWebBrowser does not expose similar functionality.

The Delphi back-end is already rest-based, but the whole point of using HTML/JS for the front-end is to avoid having to design the UI for each platform (web/desktop/android) and rely on CSS to style the UI for every device without having to deal with device resolution/scale within Delphi code.

 

[Guest]

Should not you use ReacjNative then instead of FireMonkey?

14 minutes ago, Yaron said:

The Delphi back-end is already rest-based, but the whole point of using HTML/JS for the front-end is to avoid having to design the UI for each platform (web/desktop/android) and rely on CSS to style the UI for every device without having to deal with device resolution/scale within Delphi code.

 

 

[Yaron 53]

7 minutes ago, TomDevOps said:

Should not you use ReacjNative then instead of FireMonkey?

 

I'm not sure I understand you, do you mean to write the Android app in ReactNative instead of Delphi?

I don't have any experience with ReactNative, that's why I'm trying to use Delphi to simply wrap the html generated by my back-end code in a TWebBrowser component.

 

[Guest]

8 minutes ago, Yaron said:

I'm not sure I understand you, do you mean to write the Android app in ReactNative instead of Delphi?

I don't have any experience with ReactNative, that's why I'm trying to use Delphi to simply wrap the html generated by my back-end code in a TWebBrowser component.

 

I think I know what you want to achieve, I understand that you want to have HTML/CSS/JS to avoid designing for each platform, so make everything in React, then your web-app and mobile-app can share HTML/CSS/JS stuff 🙂 

 

If React is not an option, then use REST for communication and WebView for displaying the results only, this is what I did for one of the app.

Edited October 15, 2019 by Guest

[Yaron 53]

3 minutes ago, TomDevOps said:

I think I know what you want to achieve, I understand that you want to have HTML/CSS/JS to avoid designing for each platform, so make everything in React, then your web-app and mobile-app can share HTML/CSS/JS stuff 🙂 

 

If React is not an option, then use REST for communication and WebView for displaying the results only, this is what I did for one of the app.

I have already written a Delphi ISAPI DLL that hooks into the microsoft IIS server and outputs HTML/JS/CSS.  That part is done and working well in all browsers.

 

Now, for convenience  I would like to create a separate Android app (using Delphi) that uses the TWebBrowser component to open the web server's URL and automatically log-in by filling in the html form fields (user/pass) and press the "submit" button.

With this second part I'm struggling since I don't know how to get to the HTML content displayed by the TWebBrowser component and how to trigger the form's "submit" button.

[Guest]

1 hour ago, Yaron said:

I have already written a Delphi ISAPI DLL that hooks into the microsoft IIS server and outputs HTML/JS/CSS.  That part is done and working well in all browsers.

 

Now, for convenience  I would like to create a separate Android app (using Delphi) that uses the TWebBrowser component to open the web server's URL and automatically log-in by filling in the html form fields (user/pass) and press the "submit" button.

With this second part I'm struggling since I don't know how to get to the HTML content displayed by the TWebBrowser component and how to trigger the form's "submit" button.

Now I don't get this - why do you want to login this way? Why you do not want to use REST and login that way? I do not know if WebView allow to execute your own JavaScript - if so, you could pass credentials to the form this way and then call submit, but again, I would not do this way.

 

I try to understand your architecture and design choices, can you elaborate, please?

Edited October 15, 2019 by Guest

[Remy Lebeau 1396]

1 hour ago, Yaron said:

Now, for convenience  I would like to create a separate Android app (using Delphi) that uses the TWebBrowser component to open the web server's URL and automatically log-in by filling in the html form fields (user/pass) and press the "submit" button.

Sadly, you won't be able to do that with FMX's TWebBrowser.  It simply doesn't have the necessary functionality for that (only VCL's TWebBrowser does).  If you really want to use FMX's TWebBrowser, then I suggest using TIdHTTP or other HTTP client for the actual HTTP communications, that way you can automate it however you want, and just use TWebBrowser to display the HTML (or whatever) you retrieve, such as via its LoadFromStrings() method.  And then use TWebBrowser events to block the browser from making HTTP requests, redirecting them to your HTTP client of choice.  The experience won't be as seemless as a standard browser, but it will give you more control.

 

[Yaron 53]

15 minutes ago, TomDevOps said:

Now I don't get this - why do you want to login this way? Why you do not want to use REST and login that way? I do not know if WebView allow to execute your own JavaScript - if so, you could pass credentials to the form this way and then call submit, but again, I would not do this way.

 

I try to understand your architecture and design choices, can you elaborate, please?

 

I believe I might be able login by using LoadFromStrings() to feed the html form and auto-activating JS to TWebBrowser with the form's action set to my login url, which will bypass the need to actually modify the html content.

[pcplayer99 11]

I remember Fmx TWebBrowser can be called javascript by delphi outside the TWebBrowser.

[Yaron 53]

I eventually created an auto-executing HTML form and used TWebBrowser.LoadFromStrings() to load the HTML and automatically login.

 

I even made the login 'pretty' by having the form hidden and showing a logo with a CSS based "processing" animation.

 

The end result looks smooth & professional.

Edited October 17, 2019 by Yaron

[toolfolks 0]

Hi

Delphi 11 FMX TWebbrowser.
How do I do the following:
Webbrowser1.naviagte('http://www.visitthewebiste.com') // app parameters to fill in the input id toolSearchField
then click the sratchet button (btn-search)
HTML Elements are :
<input id="toolSearchField" type="text" class="form-control toolSearchField" placeholder="What are you searching for?" onkeyup="searchAsYouType(event)">

 

<a class="btn-search" onclick="searchAsYouType(event)"><img class="btn-search-img" alt="Search" title="Search" src="https://img.ffx.co.uk/website2/icons/search_icon.png"></a>

 

is this possible?