Unit testing cross platform code

[Darian Miller 361]

Which tool(s) are you utilizing for unit-testing code to be used across all platforms?  

[Arnaud Bouchez 407]

We use our Open Source https://github.com/synopse/mORMot/blob/master/SynTests.pas unit.

It is cross-platform and cross-compiler (FPC and Delphi).

 

But it is mostly about server-side process and Win32/Win64 for Delphi.
It is cross-platform (Win, BSD, Linux, i386/x86-64/arm32/aarch64) for FPC.

[David Heffernan 2345]

Surely all the widely know unit testing libraries are cross platform. 

[Guest]

DUnit for Delphi, fpcunit for FPC, one source code for the tests as they are largely compatible, you only need to conditionally compile with the right uses clause.

There's an example of that in chakracore-delphi.

[Darian Miller 361]

15 hours ago, Arnaud Bouchez said:

But it is mostly about server-side process and Win32/Win64 for Delphi.
It is cross-platform (Win, BSD, Linux, i386/x86-64/arm32/aarch64) for FPC.

 

Any plans to support more platforms in Delphi?  I don't use FPC at all.

[Darian Miller 361]

5 hours ago, Ondrej Kelle said:

DUnit for Delphi, fpcunit for FPC, one source code for the tests as they are largely compatible, you only need to conditionally compile with the right uses clause.

 

 

From what I understand, DUnit is Win32/Win64/MacOS 32-bit only.  

 

[Wagner Landgraf 43]

DUnit works on all supported Delphi platforms.

[Darian Miller 361]

5 hours ago, David Heffernan said:

Surely all the widely know unit testing libraries are cross platform. 

 

Cross platform, as in Win32/Win64 sure... 

[Darian Miller 361]

Just now, Wagner Landgraf said:

DUnit works on all supported Delphi platforms.

 

Do you use it today on Linux/Android/iOS?  

[Wagner Landgraf 43]

Just now, Darian Miller said:

 

Do you use it today on Linux/Android/iOS?  

Yes.

Edited October 9, 2019 by Wagner Landgraf

[Darian Miller 361]

1 minute ago, Wagner Landgraf said:

Yes.

Is there FMX support for client-side testing?  I must have missed something obvious - wouldn't be the first time.  Can you share a simple test project for all platforms?

[Darian Miller 361]

Also related, DUnitX says 

 

Cross platform currently supporting:

• Win32,Win64 and OSX Compilers.

 

https://github.com/VSoftTechnologies/DUnitX

[Wagner Landgraf 43]

Just now, Darian Miller said:

Is there FMX support for client-side testing?  I must have missed something obvious - wouldn't be the first time.  Can you share a simple test project for all platforms?

The thing here is that you don't have UI for the test runner. You can compile and run all your tests, but to gather information about the results, then you have to use custom output. Well, for Linux you can use the console output. For Android/iOS, I've built a custom one myself where I send test results through TCP to a VCL client in another computer.

But there are great tools out there, I believe TestInsight from @Stefan Glienke might do the job for you: https://bitbucket.org/sglienke/testinsight/wiki/Home.

[Darian Miller 361]

Just now, Wagner Landgraf said:

The thing here is that you don't have UI for the test runner. You can compile and run all your tests, but to gather information about the results, then you have to use custom output. Well, for Linux you can use the console output. For Android/iOS, I've built a custom one myself where I send test results through TCP to a VCL client in another computer.

But there are great tools out there, I believe TestInsight from @Stefan Glienke might do the job for you: https://bitbucket.org/sglienke/testinsight/wiki/Home.

 

Well, that's what I was getting at... full support isn't built-in.

 

Do you test FMX client applications on Android this way?  

 

[Wagner Landgraf 43]

1 hour ago, Darian Miller said:

 

Well, that's what I was getting at... full support isn't built-in.

 

Do you test FMX client applications on Android this way?  

 

I mentioned above, I don't use TestInsight but I use a similar approach which I develop myself for my own use only. That's how I test my code on all platforms.

[David Heffernan 2345]

3 hours ago, Darian Miller said:

Cross platform, as in Win32/Win64 sure... 

Yeah, I know what cross platform means.

 

What would be platform specific about a unit testing library? 

[Dalija Prasnikar 1396]

1 hour ago, David Heffernan said:

Yeah, I know what cross platform means.

 

What would be platform specific about a unit testing library? 

Retrieving the results, memory management, zero based strings... 

[David Heffernan 2345]

1 hour ago, Dalija Prasnikar said:

Retrieving the results, memory management, zero based strings... 

Ugh. How could I have forgotten that Delphi is actually two different languages. 

[Darian Miller 361]

3 hours ago, David Heffernan said:

What would be platform specific about a unit testing library? 

 

There are plenty of differences beyond basic code checks...   File I/O, socket implementation, authentication, database drivers, etc.   Also - many people test GUI interactions with DUnit which is VCL based.  There needs to be a FMX based version.

 

Then you have the whole interaction with running tests on a different system than what you are developing on and providing feedback to the developer.  Then of course you'd want to run all the tests automatically with Jenkins or some other tooling so now the C/I system also needs to build/run/report/monitor tests on different platforms.  You should also be running tests across different versions of each platform.  

 

I assumed that there was a solution for actual cross platform testing for Delphi.  

 

I did find this today:  https://sourceforge.net/projects/dunitfmx/

Project description: "DUnit translation to FMX and Mac OS X"  with an additional note:  "When project finish, it shall work for run unit test in Windows and Mac OS X. And of course for Android and IOS too."

 

But this repo hasn't been updated in 5 years.

 

I started building my own custom tooling and the more I sketched it out, the bigger the project got so I thought I'd pause and post a simple question to see what sort of responses I'd get.

 

[Darian Miller 361]

Related: Jolyon has some recent articles on his blog, and a new SmokeTest 2.0 tool on GitHub: https://github.com/deltics/deltics.smoketest

 

He's apparently currently building a whole new set of tooling for Delphi 7-10.3, including a Nuget for Delphi:  http://www.deltics.co.nz/blog/

[David Heffernan 2345]

9 hours ago, Darian Miller said:

There are plenty of differences beyond basic code checks...   File I/O, socket implementation, authentication, database drivers, etc. 

I don't see how any of that impacts a unit testing library. 

 

9 hours ago, Darian Miller said:

Also - many people test GUI interactions with DUnit which is VCL based.  There needs to be a FMX based version.

   

I wouldn't describe that as unit testing.

 

My take is that the if we had a single language then a well written library would naturally be cross platform. But Dalija is right, there are two languages, ARC and non ARC. 

 

And you are also right about the runners. If you expect GUI then they could require distinct UI. 

Edited October 10, 2019 by David Heffernan

[Fr0sT.Brutal 900]

Hmm, isn't a single code base for all platforms the main goal of cross-platform? I don't realize what's the problem in using testing framework on several platforms. Of course not talking about GUI runners but any other methods of test framework shouldn't care about platform at all.

Edited October 10, 2019 by Fr0sT.Brutal

[ByteJuggler 45]

1 hour ago, Fr0sT.Brutal said:

Hmm, isn't a single code base for all platforms the main goal of cross-platform? I don't realize what's the problem in using testing framework on several platforms. Of course not talking about GUI runners but any other methods of test framework shouldn't care about platform at all.

<humour ascerbic="true">This is due to the relationship between "theory" and "practice".  In theory there's no difference between theory and practice.  In practice, there is. </humour>

[Vincent Parrett 750]

On 10/10/2019 at 1:51 AM, Darian Miller said:

Also related, DUnitX says 

 

Cross platform currently supporting:

• Win32,Win64 and OSX Compilers.

 

https://github.com/VSoftTechnologies/DUnitX

The readme is a bit out of date, embarcadero did contribute some changes to make it work on mobile and linux.

[Darian Miller 361]

41 minutes ago, Vincent Parrett said:

The readme is a bit out of date, embarcadero did contribute some changes to make it work on mobile and linux.

 

Thanks.  I obviously didn't dig too deep... started a new Unit Test project and selected platform and there was Win32/Win64/MacOS platform options only.  I'll definitely dig in deeper!  Thanks