DUnitX passed in params are confusing

[JamieMi 0]

Windows 11 Pro.  Rad 12.  DUnitX tests.  Trying to recreate example where I build easy math unit and trying to unit test the tests.  

---------------------- Example ---------------------------

    [TestCase('Test 1', '')]
    [TestCase('Test 2', '-1,-5,-6')]
    [TestCase('Test 3', '6,-4,2')]
    procedure TestAdd(Value1, Value2, _Result: Int64);

------------------------------------
If I use int64 values all is good.  Trying to test bad values (i.e. non-int64 values) fail.  TestCase 1, for example, actually passes in NULL, but when it runs the test passes.  Likewise, if I use non-int64 values, it passes every time.  The following all pass.

 

[TestCase('TestX', '')]

[TestCase('TestX', 'A,B,C')]

[TestCase('TestX', '"A", "B", "C"')]

[TestCase('TestX', 'X,Y')]

 

It looks like the DUnitX is massaging the arguments Value1, and Value2 to make them 0 if they are illegal.  I expected to see an exception when the test is run.

 

Any ideas what is going on?

[Uwe Raabe 2057]

Seems your are trying to test the test method or even DUnitX here instead of the method of your math unit.

[corneliusdavid 214]

I've actually wondered about this myself.  Isn't the purpose of the unit test to test what happens if you get invalid parameters? So this test would be expected to fail because invalid parameters (string) were sent in to a function expecting floating point.

 

However, I suppose it could be argued that a calling a function whose parameters require a floating point, would not even compile in Delphi if sending a string, so a unit test of this sort is pointless?

[Anders Melander 1782]

3 minutes ago, corneliusdavid said:

calling a function whose parameters require a floating point, would not even compile in Delphi if sending a string,

Format would like a word with you about that.

[corneliusdavid 214]

12 minutes ago, Anders Melander said:

Format would like a word with you about that.

If course, if using untyped or variant parameters, but OP declared the test procedure with a specific type (Int64--I mistakenly mentioned floating point), which would not compile.

[JamieMi 0]

I have been a tester and automator for the last 30 years.  It looks like DUnitX is fine for positive testing, but somewhat less for negative testing.  If I build a unit that is going to do mathematic calculations, I can come up with lots of tests that I would run manually, including passing in NULL for each term, passing in chars, passing in too large numbers, passing in objects, passing in Booleans, etc.  Is DUnitX designed for only positive testing?   Historically, this is where I tend to find lots of bugs...

[Der schöne Günther 316]

Keep in mind that while you *can* write DUnitX tests like this, you don’t *have* to.

You can still just have a method and define the parameters in there. To be honest, this is what I mostly do as well.

 

The attributes are nice when you have a test where you always expect the same behaviour, and you just want to test it with a bunch of different input values.

 

(Can’t comment on the negative/positive testing. I mostly write tests for my very own stuff, and I’m probably horrible at it)

[Uwe Raabe 2057]

The point is that if you want to test for invalid parameters, you have to pass them inside another test method instead of a separate test case.

 

Currently we have one test method

procedure TestAdd(Value1, Value2, _Result: Int64);

Unfortunately we don't have the implementation, so I have to do a bit of guessing here:

procedure TMyTestClass.TestAdd(Value1, Value2, _Result: Int64);
begin
  Assert.AreEqual(_Result, Add(Value1, Value2));
end;

Whatever is passed to the TestAdd method, the parameters given to the Add method are always valid types.

 

To test some invalid parameter we need another test method TestAddInvalidParams passing invalid parameters to the Add method.

procedure TMyTestClass.TestAddInvalidParams;
begin
  // call Add with some invalid params and chec if it fails
end;

Passing invalid parameters to TestAdd via a TestCase attribute only tests the capability of DUnitX to detect invalid parameters in a TestCase attribute. It does not test Add to fail with invalid parameters.

 

BTW, whether you even can pass invalid parameters to Add depends on the declaration of Add which we cannot see.

Edited December 18, 2023 by Uwe Raabe

[Roger Cigol 103]

It might be useful to edit the title of this post to "DUnitX passed in params are confusing"  <ie add in the "X" on DUnitX> since this post refers to DUnitX (and not to DUnit). This may help people in the future who search on "DUnitX" for DunitX related postings.

[Sherlock 663]

@Roger Cigol Done

[JamieMi 0]

Thank you everyone.  I have got the negative/positive testing under control using DUnitX with your help.  I have been searching all over for a good reference to DUnitX so I can use it more efficiently.  Most of the stuff I have found use the same lame examples that do an add, subtract, etc.  Can anyone point me to a solid reference book or site for using this tool?  Thank you in advance.

 

jamie

[Sherlock 663]

Consider looking at other projects tests. For example the tests in DUnitX itself: https://github.com/VSoftTechnologies/DUnitX/tree/master/Tests

or DWScript: https://bitbucket.org/egrange/dwscript/src/master/Test/

or SKIA4Delphi: https://github.com/skia4delphi/skia4delphi/tree/main/Tests

and last but not least Synopses mORMot: https://github.com/synopse/mORMot2/tree/master/test

 

and kinda out of competition there is this: https://github.com/NickHodges/DelphiUnitTests a noble initiative to create and collect unit tests for Delphi itself... sadly it did not take flight.

[Uwe Raabe 2057]

57 minutes ago, Sherlock said:

sadly it did not take flight.

Indeed, a bit sad. I still have hope that this will get some traction shortly.

 

It might be worth to know that a fork of that repo, namely https://github.com/UweRaabe/Delphi-Unit-Tests, is actually part of the regression tests run by Embarcadero. It also has a couple of contributors as well as branches, albeit lacking some activity, too.

[Sherlock 663]

Oh, goody! I was actually looking for that, when I was interrupted....and forgot about it