Unit testing for beginners

[Mike Torrettinni 198]

I guess it's time I start to learn this topic. There's plenty resources I can find on google, but seems more scattered than focused material that would teach from first test case to more substantial examples.

 

I'm looking for something with more meaningful examples because this is completely useless for me: https://blogs.embarcadero.com/learn-how-to-do-unit-testing-in-delphi-with-the-powerful-dunitx-framework/

I just started new project, created a couple of file parsers and now I would like to implement unit testing. So, Calc examples... I just don't get it. 

 

Any suggestion on resources for beginners?

Edited October 27, 2021 by Mike Torrettinni

[Bill Meyer 337]

10 minutes ago, Mike Torrettinni said:

I just don't get it. 

Have you looked at this? https://delphiprogrammingdiary.blogspot.com/2018/09/unit-testing-by-using-dunit-in-delphi.html

 

DUnit was the first unit test base for Delphi. DUnitX came later, and I think you will find the articles on DUnit are a simpler starting point than those for DUnitX. Nothing wrong with DUnitX, it's just that many writers will likely assume you already use DUnit.

[Mike Torrettinni 198]

1 minute ago, Bill Meyer said:

Have you looked at this? https://delphiprogrammingdiary.blogspot.com/2018/09/unit-testing-by-using-dunit-in-delphi.html

 

DUnit was the first unit test base for Delphi. DUnitX came later, and I think you will find the articles on DUnit are a simpler starting point than those for DUnitX. Nothing wrong with DUnitX, it's just that many writers will likely assume you already use DUnit.

Thanks, I saw that blog, but it uses similar Calc example. I'm having the same problem with such examples like when I was learning about classes and I found plenty examples with animals, TAnimal, TDog, TCat.

I guess my imaginations is not good enough to go from test case of x+y to real life example, like file parser.

 

I don't need extreme version, yet, so DUnit is just fine for me. I found the one short chapter on unit testing in Expert Delphi book, by Pavel Glovacki, no other books on this topic, for Delphi.

[Bill Meyer 337]

2 minutes ago, Mike Torrettinni said:

I don't need extreme version, yet, so DUnit is just fine for me.

Read this, if you haven't: http://dunit.sourceforge.net/README.html

 

If that doesn't help, I will root around to see whether I have something from old work that I could share.

[David Heffernan 2353]

You don't need a Delphi specific text to learn about unit testing. In fact I expect that the best texts won't be Delphi specific. 

[Stefan Glienke 2019]

If you type "how to unit test and what to test" into google the first page lists a plethora of good articles on the subject

[Vincent Parrett 765]

5 hours ago, Mike Torrettinni said:

I don't need extreme version, yet, so DUnit is just fine for me.

DUnitX isn't an extreme version (whatever that is) - just a diffferent testing framework.. delphi really only has 2 well used unit test frameworks (other languages have many) - pick one you like. 

 

https://www.finalbuilder.com/resources/blogs/introducing-dunitx

 

 

[Mike Torrettinni 198]

1 hour ago, Vincent Parrett said:

DUnitX isn't an extreme version

What does X stand for?

[Vincent Parrett 765]

1 minute ago, Mike Torrettinni said:

What does X stand for?

Nothing really, I just added it to differentiate it from DUnit 😉

[Lars Fosdal 1793]

The X is for eXcellent 🙂

 

Edit: On the serious side, we chose DUnitX because the use of attributes allowed us to do more testing with less code.

Also, it is super easy to integrate with Continua CI.

 

Many of the tests are there to verify that we f.x. have working FromString/ToString methods for all our enumerated types, validation of  algorithms, and such - but a lot of the tests are actually closer to integration tests than to unit tests.  F.x. there are a large number of tests that do CRUD testing for our database objects.

 

We should add more tests, to be honest - but so much to do, so little time, so the Pareto rule, rules.

[Vincent Parrett 765]

24 minutes ago, Lars Fosdal said:

The X is for eXcellent 🙂

Yes, lets go with that 😉 

 

27 minutes ago, Lars Fosdal said:

Also, it is super easy to integrate with Continua CI.

It's alway nice to see no red numbers there 🙂

 

[Fr0sT.Brutal 900]

IMHO this picture

 

is not applicable here. If you learn how to unit test Calc, you're ready for testing anything. Just write tests and adopt some tricks on the way. You can examine some unit-tested project for inspiration.

I personally came to these conclusions:

- Use as little assert functions as possible. You don't really need Assert.IsAssigned, Assert.StartsWith etc - Assert.IsTrue/False or even Assert.AreEqual covers all these cases. In fact, I tend to use only AreEqual, AreNotEqual, WillRaise. Even WillNotRaise is not so necessary - maybe just for test counter, though it could be replaced with DoSmth; Assert.IsTrue(True). Reason is that Delphi lambdas add too much noise. With all that you depend less from testing framework and could easily switch it or conditionally plug another one (f.ex., DUnitX for Delphi and fptest for FPC with a single codebase).

- I don't bother testing obvious things like Assert.IsTrue(Assigned(TSmth.Create))

- I don't separate each check into its own method. If a test fails, no matter if others are OK or not - you have to fix it anyway. So I test all cases of a method Foo instead of Test_Foo_InvalidArgs, Test_Foo_NoArgs, etc.

- Sometimes I write tests first and then implement a method, this helps to understand what I want as a result.

- I try to test as much edge cases as possible - they are frequent causes of app failures and crashes.

- I avoid placing test data in attributes. Just don't like the idea. If I have to repeat the same thing with different args, I just extract it to a method and run from test body. External data sets (files with test data and expected results) are nice as well as they separate code from data so you can modify test cases without touching the code.

 

These are just my personal habits.

Edited October 28, 2021 by Fr0sT.Brutal

[Uwe Raabe 2064]

45 minutes ago, Vincent Parrett said:

It's alway nice to see no red numbers there

OT: I was very pleased to see GitHub recently offering Colorblind Themes for people like me, who have difficulties to distinguish between red and green. (Actually I am able to distinguish both colors, but they look almost the same and I cannot say which one is green and which is red)

[Stefan Glienke 2019]

2 hours ago, Fr0sT.Brutal said:

- I avoid placing test data in attributes. Just don't like the idea. If I have to repeat the same thing with different args, I just extract it to a method and run from test body. External data sets (files with test data and expected results) are nice as well as they separate code from data so you can modify test cases without touching the code.

The benefit is having individual test cases generated which can be executed individually - if you ever messed with conditional breakpoints or pass counters to stop at exactly those args that cause the failure you know what I mean.

[David Heffernan 2353]

2 hours ago, Fr0sT.Brutal said:

If you learn how to unit test Calc, you're ready for testing anything.

And then proceeds to offer six bullet points of advice! I think that there is a lot more to unit testing than meets the eye.

 

Unit testing is clearly about proving correctness of your code at the time you write it. It's also obviously about ensuring correctness of the code in the future in the face of modifications to code, refactoring etc.

 

But I think what is often overlooked is that unit testing (and indeed other forms of testing) is very much about defining what your code does, pinning down the contract between the code and its clients (fancy word for the things that call the code under test). Well written tests can provide clear documentation of the code under test. This can be extremely useful when you need to remind yourself what a particular piece of code does. The implementation of the code may be hard to read and extensive. The documentation of the code may be sparse or non-existent. But a well-written test can provide the information you need to fully understand what you can expect a function or class to do.

 

Furthermore, I routinely find that when I write tests, especially with this perspective of a test as documenting the contact, that I am unhappy with the original implementation and the contract that is implied. I will then refactor the code to achieve the contract that I want. This implies that testing early is good. Test driven development takes that view to the ultimate of writing tests before the code. But even if you aren't going that far, write tests as early as possible. Then when you discover that the contract isn't ideal, you can change it and reduce the impact of that change. Change to design always has smaller impact the earlier it is done in a design process.

[Fr0sT.Brutal 900]

48 minutes ago, Stefan Glienke said:

The benefit is having individual test cases generated which can be executed individually - if you ever messed with conditional breakpoints or pass counters to stop at exactly those args that cause the failure you know what I mean.

You're right, I solve this problem by adding test data to assert message.

21 minutes ago, David Heffernan said:

But I think what is often overlooked is that unit testing (and indeed other forms of testing) is very much about defining what your code does, pinning down the contract between the code and its clients (fancy word for the things that call the code under test). Well written tests can provide clear documentation of the code under test. This can be extremely useful when you need to remind yourself what a particular piece of code does. The implementation of the code may be hard to read and extensive. The documentation of the code may be sparse or non-existent. But a well-written test can provide the information you need to fully understand what you can expect a function or class to do.

Yep, I think so too. Writing tests I sometimes find design/code/API flaws and fix them before a method is used widely in app

[Mike Torrettinni 198]

4 hours ago, Fr0sT.Brutal said:

External data sets (files with test data and expected results)

Not sure if what you are referring here is same as I do now, but I have 'test cases' where I have exact input and output as text files. I run test with previous and current build and compare results (text files) if new build changed anything. Tests like these makes sense.

 

So, I'm having hard time going from such test to this:

6 hours ago, Lars Fosdal said:

Many of the tests are there to verify that we f.x. have working FromString/ToString methods for all our enumerated types,

 

I understand we have to test almost every method we have, but Calc and it's family of tests have limited input. How do you test parsing a line from text file?

 

For example, if text line is expected to be in this format: A B C D or A B1, B2 C D. You can test for a few examples for parser to parse correctly and identify A B C and D parts.

But if you define test case for these 2 examples, what's the point of test case, it already works.

 

 

 

[Lars Fosdal 1793]

It would be natural that the test cases also contain negative tests - i.e. stuff that is supposed to fail. 

Only testing happy path = recipe for disaster. You want to know that if you get garbage in, at least your code can recognize that it is garbage, report the problem, and move on.

[Lars Fosdal 1793]

20 minutes ago, Mike Torrettinni said:

How do you test parsing a line from text file?

It depends on the parser.  Sometimes it can be just "can this string be recognized as a valid representation of a value?", other times you could parse a generated artifact, or a predefined artifact.
 

[Mike Torrettinni 198]

Just now, Lars Fosdal said:

It would be natural that the test cases also contain negative tests - i.e. stuff that is supposed to fail. 

Only testing happy path = recipe for disaster. You want to know that if you get garbage in, at least your code can recognize that it is garbage, report the problem, and move on.

OK, this makes better sense.

Because testing Calc examples of x+y makes sense to test -MaxInt.. 0,1..MaxInt examples. With string or lines of string... it's 'unlimited' possibilities.

[Lars Fosdal 1793]

Just from the top of my head...
3 + -3
3 + - 3

3 +- 3

- 3 + 3

-3+3

-3 + +3

How tolerant do you want the parser to be?

I wrote a parser that did math on arrays in a stock price technical analysis tool back in the 90's.
I built the parser so that it generated an expression tree, so the same calculation could be done rapidly on a number of data points.
I could multiply a stock price with a currency price and subtract the adjusted value from the Oslo exchange with the same listed stock on NYSE.
Good fun!

[David Heffernan 2353]

7 minutes ago, Mike Torrettinni said:

Because testing Calc examples of x+y makes sense to test -MaxInt.. 0,1..MaxInt examples.

That doesn't make sense. You seldom attempt to write tests that exercise all combinations.

 

8 minutes ago, Mike Torrettinni said:

With string or lines of string... it's 'unlimited' possibilities.

That's the art of testing, finding a good set of test cases that are concise enough to verify, yet broad enough to provide comprehensive coverage.

[Mike Torrettinni 198]

So, it's more like: start with test cases that produce exact results as expected, then slowly expand to test for most commonly expected 'wrong' input. Stop when it makes sense.

 

 

 

[Lars Fosdal 1793]

Pretty much.  Add corner cases as they happen.

[Mike Torrettinni 198]

From all the blogs I read, I didn't come across anything like such a simple explanation.