Mega Code Archive
Automated testing with dunit
<![CDATA[
To err is human – Automated testing of Delphi code with DUnit. By Kris Golko
To introduce bugs is human; the problem is usually not with fixing them, but with finding them before your clients do. Automated testing tools come to the rescue here, with their ability to repeat the testing process relentlessly, so developers can work with much more confidence. Why choose DUnit? Because it’s about unit testing (unit in the sense of building a block of code rather than an Object Pascal unit). Another advantage is that DUnit tests are frameworks within which to execute code, which is faster and more convenient than running an entire application. What I like most in DUnit is that I can create my test cases with my favourite development tool. DUnit supports Delphi 4 through 7 as well as Kylix.
Getting started
DUnit is distributed as source code; it comprises a number of Delphi source files, examples and documentation. It can be downloaded from the project web page on SourceForge sourceforge.net:/projects/dunit. To install, just unzip the file into a directory of your choice, preserving subdirectories. The units used to create and run tests are in ‘src’ subdirectory which should either be added to the Library Path in ‘Environment Options|Library’ or the Search Path of Project|Options in your test project.
How to write test cases
This is easy: all you have to do is to create a class inherited from TTestCase, TTestCase is declared in the TestFramework unit. To add tests to a test case class, simply add a published procedure for each test to your derived class. A test runner object uses RTTI to detect what tests are available in a test case class.
There are two very useful methods: Setup and TearDown. Execution of each individual test case within a class begins with Setup, then comes the published procedure which constitutes the test, which is then followed by TearDown. Setup can be used to instantiate objects needed to run tests and initialise their states according to the test’s operations. Setup should also contain actions needed to be performed before every test is executed, like connecting to a database. TearDown method should be used to clean up and dispose of objects.
A published procedure typically contains multiple calls to the Check method. The Check method is declared as follows:
procedure Check(Condition: boolean; msg: string = ''); virtual;
The condition parameter is usually in the form of an expression, which evaluates to a Boolean value. If the expression passed into the Check procedure evaluates to false then the test is marked as failed and the execution of the test is aborted.
...
Check(Expected = Actual, 'Actual number different than expected');
...
Tests are organised in bundles, called suites. Test suites can contain multiple tests and other test suites, thus providing a way to build a tree of tests. Central to DUnit’s operations is the test registry, which keeps all the suites in the test application. Typically, the test registry is built while a test application initialises. Units which declare test cases by convention have an initialization section where test cases are created and added to the registry.
Test suites can be created by instantiating the TTestSuite class declared in the TestFramework. The most convenient and often used way to create a test suite is to use method Suite of TTestCase class, which creates a new TTestSuite containing only the TestCase: note that it’s a class method.
The RegisterTest and RegisterTests procedures add tests or test suites to the test registry. The simplest example is to create a test suite containing a single test case and then register it as follows:
Framework.RegisterTest(TMyTest.Suite);
How to run tests
DUnit includes two standard test runner classes: TGUITestRunner with interactive GUI interface and TTextTestRunner with batch mode command line interface. TGUITestRunner is declared in the GUITestRunner unit along with the RunRegisteredTests standalone procedure, which runs the registered test suites using TGUITestRunner. The CLX version of TGUITextRunner is declared in the QGUITestRunner unit.
TTextTestRunner is declared the TextTestRunner unit along with the corresponding RunRegisteredTests standalone procedure. Calls to RunRegisteredTest are usually qualified with a unit name, since there’re multiple global RunRegisteredTests procedures, for example:
GUITestRunner.RunRegisteredTest;
Existing users of DUnit will notice that one of the more recent changes to DUnit is the addition of RunRegisteredTests class methods and the deprecation of standalone RunRegisteredTests, since multiple global procedures with the same name unnecessarily clutter the name space. The call to RunRegisteredTests is now recommended to be qualified with the test runner class name rather than a unit name:
TGUITestRunner.RunRegisteredTest;
An example
Let’s put it all into practice. The example application collects ratings in the way that many web sites provide a way to “rate this site”. Most notably, CodeCentral has an interesting rating system where you can set choices according to your personal preferences (my favourite is Ancient Greek Mythology). The basic logic of our rating system is defined by the IRateCollector interface.
IRatingCollector = interface
function GetPrompt: string;
procedure Rate(Rater: string; Choice: integer);
function GetChoiceCount: integer;
function GetChoice(Choice: integer): string;
function GetChoiceRatings(Choice: integer): integer;
function GetRatingCount: integer;
function GetRating(Index: integer; var Rater: string): integer;
function GetRatersRating(Rater: string): integer;
end;
The TRateTests class allows us to develop and test the business logic independently from developing the user interface. It’s actually advantageous to develop and test the business logic before starting the user interface. The test cases are designed to test any implementation of IRatingCollector.
TRateTests = class(TTestCase)
private
FRateCollector: IRatingCollector;
protected
procedure Setup; override;
procedure TearDown; override;
published
// tests here
end;
The Setup and TearDown procedures are used to instantiate and dispose of an implementation of the IRatingCollector.
const
SAMPLE_RATE_PROMPT = 'Rate DUnit (Mythological Slavic Feminine)';
SAMPLE_RATE_CHOICES: array[0..3] of string = ('Lada', 'Jurata', 'Marzanna', 'Baba Jaga');
procedure TRateTests.Setup;
begin
// modify this line only to test another implementation of IRatingCollector
FRateCollector := TSimpleRatingCollector.Create(SAMPLE_RATE_PROMPT, SAMPLE_RATE_CHOICES);
end;
procedure TRateTests.TearDown;
begin
FRateCollector := nil;
end;
Procedures declared in the published section are basic testing units; writing them requires inventiveness and creativity. In our example, TestChoices checks if the list of choices is as expected.
procedure TRateTests.TestChoices;
var
I: integer;
begin
Check(FRatingCollector.GetChoiceCount = Length(SAMPLE_RATE_CHOICES),
Format('There should be exactly %d choices',
Length(SAMPLE_RATE_CHOICES)]));
for I := 0 to FRatingCollector.GetChoiceCount - 1 do
Check(FRatingCollector.GetChoice(I) = SAMPLE_RATE_CHOICES[I],
'Expected ' + SAMPLE_RATE_CHOICES[I]);
end;
The TestRate procedure checks if executing the Rate procedure results in increasing the number of rates for the rated choice:
Procedure TestRate
...
FRatingCollector.Rate(NextRater, 0);
Check(FRatingCollector.GetRatingCount = RatingCount + 1,
'Expected ' + IntToStr(RatingCount + 1));
...
end;
Tests should be as comprehensive as possible, but it’s very difficult to cover all possible scenarios. While bugs are being reported, tests should be revised.
It’s very important that tests cover for extreme conditions; in our example, the choice or the rater passed to the Rate procedure might be invalid. The tests check if an exception is raised when an exception is expected. The following code checks if EinvalidRater exception is raised when a rater tries to rate second time.
ErrorAsExpected := false;
Rater := NextRater;
try
FRatingCollector.Rate(Rater, 0);
FRatingCollector.Rate(Rater, 0);
except
// exception expected
on E:EInvalidRater do
ErrorAsExpected := true;
end;
Check(ErrorAsExpected, 'Exception expected if a rater has already rated');
Finally, the registration of the test in the initialization section:
RegisterTest('Basic tests', [TRateTests.Suite]);
The project file shows the typical way to run tests in GUI mode.
GUITestRunner.runRegisteredTests;
To use CLX rather than the VCL, replace GUITestRunner with QGUITestRunner in qualified calls to runRegisteredTests as well as in the uses clause.
Extreme cross-platform programming
When porting DUnit to Kylix, there are two categories of problems, the difference between Windows and Linux system calls and the differences between VCL and CLX. Surprisingly, covering OS differences is easier. The first step is to put conditional statements in the uses clauses, units like Windows or Messages are of course not available in Kylix; prototypes of basic system calls can be found in the libc unit and basic type definitions in the Types and Qt standard units. Some system functions have been replaced by Linux equivalents, others have to be implemented.
It took a number of tricks to port to CLX. CLX and VCL visual components are only slightly different, but despite this, sometimes porting can be quite difficult.
DUnit for Delphi and Kylix compiles from the same source with the exception of GUITestRunner/QGUITestRunner GUI test runners, there are however a lot of conditional statements in the source to enable this cross platform support.
This is just the beginning
Once the basics have been mastered, there are many additional features within DUnit which enable more complex tests to be performed. For example, there are ready-made classes which can be used for a specific purpose, such as testing for a memory leak. The TestExtension unit contains a number of useful classes based on the Decorator design pattern. One of the most important is TRepeatedTest class, which allows you to execute a test case a given number of times. In this example, TRepeatedTest is used to call the Rate procedure several times in succession.
RegisterTest('Repeated Rate',
TRepeatedTest.Create(TRateTests.Create('CheckRate'), 5));
The class TGUITestCase, supports testing of the GUI interface. TGUITestCase is declared in the GUITesting unit. See RateGUITests unit for an example of using it to test the dialog box to submit the rating.
As the Delphi source for DUnit is freely available, so an experienced Delphi developer can easily extend DUnit, for example by creating new extensions.
Testing as a liberated art
Testing brings better results if tests are based on knowledge of the application design. If UML diagrams have been created, they can be used as a basis for the construction of tests, thus completing the requirement, analysis, implementation, testing cycle.
Ideally, tests should be developed at the same time as the development of project code. Technically, tests can be created for applications which are already complete, however, these applications are often not suitable for unit testing since they don’t have a well-developed modular structure. Using automated testing with DUnit promotes better application design as well as making it easier to refactor code, but I think the biggest difference it makes is at the stage of application maintenance. Maintainers can be assigned to units (in the sense of modules) rather than complete projects and they can bug fix and test units without building and testing a whole application. Sometimes a problem can be solved at the unit level and involvement of an experienced developer is necessary, but in the case of a large amorphic application, experienced developers have to be involved all the time.
]]>