FitNesse – Test automation in a wiki

the assignment
When I started working at my current assignment I got told that the tools for automation had already been chosen, one of the tools being FitNesse. The way the tools were chosen is not my preferred way of picking a tool, the fact that the assignment was to “implement test automation across the organization through the use of the chosen tools” made me slightly worried whether or not FitNesse and the rest would indeed turn out to be the right choice.

Prior to this assignment I had heard a lot about FitNesse but had never had any hands-on experience with it, nor did I know anyone with hands-on experience with it.
Having worked with FitNesse for a few months now i feel the time has come to share my thoughts on it, what do I like, what do I believe is up for improvement, how is it working for me for now etc.

learning curve
Getting started with FitNesse was not all too intuitive. Getting it started is easy enough, but once you have it running it is not clear where to start and where to go from the FrontPage. Since we were not planning to use the standard fixtures but instead were planning to create our own we started on the fixture side rather than with FitNesse itself directly. Having created a generic login functionality in the fixture translating actions back int FitNesse became a lot more intuitive.

possibilities
The base fixtures such as the DoFixture, WebFixture etc. are very powerful in itself, I however feel they somewhat miss the point of automating in clear text: the tests are not easy to read, logical to follow or intuitive to write. We chose to work with SLIM rather than with FIT since FIT gives too much flexibility in usage of (almost) psuedo-code. Examples as used in the acceptance test in FitNesse are not clear enough for our purpose at this client. The test team is, to say the least, not very technically inclined and examples such as below do not really help them very much:

This is still somewhat readable

!|Response Examiner.|
|type  |pattern|matches?|contents?|
|contents|Location: LinkingPage\?properties|true||

A while loop implemented in FitNesse however quickly turns into black-magic in the hands of the technically less inclined:

|While|i=0|i<5|i++|
|put book in the cart|i|
|confirm selection|i|

With our custom implementation we now have test cases that can be read by most people within the organization and will be quite well understood, for example the below scenario for transferring money from one account to another:

|Scenario|Transfer Money|amount|From Account|accountFrom|To Account|accountTo|With Description|desc|
|Start               |TransferMoney |
|Go To|
|Select Van Rekening |@accountFrom |From Dropdown|
|Select Naar Rekening|@accountTo|From Dropdown|
|Enter Bedrag        |@amount|In Textbox|
|Enter Omschrijving  |@desc|In Textbox|
|Click On Verwerken|
|Select Content Frame|
|Is Text             |Het effect van deze overboeking op uw vrije bestedingsruimte is|Present|
|Click On Verwerken|
|Start               |CurrentAccount|
|go to|
|check               |Get data from column|Bedrag|3|

flexibility
Having started with Selenium as the driver below FitNesse enabled us to quickly build out quite a lot of test cases for the web applications. Part of the beauty of FitNesse in my opinion is that it is driver agnostic. In other words, it doesn’t really care what the system under test is, be it a website, a JavaApplet, a database or desktop applications. We are currently starting to work on TIBCO interfaces and will soon have to move over to Delphi and C# desktop applications. With quite some traditional test-automation-frameworks this would force us to start working either with a different tool or at least in quite a different way. The great thing about FitNesse is that it is so flexible that we can not only test desktop applications, we can also test across several parts of the platform. For example start executing some functions on the web application, verify these actions directly in the database, start a management desktop application and approve the actions initiated from the web application, all within one test case. A test case that big would make the test fragile, but the great thing is, it is possible if you really would want to.

refactoring
Quite some of the tests currently in FitNesse have been built up based on a functional mapping we initially made of the system, rather than the flows through the application. This is not quite ideal when running the tests, let alone when trying to sort through them and building up a suite for a particular type of flow or functionality.
Refactoring in FitNesse is probably the part where I believe a lot of improvements can be made. The current functionality, based on regular expression search is fairly crude.
FitNesse being a wiki, does have a wonderful different perk when needing to execute some bigger refactoring or moving around of test cases. All tests are text files within directories in the filesystem of your PC. In other words, if the built-in refactor function is too crude, a tool like Notepad++ or TextPad can be of immense value. These can search through files across directory structures and do a big part of the refactoring work for you. If you need to move whole folder structures, again you can copy them around directly on the file system.

judgement
My feeling regarding FitNesse so far is that it is a great tool which thus far seems to be underestimated out there in the world of test automation. Even when working directly with the standard fixtures FitNesse makes for easy to use, simple and quick to implement test automation. The main challenge is the initial learning curve, getting started with it and making the right choice in whether to go with Fit or Slim are for the newcomer the main hurdles.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s