Zen's Blog

In a previous post (not true, not written yet), I outlined some of the properties that an automated test must ascribe to in order for it to be considered “well structured”. In this post I’ll dig into the concept of independent tests a bit further.

I have found that even though most developers by now know that tests should be written so that they are independent of each other, I’m not sure if they have realized the full implication of that statement. Let me explain what I mean.

For an automated test to be considered independent, it should not only code-wise not rely on state created in previous test cases, hence allowing the test code to be moved or refactored without affecting other tests; it should also be practically possible to execute any test case in any order –without rewriting the tests.

The implication of this is that all test suite creation must be completely separate to how we construct and write our tests. It is not okay to be forced to execute all test cases in a file. It is not okay to be forced to include all files in a directory in a given test suite.

The consequence of this is that the test runner must be able to take a suite-description that allows for execution of individual test cases. Even if the test case is stored in a file with a bunch of other test cases, and the file is stored in a directory with a bunch of other test files.

Further, the development environment itself must be flexible enough for easy temporary modification, such as a command line argument.

$> make run-tests #runs all test cases
$> make run-tests TEST=”monkey_can_peel_banana, fish_cannot_peel_banana” #runs specific test cases

Why is this important?

I, as a user of a test framework, would go bananas if I was forced to run all test cases every time. Yes, we strive for quick running tests, but quality cannot be achieved with unit tests alone. We also need higher level tests such as integration tests, component tests, system tests, feature acceptance tests, stress-tests…and they can take time.

Secondly, if we cannot execute test cases in different contexts, we are bound to copy paste the tests or not run them at all. For example, test case TC-1045 may be part of both customer A’s acceptance test suite, as well as customer B’s. We don’t want to be forced to copy the test code just because the framework doesn’t allow for the same test case be run in two different suites.

This has implications on how we structure our tests. Test fixtures become more important, since the execution context may have wider variations. It also puts greater strain on the test framework itself. The proper interpretation of setup/teardown, before/after suite and other such event-hooks becomes very important.

In summary, independent does not only mean code independent, but also practical test execution independence.

Posted in: Software Design, Testing by Zen /
Comments Off on Independence in Automated Tests

A software system should be based around a model, a metaphor that helps the developers how to think about the systems functionality and how to construct its features. However, it is important to realize that the metaphor is not an algorithm that can predict exactly how the system should be constructed. It gives hints, it helps with a general direction, a tone, to what to do –the metaphor only provides heuristics to constructing the system!

Posted in: Software Design by Zen / Tags: design principles

Comments Off on Metaphors in Software are Heuristics, not Algorithms

When partitioning your code it is important to consider the when, the what and the how. It can decide if your program will be well structured or become a tangled mess. These three aspects should be separated whenever possible. It is a strategy for thinking about dependencies and separation of concerns.

The When

An action could be triggered by an event or signal, a click on a button in a user interface, the call to a command-line tool, receiving a message on a socket. The infrastructure that holds and decide when the what is performed should be kept separated whenever possible.

Further, the when should have no direct knowledge of the what, so invocation should be done using hocks and other means of abstractions. This results in a sound infrastructure, isolated from the specific problem domain, lending itself for reuse and easier modification through extension.

This can imply different registration-mechanisms; where the application construction wires/glues the application together by instantiating and registering functions that performs the what to be called conditioned by specific triggers. Hence making the infrastructural-framework-core the decision maker of the when.

The What

What should be done! Think of it as a recipe for solving the problem at hand or creating whatever it is you are trying to accomplish. It is expressed in terms of abstractions. You shouldn’t be concerned with how, only what should be done.

This is usually placed in service-objects/services. It could be the EditEmployee-method that restores an employee object from persisted data, modifies it using accessors on the employee object and then write the changes back to the data storage. It does not concern it self with how Employee objects are restored, how they are edited or how they are saved. Only that these things should be done as part of the Edit-operation.

This allows the higher-level logic to be expressed in terms of abstractions that are consistent, cohesive, that can make use of reusable object as well as making it easier to understand the intent of the expressed functionality.

The How

This is the low-level classes that deal with concrete implementations of the lowest building blocks of the application. It could be the EmployeeDB class that knows how to persist an Employee instance to the database/persistence layer. It is tiny building blocks that are easy to test, easy to understand, easy to replace and easy to reuse.

The Test Framework – An Example

In order to keep a testing framework clean and simple to use, you should separate your whens, whats and hows. The core framework should deal with the when. The test-runner dictates when a test case is run, when its before-method and after-method (setup/teardown) hocks are executed and when the procedure for recovery should be invoked. However, the core framework should not concern itself with what it means to recover and cleanup; only provide hocks that allow other code to inject the what.

The test cases should hold the what. It decides what is created (setup/before-test-method), what is destroyed upon cleanup (teardown/after-test-method) and what is executed as part of the test. It however, does not concern itself with the how.

The how is taken care of by the production code; and if needed in order to keep the test-case-code clean, help-libraries that raises the level of abstraction in the scope of a test case. The help-libraries should not be part of the framework.

This might seem as an overzealous approach to separation of concerns. However, if not abided by, it becomes all to easy to put too much functionality into the framework making it harder to reuse, and also forcing you to create mechanisms for excepting special behavior.

Posted in: Software Design by Zen / Tags: design principles

Comments Off on The When, What and How of Code

I’m constantly amazed at how bad programmers are when it comes to designing good logging and reporting utilities. Hopefully your tool support input from stdin and allows for writing logs or interactions to stdout. If you haven’t gotten that right, I’m not sure what you are doing.

But when it comes to report- and logging-formats many programmers go astray.

All logs and reports should be generated in a structured unambiguous format that is easily parsed. If you want human readable, write an adapter that converts from the structured format to human readably.

$> mytool | mytooltohumanreadableconverter

Why bother with that, why not just generate output the way you want it right away? Here is why! You will most likely find other uses than just reading it off the console. If you have a structured format, it becomes easy to write adapters that can convert to pdf, doc, or other not only human readable but also human usable formats.

To use an example: output from tools such as a unit-test framework.

You run the tool in console and the output looks great. It is easy to read as a human and everything is fine. What if you want to run your test on your CI-build-server, such as Jenkins? You want to aggregate hundreds of test runs, collect statistics over time? Parsing the human readable output can be a nightmare. What about running the tests from within an IDE –such as Eclipse?

With a structured format, it is easy to write an adapter that converts the output to single-line, structured text that is easy to write an error-parser using the generic one provided in Eclipse. With not much effort you can see the errors highlighted in the source code –with correct meta-data about the error. If the human readable output contains line-breaks, not uncommon to make an entry easy to read for humans, it becomes very difficult for the Eclipse generic error parser to find all the information in a cohesive way.

• Structured output also allows for the writing of additional tools for log and report analysis and visualization.
• It becomes easy to parse the output and write it to a database.

The options become endless. But if you go the route of human readable, it becomes close to impossible to do anything else than just human readable –in that specific setting.

The same can be said about program output to standard out. Imaging the output to be structured, than parsed and converted to human readable. All of a sudden it becomes very simple to write interactive programs that can talk to other programs using the stdin – stdout interface. You can write an adapter and connect it to a socket. All of a sudden you have a program that works remote, and across platforms.

Using stdin and stdout to allow piping information is not enough. Structured output and input makes your program more versatile and makes it much easier to fit into a full tool-chain or other grander settings.

Posted in: Software Design, Tools by Zen / Tags: design principles, Tools

Comments Off on Why Using Stdin and Stdout is Not Enough!

Whenever I find myself with a code design that doesn’t work for me; it might be that the code is hard to test; or it require large rewrites for small changes in functionality; or it simple doesn’t feel right; I most often find that I have deviated from the fundamentals of good code design. As long as I remember the basics, I’m mostly okay.

Robert C. Martin, perhaps better known as Uncle Bob, has popularized a set of design principles through his acronym of acronyms – SOLID. In this technical talk, Martin explains the two first principles hidden in the acronym; Single Responsibility Principle (SRP) and Open-Closed Principle (OCP).

[infoq.com] – The Principles of Agile Design

If you haven’t heard about the SOLID principles before, this is a must see. Even experiences developers can benefit from this talk. It is never wrong to brush up on the fundamentals by recapping the basics of good code design.

Posted in: Software Design by Zen / Tags: design principles, SOLID

1 Comment