Visualizing the architectural testability of a software system

If you ever find yourself designing something a certain way because you think it would be better that way, then you’re probably performing art and not design. Art is about self-expression. Design is selfless.

(Jeff Harris, Matter)

Some time ago I have written in this blog about the importance of measuring the quality of a design from the perspective of maintainability and testability. Quality can signify different things for different people and there is no one single “good” interpretation. Most of the time, for me this term assumes an objective, quantitative, measurable connotation. Jeff Harris reinforces this idea attributing a selfless expression to any good design. Harris is not a computer scientist (in fact he is a graphic designer) but I think he talks about the same concept, even if it is materialized in a different way. Of course there is some “art” in the process of software design as there is in the graphic design, but what I like to stress here is the need to have objective metrics that help us to manage the complexity of a software system in an effective but selfless way.

In this post I simply show how such metrics can be collected and shown in a non obtrusive way by using a little automation. To demonstrate the concept I have exploited the extensibility of Sparx Enterprise Architect CASE tool building an add-on that reads the UML model and calculates the metrics for the current diagram. Each class is annotated with a tag corresponding to the particular metric calculated (in particular I have used the Component Dependency metric to evaluate the encumbrance of a class). Then a note is created for the overall diagram with the metrics for the entire graph of objects depicted in the diagram itself (again, starting from the CD metrics, the automation calculates the Cumulative Component Dependency value and its normalized variations: ACD and CCD). An example of the result is illustrated below.

Basic infrastructure of the UMLToolKit.Core package

Basic infrastructure of the UMLToolKit.Core package annotated with testability metrics

This little automation can provide several benefits for the software modeler:

  • it enables an early evaluation of maintainability and testability since the first stages of the modeling activity;
  • every evaluation is performed on demand either for the whole model or for a single model slice (typically for one or more specific diagrams);
  • the metrics can be hidden in every moment at the diagram level, using a feature of the modeling tool (which show/hide tags);
  • the evaluation can be done at the class level or at the package level, allowing an incremental and hierarchical analysis of a system;
  • the analysis can be performed even before a single line of code has been written, enabling a validation of the design, independently of its completeness and the level of detail used in the diagrams.

These metrics are not new but in my experience they are not widely used as one would expect, especially considering the value of prognosis they provide. Experienced architects know very well the importance of a good dependency management in a large project. Nevertheless, there are few tools that elaborate their calculation from the code, such as Sonargraph and almost no one CASE tool (at the best of my knowledge) that introduces this calculation at the UML level. I believe that this little automation is worth the time spent to create it, especially for the way it is introduced (unobtrusively). I would be very glad to hear your experiences and thoughts about this way of gathering indicators of testability and maintainability directly from UML models. Feel free to comment.

A final note: Below a snapshot of a diagram illustrating how the add-on works at the package level. The picture has been taken during the development of the plug-in so it represent a good example of early evaluation. Indeed the snapshot illustrates the high-level structure of the UMLaidToolkit project: an effort to build add-ons including a broad range of extensions for Enterprise Architect.

The evaluation of the testability of the high-level package organization for the tool UMLaidToolkit

The evaluation of the testability of the high-level package organization for the tool UMLaidToolkit


3 thoughts on “Visualizing the architectural testability of a software system

  1. Hi Rick, thank you for including the reference material to the work from Bellcore. The commonalities between my article and the paper you cited are three:
    (i) the idea to embrace an early prediction of some quality characteristics of a complex software system;
    (ii) using an architecture-centered approach where the architecture is viewed as a set of interconnected components, rather than a single black-box;
    (iii) the object-oriented programming style of development.
    Apart from these “context” commonalities, the work you mention is totally different. It predicts the reliability, the performance, and the performability of a software architecture, based on Markov Chain models. The approach described by Gokhale and colleagues defines an experimental methodology based on intensive testing in order to determine the analytical parameters used to estimate the failure behavior of each component.

    My work is quite simpler and with a totally different goal. It predicts the testability and maintainability of a software architecture, without any claim about performance, reliability, or failure estimates. Moreover, it is not based on testing. The testability is viewed here as a property of the design (i.e. an inherent quality attribute of the architecture). My work is based on the Component Dependency metrics family originally defined by John Lakos at Mentor Graphics and published (among the other resources) on the book “Large-Scale C++ Software Design”. The difference between my work and that of Lakos resides in the model-driven approach. Lakos proposed a technique working directly on the code whereas UMLaidToolkit is an attempt to bring this (and other ideas) in the early steps of the modeling activity, inside a UML tool.

    Hope that this help to clarify your questions. Feel free to reply or contact me privately, as you prefer. Thank you for commenting this article.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s