Brief Site Map Original Link New Brief 8/13/97, 2:26:48PM by NetAttaché Light

Ravi Palepu

Modelling the Real World:
Application of Patterns to Reduce Complexity in the Software Development Process


Ravi Palepu


School of Computer Science
Carleton University
1125 Colonel By Dr
Ottawa Ontario Canada K1S 5B6
palepu@scs.carleton.ca


Abstract

The application of patterns from the study of architecture to software engineering has brought new life into the OO Design community as it provides for a vehicle to move OO into what seems to be a promising direction [15,33]. By providing accessibility to more people, and enabling more domain-level reuse, it offers a path to reducing the increasing complexity of software development. In this paper, we will briefly describe patterns, their origin, their applications and their roles in the reduction of complexity. Some ideas that may improve this process are also discussed.

April , 1995 : Short Version

Introduction

Motivation for Interdisciplinary Study

It is well known that, in the preindustrial age, an interdisciplinary scientist was more common than those individuals who specialized in one field. These scientists applied many concepts and techniques from one domain to another. As the domains of science became more and more extensive, scientists also became more specialized. In contemporary times, a scientist usually spends the majority of their effort within the confines of a certain range of topics within a particular domain. The enormous amount of knowledge required to be an active researcher in a particular domain prevents many individuals to become experts in many fields.

The scientific community deals with this trend by assembling multi-disciplinary teams to take on complex and diverse tasks. Although these interactions between diverse scientists often have led to the formulation of interesting metaphors, the application of a technique native to one domain to another is often considered a novelty.

Recently several scientists demonstrate that the use of interdisciplinary study is not only valuable for its metaphors and analogies but also for its application. The most exciting work has been carried out by Adleman's use of DNA to compute the solution of the Hamiltonian path problem[0]. By representing a vertices by nucleotide sequences and binding them together to form the graph, the resulting DNA molecules encode the solution to the problem. By utilizing the enormous parallelism of the chemical reactions, this method determines a solution faster than a typical supercomputer by a thousandfold. This is a very valuable result as many other NP Hard problems such as optimal path scheduling and boolean logic satisfaction can be reduced to the Hamiltonian Path Problem. These interdisciplinary approaches may shed light onto the solutions of other difficult problems.

The OO software engineering community has borrowed much from the discipline of architecture in an attempt to help improve the software building process. Taking most of their ideas from the works of the modern architect Christopher Alexander, they attempt to make software engineering more like other design-intensive professions with histories that are proven to be successful[28].

Alexander's most influential work to software engineers is composed of a pair of books that describe the rationale, methods and details of patterns. Timeless Way of Building and A Pattern Language describe patterns and their applications to building objects ranging across a large level of granularity [3,4]. The first text has more meta-level discussion of patterns and describes documentation techniques and rationale, while the second book gives concrete details[28]. By abstracting common elements and themes found through experiential observation, he provides the reader with the rationale for his efforts. Alexander writes: 'Architectures can't be made, but only generated, indirectly , by the ordinary actions of the people, just as a flower cannot be made but only generated from the seed.' [3] In the second book he describes patterns found in building things ranging from cities to windows.

Within the context of software development, patterns can be considered as representing recurring structural or behavioral solutions in software[11]. These solutions can be found in almost any context imaginable as nature often uses similar behaviour to solve similar problems. The Object Oriented Software Engineering community has been borrowing from the discipline of architecture to help categorize, communicate about, and document these patterns that help solve problems.

The essence of documentation of a pattern provides the contexts in which it is suitable, constraints, and forces that may effect a solution or its consequences. Communication about patterns is enabled by the introduction of a vocabulary that describes the pattern and its related components (name, context, tradeoffs)[41]. By classifying these components and the nature (structural or behavioral [15]) of the solution, software engineers can categorize patterns.

Patterns can be collected in a structured manner to make up what Alexander calls a pattern language. The patterns are related by building on each other in order to transform needs and constraints into an architecture[28]. In the Alexandrian sense, each pattern is applicable to a particular context and the product of this application is a new context. The next pattern in the language must be applicable to this new context. This implies that there is a strict ordering of patterns. Since OO software is built using more iterative approaches, the application of pattern languages have been modified to reflect the iterative nature.

An architectural framework is a reusable design of a program or a part of program expressed as a set of classes. It is often domain specific and it can be considered an object of domain- level reuse[25]. The MVC Paradigm [24] is one of many examples[20,23,39] of a framework. A framework consists of classes that collaborate and detailed instructions on how to use and tailor these classes to meet other user needs. Hence, they consist of not just code so they are more abstract than most software, making them more difficult to document. A framework can be decomposed into components called design patterns. These design patterns are often domain independent and provide solutions to a number of contexts. The collaboration of the patterns can be documented in a pattern language.

Design Patterns:
The Next Step in the Evolution of OO Design

OO technology helped bring out the idea of 'building' software by placing much emphasis on the analysis and design aspects of the software lifecycle. The placement of the emphasis intended to promote the building of quality software (meeting the specifications and being adaptable for change). The software industry had previously suffered from the lack of focus on the early stages of the lifecycle[7]. In 'The Prince', Niccolo Machiavelli writes: He who does not lay his foundations beforehand may by great abilities do so afterwards, although with great trouble to the architect and danger to the building[32].

Grady Booch helped herald OO into the limelight with his landmark book Object Oriented Design with Applications[7] . His analogies to botany and morphology helped motivate the reader into realizing that decomposition is the essence of reducing complexity. By placing emphasis on the basic objects themselves, he did not take the analogy to a more profound extent. Biologists are now tending to study objects within their context or within their ecosystem. OO Design has also moved towards studying objects within systems as well. With what should be the next landmark book, Design Patterns : Elements of Reusable OO Software , Gamma et al have strived to shed light on how to design reusable OO software[15,17].

Unlike Booch's work which just deals with methodology and notation, this new book documents a standard way of presenting important information or patterns found in OO software. It also documents several of the most common patterns and provides examples in varying contexts.

More Alexandrian Influence on Design Patterns

In another influential text, Notes on Synthesis of Form [1], Alexander describes 'fit which he defines as mutual acceptibilty between context - that part of the world that we have no control - and form - the part over which we do control' (see [35] for more information). This provides concepts to describe the applicability of a pattern to a particular context. For example, in deciding which of two patterns would be more suitable for a given context, one could compare each of their fittings in terms of extendibility, reusability or other criteria. ( This selection of criteria is useful but remains an open problem. It would be useful for designers to develop the ability to reason about how a pattern's form can be changed in order to better fit the context.)

Complexity

'The complexity of software is an essential property, not an accidental one.'[9]. Brooks realized that complexity is a part of the nature of software development rather than just being constructed by the ineffectiveness of human effort. Software engineers have been traditionally dealing with this complexity by adding layers of interfaces that shield developers from the internal mechanics (usually more complicated) of the system (a.k.a. encapsulation). As projects become larger and larger in terms of developers and code sizes[6], organizational chaos tends to take effect. The complexity of the entire software process increases to levels where production is hampered.

Other disciplines must also deal with the complexity in nature. The physicist and Nobel laureate Ilya Prigogine states that 'our understanding of nature is undergoing a radical change toward the multiple, the temporal , and the complex. Curiously, the unexpected complexity found in nature has not led to a slowdown in the progress of science but on the contrary to the emergence of new conceptual structures that now appear as essential to your understanding of the physical world.'[31,38]

These conceptual structures may be radically different in nature as Brooks states: 'If as I believe, the conceptual structures we build today are too complicated to be specified accurately in advance, and too complex to be built faultlessly, then we must take on a radically different approach.' [9]

The use of patterns is not very similar to any other methods dealing with design. Design patterns provide complexity reduction by bringing a vocabulary to the process of designing as well as helping designers use the proper tools (patterns) for a given context. In the context of the implementation, design patterns tend to promote the use of components over inheritance based relationships. This seems to support reusability while decreasing complexity as it aligns with modern complexity reduction approaches [31]. These approaches place more emphasis on the relations of objects whereas traditional approaches used ontological schemes which emphasized objects and their properties.

Although skilled OO designers will not consider design patterns to be radical, they may consider the application of patterns to the software development process as a considerable shift in thinking. By bringing patterns to a much higher granularity, we can utilize them to help reduce and control complexity as they do in the lower granularity of design patterns.

The next section describes the application of patterns of a higher granularity of design patterns[10,12].

Application of Patterns to Process

Coplien has developed a pattern language that applies to new software development organization[12]. Its patterns deal with topics from structure, organizational behaviour, control, development policies, quality assurance , to communications. These organizational patterns, as Coplien coins them, can be considered to be a development process generative pattern language. The application of these patterns would allow us to understand organizations and its processes. It is the generative approach, having the ability to generate or build new processes and structures, that is radical from previous organizational analysis techniques[12].

This pattern language was developed by capturing patterns found underlying in successful projects. Patterns enhance the modelling of successful practices and structure by making sure that information such as applicable contexts and associated existing forces are documented. Also with an increased knowledge of the organization itself, utilizing patterns may help improve the prospects for success. This is similar to the way Dale Carnegie finds patterns in people with good interpersonal skills and effectively teaches them to others (utilizing diverse scenarios to demonstrate different contexts).

Did not IBM try to do this with AD/Cycle's Work Methodology?

Applying process management to the organizational analysis is an analogue to IBM's AD/Cycle use of data modelling to help quantify the management and processes of the software development lifecycle[36]. Coplien's organizational pattern language is much more abstract and contains more scope than AD/Cycle. Also unlike AD/Cycle , the pattern approach is not as radical as the approaches to take were taken from empirical experience and data. This should mean that these process-related patterns are more applicable and perhaps will be suitable for implementation. This is supported by the testimony of several managers, in highly productive cultures, who have confirmed many of the patterns in the text[12].

Many of the patterns offer sound common sense advice. The real value is found in the bundling of that advice with applicable contexts and noted exceptions. See Appendix A for an example pattern in the Coplien's organizational pattern language.

These kinds of higher level pattern languages provide us with new conceptual structures that are helpful in abstracting us away from complexity while maintaining a common communication vocabulary. Hence, maintaining our understanding of how a organization and its processes work together.

We have seen how patterns can help us deal with complexity in the software development process and now we will look at how patterns may help us in the lower granularity of design patterns.

Maintenance

Patterns may also be useful in reducing the complexity in maintenance related tasks. We shall look upon maintenance as being closely coupled to program comprehension and testing[34]. In attempt to show the lack of emphasis placed on maintenance issues, Hager writes 'Unfortunately, current methodologies for specifying , designing , documenting coding and testing software do not provide adequate visibility to maintenance concerns. The difficulty of generating software that is easily modified becomes evident when the full engineering life cycle costs are examined.'[18].

Maintenance Issues in OO: Comprehension

When someone is put into the position where they inherit much OO code, there is little in terms of CASE tool support that could help them understand this new code. We shall look at some maintenance supports tools have been proposed for OO systems and see if they can also be applied to design patterns.

It is well known that OO promises easier maintenance due to better encapsulation (ie. less ripple effect), but that only covers only a small portion of our concept of maintenance[7,18]. Ease of system comprehension is very important to OO technology because it helps determine whether developers find reusable code to be usable or exploitable. Users of reusable code must be able to understand how to use it and also be able to modify it to meet their specific needs.

System comprehension generally starts with high-level understanding and then given time it develops into a more detailed system understanding[43]. Class hierarchy and system browser tools can indicate the inheritance and component relations between classes. These are well known tools that have been available for most OO languages.

OO environments are lacking non-static tools dealing with dynamic binding and tracing type dependencies[14,29,43]. It would be useful if the chain of methods invocations could be drawn. It would also be useful to trace through the execution of a use- cases[22] without being bogged down with herculean number of messages that are inherent in such dispersed program structures[43]. Of course, during most execution tracing environments, the user has the ability to skip the long chains of method invocations as opposed to stepping through them. Research has attempted to abstract the major players of the collaborations in order to restrict or abstract out certain methods that are listed in chain (including one scheme that utilized 'contracts' [19]).

With the use of patterns, the collaborators can be easily identified. This gives a tool the intelligence it needs to display only selected method invocations during an event trace. By enhancing the ease of program comprehension, patterns reduce the complexity of maintenance and testing processes[34].

Please see Appendix B for some new ideas for CASE tools that would help support or utilize the use of patterns.

Application of Pattern Languages to Frameworks

As mentioned earlier, pattern languages are often used to document frameworks. Since frameworks are designed by domain experts and are used by non-experts, it is important to show how to use the framework. By presenting the framework in the form of a pattern language or a set of patterns, we can described the framework in a structured essay[23].

The three main components are :

Each pattern should contain as many examples as possible in order to illustrate the context (more so the range of contexts that are suitable for the pattern).

There are several pattern languages developed for frameworks. HotDraw [23] is a structured drawing editor framework. Conduits+ is a framework for network protocol software (generalized communication framework for implementing the signalling system of multi-protocol ATM access switch)[20]. Other frameworks include ET++ and MacApp[23].

Not only do patterns make suitable documentation tools for frameworks, they also may make suitable documentation tools for any reusable components including class libraries. Here we see the a reduction in complexity in comprehension with respect to reusability.

Discussion and Conclusions

Where are we now?

Robert Martin writes: 'Patterns are a way to capture good design principles, and they are good techniques. They may be helpful to designers who would like to look up potential solutions to particular problems. They are an effective way to communicate solutions to problems.'[33] At the PLoP conference, Martin felt that there was much discussion about patterns in terms of categorization and documentation but there was not much substance related to the actual application of design patterns. We speculate that this is because of the lack of understanding of how to fit patterns' forms into new contexts. Although Gamma et al's book demonstrates each of its patterns in a few domains, it does not and perhaps cannot give the unexperienced reader the ability to build OO software. It acts as a source and provides insight and exposure to other people's experience. Nonetheless, the description of each pattern's suitable contexts, advantages/disadvantages, and constraints help provide the reader to appreciate what goes into deciding the suitability of a pattern to a solution of a problem.

With the ability to communicate these patterns, the software community is called upon to share its experience with patterns. Either with new ones or by contributing more examples within a variety of contexts . The tutorial software and computer based teaching may be a proponent of heralding design patterns into the minds of OO software designers.

Will patterns stick around or are they more inflated hype?

It depends on what level of granularity of pattern application one is talking about. Design patterns seem to be very useful as do pattern languages for frameworks. Organizational pattern languages may be too radical of a shift for organizations. 'There is nothing more difficult to carry out, more doubtful of success, nor more dangerous to handle than to initiate a new order of things...the reformer has enemies in all those who profit by the old order, and only luke-warm defenders among those who would profit by the new order.' - Machiavelli [32]. It is interesting to note that before the PLOP conference, the organizers had a session that dealt with how to minimize the inflation of both media hype and in terms of the expectations from industry[13].

Could patterns reduce the creativity in designers?

It is well known that often architects feel that their creativity is diminished by the bounds of the patterns available and especially, government regulations. Software developers often also display a similar dislike to conforming to any regulations ( 'hacker' nature). Good designers will still reuse solutions that are proven to work in a given context. Instead of just copying and adapting the code, the developer just specializes a generalized case of the solution. Coplien argues that the creativity will be channelled into shaping patterns' forms to a given context[11].

Can we give something to other disciplines?

In an attempt to find some analogous software support systems that classify and catalog 'patterns' in the domains of biology and architecture, it was discovered that much of this area has yet to be developed. Other than simple documentation systems and trivial expert systems, no other comprehensive software (analogous to CASE tools) is available to help designers and observationists with their work. With this in mind, there is an opportunity to write a framework that would provide CASE support for several disciplines.

Any other comments?

The use of frameworks, architectures and high level patterns are helping making large scale domain reuse a reality. Design patterns are just a step towards realizing the promises of OO technology. As the number of patterns rises, we must be able to make these reusable designs easily accessible. Therefore we need also to develop classification systems so we can catalog the patterns in a pattern handbook and identify relationships or similarities between the patterns [3,26].

Appendix B (a number of proposed tools) contain new ideas that were outside the scope of this paper.

Bibliography

[0]Adleman. 'Molecular Computation of Solutions to Combinatorial Problems'. Science V266(11) 1994.
[1]Alexander. Notes on Synthesis of Form. Harvard University Press, 1964.
[2]Alexander et al. Houses Generated By Patterns. Centre for Environmental Structures. Berkeley, 1970.
[3]Alexander. Timeless Way of Building. Oxford University Press, 1979
[4]Alexander. A Pattern Language . Oxford University Press, 1977.
[5]Beck. personal correspondence with K Beck.
[6]Boehm. Software Engineering Economics. Prentice Hall. Englewood, 1982
[7]Booch, Object Oriented Design with Applications. Benjamin- Cummings, 1991.
[8]Brahan, Farley, Orchard, Parent, Pan. 'A Designer's Consultant'. Proceedings of Expert Systems'92. December 1992.
[9]Brooks. ' No silver bullet: Essence and Accidents of Software Engineering'. Computer, April 87
[10]Coad. 'Object Oriented Patterns'. Communications of the ACM V35(9) 1992.
[11]Coplien. 'Software Design Patterns: Common Questions and Answers' AT&T Bell Laboratories
[12]Coplien. 'A development process generative pattern language'. Proceedings of PLoP/92, Monticello, August 1994.
[13]Coplien. 'Progress on Patterns: Highlights of PLoP/94'. Proceedings of Object Expo Europe, London, Sept 1994.
[14]Deubler, Koestler. 'Introducing OO into Large and Complex Systems'. IEEE Transactions on Software Engineering,V20(11), 1994.
[15]Gamma et al. Design Patterns: Elements of Reusable OO Software. Addison-Wesley, 1994
[16]Gordon, Bieman. 'Rapid Prototyping: Lessons Learned' IEEE Software. January 1995
[17]Gossain. 'Book Review: Design Patterns: Elements of Reusable OO Software' JOOP, January 1995
[18]Hager. 'Software Cost Reduction Methods in Practice' . IEEE Transactions of Software Engineering V15(12) 1989.
[19]Helm, Holland, Gangopodhyay. 'Contracts: specifying behavioral composition in OO systems'. Proceedings of OOPSLA/90-SIGPLAN NOTICES V(25) Oct, 1990 .
[20]Hueni, Johnson, Engel. 'A Framework for Network Protocol Software'. To appear in OOPSLA 1995.
[21]Inglis .'Instance-based programming and OOP as we know it'. Object Magazine, June 1994
[22]Jacobson . Object Oriented Software Engineering, Addison- Wesley, MA 1992.
[23]Johnson. 'Documenting Frameworks using Patterns ' OOPSLA 1992 Proceedings .
[24]Krasner, Pope. 'A Cookbook for MVC UI Paradigm in ST80'. JOOP 1(3)88
[25]Kreuger. 'Software Reuse'. ACM Computer Surveys, 24(2) June 92
[26]Lajoie, Keller. 'Design and Reuse in OO Frameworks: Patterns, Contracts and Motifs in Concert'. ACFAS, Montreal, May 1994.
[27]Lalonde, Pugh. 'Communicating reusable designs via design patterns'. JOOP, January 1995
[28]Lea. 'Christopher Alexander : An Introduction for OO Designers'. ACM SIGSOFT Software Engineering Notes, 19(1), January 1994
[29]Lejter, Meyers, Reiss. 'Support for Maintaining OO Programs'. IEEE Transactions on Software Engineering,V18(12), 1992.
[30]Lilly. 'In search of the philosopher's stone and the silver bullet' Object Magazine, March-April, 1994
[31]Long, Denning. 'Ultra Structure: A design Theory for Complex Systems and Process'. Communications of the ACM V38(1), 1995
[32]Machiavelli, Niccolo.'The Prince' Cambridge, 1988
[33]Martin. 'Guest Column: Patterns: PLoP, PLoP, fizz , fizz'. JOOP, January 1995.
[34]McCabe,Watson.'Combining comprehension and Testing in OO development'. Object Magazine, March-April 1994
[35]Mellor. 'A deeper look: ...at multiple designs'. JOOP, February 1995
[36]Palepu. 'The Role of Data Modelling in AD/CYCLE'. Co-op work- term report. IBM Canada, Acadia University, 1991
[37]Palepu. 'SCRC Cards - A OO Design CASE Tool'. Thesis. Acadia University, 1993 (demonstrated at TOOLS'94, France).
[38]Prigogine I , Order Out of Chaos : Man's New Dialogue with Nature, Bantam Books, 1984
[39]Schmidt, Stephenson. 'Experience Using Design Patterns to Evolve Communication Software Across Diverse OS Platforms'. To appear in ECOOP Proceedings, 1995.
[40]Turney. 'Exploiting Context When Learning to Classify'. Proceedings of European Conference of Machine Learning, Austria, April, 1993
[41]Turney.'Robot Classification with Context-sensitive Features'. Proceedings of Sixth International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Scotland. June, 1994
[41]Viljamaa. 'The Patterns Business: Impressions from PLoP-94'. ACM Software Eng Notes Vol 20 No 1 Jan 1995
[42]Vlissides. 'Reverse Architecture'. Design Patterns: Elements of Reusable OO Software (Gamma, Helm, Johnson, and Vlissides). Addison Wesley, 1995.
[43]Wilde, Huitt. 'Maintenance Support of OO Programs'. IEEE Transactions on Software Engineering,V18(12), 1992.

Appendix A Example of Organizational Pattern[12]
(slightly modified)

Pattern Name
Code Ownership

Problem:
A developer cannot keep up with a constantly changing base of implementation code.

Context:
A system with mechanisms to document and enforce the software architecture, and developers to write the code.

Forces:
Something that is everybody's responsibility is no one's responsibility.
You want parallelism between developers so multiple people can be coding concurrently.
Most design knowledge lives in the code; navigating unfamiliar code to explore design issues takes time
Provisional changes never work
Not everyone can no everything all the time

Resulting Context:
The architecture and organization will better reflect eachother. Related Patterns are:......

Design Rationale:
Lack of Code ownership is a major contributor to discovery effort in large-scale software development today.
Arguments against code ownership have been many, but empirical trends uphold their value.
Implicit encapsulation with code ownership
Law is property, and lack of identifiable property leads to anarchy (Rousseau et al.)
It has been argued that code ownership should be applied only to reusable code. Such a constraint would be worthy of consideration if someone comes up with a good distinction between usable and reusable code.


Appendix B



Proposed Future CASE Tools and Training Devices:

Simple Stub Generator Tool

Automatic generation of patterns to code with a GUI similar to visual programming (Visual Age).

Expert Systems

1) Expert system providing decision making support based on a description of the context. Heuristics could aid the choosing among alternative patterns (without claiming quality for the actions suggested)[41].

2)In a tutorial-like manner[8], expert systems support users with tasks help. These systems diminish their interaction as skills are learned. This training facility can be a nice instructional tool for design.

3)A similar tool but geared for easing into the comprehending a framework.

Reverse Architecture Support

Reverse Architecture can be used for program understanding[42]. Research into the finding of known patterns in a system based on structure and perhaps other contextual information.

Rapid Prototyping Environment with Patterns and Component Library Although studies have shown that it is a myth that users mistake a prototype for the real product, a problem still exists in the phenomena that users often change their requirements and have a tendency of adding more and more features[16]. The use of patterns in prototypes allow for easier flexibility, extendibility and adaptability which turns into better response times to implement user requests. Also, prototyping using patterns can help designers decide which patterns are most suitable for a task.

Instance-based programming is common in the realms of component assembly environments such as Digitalks PARTS workbench or NeXTSTEP or Novell's Appware. The integration with instance based programming maybe helpful to prototyping[21].

Off-Line Training Tool

Use CRC cards as a learning device to help teach patterns. An on- line CASE tool with animation and versioning of CRC representations of patterns may also be useful[5,37].
Notes: Although this paper makes numerous references, there are also many claims that may have not be substantiated to the reader's desire. The author believes that these claims can be substantiated through both reading many OO journals' insightful editorials and using common sense.

Please feel free to share your comments or start related discussions: palepu@scs.carleton.ca

Back to my Homepage
| Brief Summary | Site Map| Original Link | NetAttaché(TM) Light