Last update
PvD
Architecture
Phases in Architectural Design
An architecture is less intended to create a specific system, but more to create a generic solution for a problem area. Of course, an architecture will be implemented in some system, but a good architecture goes beyond a single implementation: it may be implemented from low-end to high-end systems, and in multiple generations of systems.
Research, Development & Engineering
These frequently used terms –often abbreviated to RD&E– indicate distinct phases in development.
- Research
- Investigation (commonly scientific) with unknown results beforehand. The objective is to expand human knowledge; the concrete results –if any– are commonly reflected in publications and patents.
New knowledge is desirable (new technology, new products, methods, medicines), but research has –by its nature– an unpredictable outcome, and is therefore a risky and costly undertaking.
Often a distinction is made between Basic Research ('see what we stumble upon'), and Applied Research ('we know this effect, but we try to understand and/or control it').
- Development
- Structured (commonly technical) exploration with defined objectives (e.g. some product with acceptable price/performance ratio).
There is no clear separation between Applied Research and Development.
- Engineering
- Technical work with a defined result according to defined rules (similar work has been done before, but conditions usually vary).
Concept Maturity Level
System design differs from 'normal' design in that covers more than a single discipline and it takes all stakeholders (owner, users, maintenance, developers, production) into account.
It commonly is for a large and complex –and therefore expensive– system.
When a system includes many subsystems of different nature (i.e. involving distinct technological disciplines), it becomes a system of systems; due to the complexity a structured approach is warranted. The following Concept Maturity Level (CML) should help the development from idea to product.
| CML1 | Brainstorming
Generating (conceptual) idea's, possibilities, opportunities; usually extrapolating on trends (technology, society, …).
|
|---|
| CML2 | Selection
Appreciate the various ideas, which implies developing selection criteria for comparison.
|
|---|
| CLM3 | Experimenting
Investigate the viability of the proposed concept (preliminary design, testing essential concepts, solving bottlenecks).
|
|---|
| CML4 | Development
Develop a prototype (single system or small '0-serie').
|
|---|
| CML5 | Demonstration
For a really new concept apply a system in a (simulated) usage environment (potentially 'the field') to show that it works: Proof of Concept (PoC).
|
|---|
| CML6 | Implementation
Normal production of the system.
|
|---|
Obviously, there is considerable iteration, in particular among the first levels.
Be aware that customers often ask an improvement in an existing solution, which is not always the way to go. If Henry Ford had asked his customers what they wanted, the answers would be 'faster horses'.
CML1, 2 & 3 more or less correspond to Research (with CML3 as Applied Research), CML4 & 5 to Development, and CML6 to Engineering. Of course, the fact that system design involves distinct disciplines makes it more complex. And also, the RD&E phases will be different for each discipline.
To the optimist, the glass is half full. To the pessimist, the glass is half-empty. To the systems engineer, the glass is twice as big as it needs to be.
Technology Readiness Levels
Technology Readiness Levels (TRL) is a rating/scale developed by NASA (and applied in the EU Horizon-project) to indicate the maturity of a specific technology.
It discerns:
- Basic principles formulated
- Application formulated
- Unvalidated concept
- Early prototype (conceptual application)
- Big prototype
- Full prototype on scale
- Pre-commercial demonstration
- First commercial demonstration
- Commercially available in niche application
- System integration required
- Mature, and predictable growth
Architecture Design
Architectural design is the development of a functional structure to tackle a particular
(technological) problem area. It commonly involves multiple disciplines (e.g. hardware & software, and taking trends into account), so it is system design.
The following provides the main phases to develop an architecture (or any large system for that matter).
Inception
Here an idea is explored for its viability (technical & economical; the exploration is short but not superficial).
- Technical viability: through feasibility studies, modelling, simulations and prototyping.
- Economical viability: through business cases: they will show the critical cost- and gain areas.
- Market confirmation: through discussions with key customers, by means of vision statements and high-level impact analyses.
Preliminary design
Design of a preliminary architecture. The purpose of this is not to really develop a system, but determine the main components, discriminating features, capabilities & restrictions/bottlenecks to allow examination of technical & economical feasibility.
- Goal: what is the system (architecture) to achieve. What are its specific features (what is it to solve better than exiting architectures).
- Context Diagram: it must be made clear what is covered by the proposed architecture. This is best done through a so-called context diagram identifying all components/functionality in the system.
This demonstrates two important aspects:
- Scope (coverage) of the system: what is in and what not;
- Interfaces of the system with its outside world.
- Main components: what will be the main/critical components, and what will it cost to obtain them (make or buy, costs, lead time).
- Restrictions: what are the basic restrictions/limitations/bottlenecks in the proposed architecture ? Why are these restrictions, are there other techniques/methods that circumvent these restrictions (this is the opportunity for innovation).
See also Design Tactics below.
Analysis
The preliminary architecture is thoroughly analysed, and the business case is created.
- Describe the proposed system (architecture), and its main features, bottlenecks, critical components (& interfaces) against the system's business case. Commonly, a lot of attention needs to be spent on the bottlenecks.
- Provide a basic business case: analyse the tensions in technology, economy and market; understand how the proposed system affects and interacts with existing businesses, operational practices, market position, business cases, etc.
Investigate how the architecture can handle shifts in the market ('flexibility').
In particular for the ICT domain, consider how the architecture will deal with the forecasted technology improvements.
- Start a requirements specification, and create a matrix to track requirement origin & dependancies.
The above phases are commonly iterated several times before a satisfactory result is obtained. These phases are collectively also known as System Definition.
At the end you have a Business Case, a System Architecture and a Requirements Specification.
Maintenance
The architect/architecture team remains involved through the development of the system regarding change requests and implementation issues.
The requirements matrix and involved design documents will be updated as necessary.
Overview
This section considers some architectural principles and criteria regarding the structure –or the 'design'– of some 'system'. As the structure of Requirement Specifications may have a profound impact on the design/implementation of the corresponding system, these considerations apply to Requirement Specifications as well.
This section is partly based on an article in Computer Networks and ISDN systems Vol.29 (1997) p.397 by K.J.Turner.
To appreciate a particular architecture, the following criteria apply:
| Principles | Techniques | Criteria |
|---|
| Composition |
Functional decomposition
Constraint decomposition
Temporal decomposition
Spatial decomposition |
Coherence
Decoupling
Proportion |
| Generality |
Generalisation
Unification
Parameterisation |
Abstractness
Commonality
Adaptability |
| Simplicity |
Idealisation
Deferment
Minimisation |
Uniformity
Elegance
Economy |
Composition
Composition refers to the division of a problem/system into more manageable components (i.e. the 'divide and conquer' concept).
Composition is obtained through architectural refinement (top-down: decomposition, break-down), by assembly (bottom-up), or both.
It is related –but not identical– to modularity which indicates the capability to
- select a subset of 'modules' when only a subset of functions is required (qualitatively), and/or
- vary the number of 'modules' according to the required capacity (quantitatively, 'scalability').
The first notion of 'modularity' (qualitatively) may have serious consequences for Testing.
The objective is to have a tree of components with progressive levels of detail. Such a tree has width (horizontal partitioning at the same level of abstraction) and depth (vertical partitioning at distinct levels of abstraction). When one obtains a directed graph instead of a tree, it is an indication that separation in levels of abstraction is called for.
Decomposition stops when the architect (specifier/designer) considers that the architecture has sufficiently manageable (& feasible) components (which may lead to varying depth in the branches of the tree). This may not be an easy decision and carries the danger of decomposing too far.
Techniques
Functional decomposition
is well-known and most used. There are however more methods.
A variant of Functional decomposition is Property decomposition as applied in O.O. design.
Constraint decomposition
(of specifications) separates various concerns of the system (i.e. the combination of aspects provides the overall; constraints not limited to just one function).
Temporal decomposition
emphasises time-related phases of a system (e.g. resulting in a state diagram or Finite State Machine).
Spatial decomposition
separates in space (topological rather than geometrical). In particular usefull for distributed (processing) systems.
The important issues here are separation, distribution and replication of elements (i.e. OO-like).
Criteria
Coherence
of a component requires that that component brings together closely related aspects (functions, constraints or whatever).
Internal details (e.g. structure) should be hidden and related data should be encapsulated in the same component.
Decoupling
reflects a separation of concerns among components.
Components should act as more or less autonomous units.
Relationships (interfaces) between components should be restricted in respect to:
- Number of interfaces:
- if a component interfaces to most other components, that component probably belongs to another layer of abstraction, either a higher layer (e.g. management/control) or a lower layer (facilitating services);
- if a component interfaces only to a single other component, the merge of these 2 components should be considered (are they really 2 distinct components);
- if a component interfaces only to 2 other components, that component seems to act as a filter (taking care of conversions and/or security and/or validation). Acting as a filter is acceptable at the system's boundary, and internally to a re-used or externally acquired component. Other use of internal filters should be discouraged, e.g. by adaptation of the interfaces involved, or incorporating the filter into one of the connected components.
A high number of intra-system interfaces is a negative indicator for decoupling (poor design). As a rule of thumb, each component should not have more than 4 interfaces, and for a system with N components, the total number of interconnections should not exceed 2N.
Note that the interface to a lower layer of abstraction is commonly not counted at individual components, but as a single interface provided by the lower level to all components (so introducing a level of abstraction will reduce the interface count).
- Complexity: The exchange of commands and responses across an interface to achieve the desired goal effectively constitutes a protocol; if it is elaborate and/or complex, it is a negative indicator for decoupling.
- Bandwidth: This reflects the amount of data to exchange and the required response time across an interface (i.e. bandwidth_of_interface = amount_of_data_to_exchange / required_response_time). It is inversely proportional to the level of abstraction.
A high bandwidth on an interface is a negative indicator for decoupling (are they really 2 distinct components); it also shows a potential performance bottleneck and may force to reconsider the decomposition (i.e. the architecture).
All interfaces of a particular component together effectively constitutes the requirement specifications for that component (such that the component may be replaced by another implementation fulfilling the same interface requirements). Therefore each interface should be specified sufficiently abstract to allow such replacement.
See also the discussion at Process Model.
Proportion
requires a balance between width and depth of the decomposition tree (i.e. insufficient functionality in decomposed components).
A wide and shallow tree has probably too many components at each level for proper understanding and control.
Three to seven components per level is considered good practice ('span of control').
A narrow and deep tree may introduce unnecessary levels of refinement that confuses the decomposition strategy (a lot of refinement but insufficient abstraction).
When there are more than (three to) five decomposition levels, the difference in scale of components at various levels becomes very large (something like 55 ≈ factor 3000);
consider the introduction of an abstract layer with 'primitive functions' for higher layers.
Generality
Generality refers to the concept of specifying/designing in more general terms than strictly required, thus allowing a wider application.
It requires a delicate balance and is not easy at all; as a consequence generality tends to be either neglected or over-applied.
An architecture specific to current requirements may be lean and effective, but extremely hard to modify when requirements change.
A very general architecture may allow for future enhancements and functions not yet envisaged, but too cumbersome for foreseeable needs.
The ease with which software can be changed has led to over-expectations regarding the easy to adapt software systems for some desired change (the utopian flexibility).
However, when varying top-level requirements can be translated to a common set of lower-level requirements, one has obtained an abstract solutions layer (i.e. an abstract machine, platform) which is capable to provide solutions over a wide range of problems in the same domain.
This last method has proven to provide satisfactory results.
The architect/system designer must seriously consider how the requirements might change due to new uses, changes in system environment and developments in technology.
Accurate predictions are unlikely, but it is possible to query systematically the stability of assumptions and present requirements (e.g. 'what if' questions).
Techniques
Generalisation
avoids unnecessary restrictions in behaviour, environment or technology. There will always be restrictions that are inherent to the problem. A useful discipline is to try imagining alternative ways of structuring or realising a specification. If there are conceivable alternatives that have a major impact on architecture, then the architecture is insufficiently general and open-ended.
Unification
is a powerful means of ensuring generality. Similarities between components should be identified and questioned. Components should be unified as long as this respects the need for coherence.
Parameterisation
is one way to make a system more general. What is given and fixed now (e.g. constants) may change in the future. Value parameterisation (as used in software procedures) is helpful here. Structural parameterisation can provide replication of elements (in this regard it is sometimes said there are only 3 important numbers for a computer: 0, 1 and n). However, excessive parameterisation may interfere with straightforward use, so sensible defaults should be provided where possible.
Criteria
Abstractness
ensures that irrelevant details are omitted from the architecture.
An abstract architecture is a model of the system, and is -like any model- scaled-down in some way.
The omitted details may concern internal interfaces, algorithms or structures.
A common way of expressing this is that architecture deals with 'what' and not 'how'.
However, in particular for real-time and embedded systems, the feasibility of solutions (e.g. regarding time- & memory- constraints) must be assured. A balance must be sought between choosing an overly abstract architecture and one that is overly concrete. Over-specification leads to details that are not properly part of the requirements (but of a particular solution).
Commonality
aims to absorb arbitrary differences.
Re-using a library of existing components encourages commonality, though there are problems in defining sufficiently general components that can be instantiated easily (O.O.T. tackles this kind of problem).
Components of an architecture need not be implementation components; they might include functional specifications or constraints for example.
See also Software Reuse.
Adaptability
to foreseeable changes is needed.
Such changes may affect requirements directly, but also indirectly through environment or technology. A general architecture will certainly be implementation independent.
Specifically:
- Technology independent: a good design should survive tens of years, so technological changes (already significant in 5 years) should be allowed for (Architecture will outlive equipment many times over).
- Scalable: the design should cope effectively with small and large applications (quantitative).
- Modular: it should allow for variants in functionality and be open for extension with new functionality (qualitative, however see Testing regarding this topic).
Simplicity
The principle of Simplicity is complementary to Generality, most likely it is even in tension with it.
Generality seeks a common solution that unifies differences, whereas simplicity seeks to ignore differences assuming they are not essential.
Simplicity leads to a more abstract architecture because irrelevant details are ignored, but the architecture is not necessarily more general due to the elimination of features.
From the point of view of people (users, developers), conceptual simplicity –which should be a result of 'architectural simplicity'– of a system is invaluable in understanding the system.
If the system provides a particular logic, people can find their way throughout the system even when they only know part of it.
Consistency in 'look and feel' is part of this conceptual simplicity.
Techniques
Idealisation
ignores constraints until they have to be taken into account. A cleaner architecture often results from taking a 'blue sky' approach initially, ignoring practical restrictions. Only later these limitations need to be addressed. This incremental style of architectural refinement allows early concentration on the essential features. Specifications are often simpler without bounds, and certainly easier to read and understand. Constraints can be added later, or as separate issues.
Deferment
is concerned with the controlled introduction of details.
It is necessary to ensure a controlled development of an architecture: details should not be introduced too soon, and should be dealt with in a systematic way.
Deferment considers when to introduce details (whereas proportion concerns the balance of width and depth of the decomposition tree).
Deferment should be exercised during architectural refinement, since there is always the temptation to make too large a jump in abstraction level. Implementation concerns should be gradually eased into the refinement: initially major matters such as physical distribution and type of resources, later boundaries should be firmed up and interfaces defined. On the other hand, decisions regarding (implementation) details may be decided rather early (and consciously) when this is an obvious choice (i.e. any other choice would have serious negative impact). And again, for real-time and embedded systems, it is very hard to choose when to introduce (time- & memory-) constraints and still assure a feasible implementation.
Minimisation
requires one solution where several are offered: Occam's razor ("It is vain to do with more what can be done with less").
In particular avoid special cases unless there is a strong justification.
Minimisation also requires consistency: a small set of basic constructs should be used with fixed and unambiguous meaning. This may lead to several basic operations instead of a single integrated operation (which may oppose performance instincts).
Criteria
Uniformity
requires unnecessary differences to be eliminated.
Symmetry is one aspect of this. Care should be taken over introducing asymmetrical treatment such as a general function that only certain components can invoke, the presence of some but not all complementary functions, or universal rules that are suspended in some circumstances.
Elegance
is subjective and hard to define; inelegance is perhaps easier to recognise. It has more to do with the 'conceptual simplicity'. Examples of inelegance include ad hoc constructions, complex interdependencies, and large monolithic architectures.
Elegance is found in compact architectures that clearly embody the essence of a problem.
Economy
should show in the use of few but powerful constructs.
Restricting the number of architectural components and their types of combination promotes economy.
There should also be economy in the distribution of features.
Ideally, features should be localised where possible (locality in the logical sense, not physical boundaries).
A special issue is the economic use of scarce resources (or expensive resources such as CPU-time, memory size, disk space or accesses, communication transfer).
Specifics for Requirement Specifications
Requirement Specifications should describe the product or service to be provided. That may sound trivial, but it is very hard to create good specifications.
Initially not all requirements will be 'hard'; many points may be desirable or optional, open for discussion with the customer, or potentially moved to a later release. However, before the project really starts the Requirement Specification must be concrete and fixed; only then effort, costs and lead time can be estimated reliably.
Requirement specifications do not only serve to define the desired product, but should also used by the manufacturer/developer to prove that the product/service fulfills the requirements and the customer has to accept it.
And after the development has finished, the requirement specification should be adapted to become the product/service specifications (describing what it is capable of, usually exceeding the requirements).
Hard Requirements
Good requirement specifications are (apart from the points discussed in Architecture above):
- Coverage or Scope
Start assuring what the system is to cover: create a Context Diagram showing what is in the system (and what not), and what the interfaces to the outside world are (each entity type).
- Abstract
Good requirement specifications are abstract; they should not prescribe a solution, but allow freedom to the designer for any solution fulfilling the requirements. There may be exceptions to this rule, for example due to regulatory requirements. Writing specs in an abstract way is not easy and makes reading harder, but it is essential for good requirement specs. In particular the functional part of specifications can and should be written in an abstract sense.
- Clarity
All requirement statements should be clear and unambiguous. Again, this may sound trivial, but don't be surprised if different people provide a completely different explanation for the same piece of text. Examples and figures may help, but these should not be part of the formal specification (to assure Abstraction).
- Consistent use of terminology and phrasing are essential. A requirement specification is not literature: do not use different words for the same concept (e.g. component/module/subsystem), or different verbs for the same meaning (must/shall/should/will/must).
- The individual requirement statements must be categorized and grouped; the document should have a clear structure, corresponding to the product's functionality.
- Quantified
Requirements –and certainly constraints– should be quantified. Quantified requirements not only provide clear test- and acceptance criteria, they may also give a good indication of the critical points in any solution architecture (i.e. its suitability). In particular performance parameters in de widest sense (dependability, accuracy, throughput, delay, etc.) should be attributed with values. In many cases a staggered approach is desirable. For example: delay should be less than 2 seconds for more than 90% of the cases, and less than 5 seconds for 99% of the cases. Other cases warrant both a relative and an absolute value, for example: system unavailability should be less than 1%, but never longer than 30 minutes.
Other issues for requirement specifications are less easy to appreciate:
- Are the requirements complete (coverage; here the context diagram sould provide some assistance);
- Are the requirements stable (no 'scope creep', 'improved insight'), otherwise consider a phased/prototype approach (changes are for the next version/iteration);
- Do the requirements cover what the user needs (not what the customer says he needs – customer and users are often different persons);
- Is the system flexible enough to cope with changes (in technology, market demands, …). Do an excercise –preferably with the customer(s)– on desired flexibility in the system.
Flexibility & Adaptibility
Specific issues are the flexibility and/or adaptability which customers/users implicitly expect to be in a product/service. The notions flexibility and adaptibility are not identical, but in this context almost equivalent.
When the requirements do specify flexibility without indicating for which parameters in which range, or specify adaptibility without indicating in which area to what respect, you can not fulfill that requirement !
For example, the required product must have a flexible number of interfaces. If the product has 10 interfaces of a particular type, and the customer wants 15, you can probably accommodate him. But not 2000 interfaces; you probably need to define a completely different product (architecture) for that.
Consider the case that you get the task to buy a car for flexible purposes. Will that be a sports car that you can drive in the city, or a minibus to carry up to 8 persons, or a minivan to carry parcels, maybe a pick-up, an off-road MPV or a camper, a big bus or a big truck ? If you have buy an adaptable car, can you adapt it to transport more people, or adapt it to carry more cargo ?
If the customer is incapable to define in which areas he may desire flexibility/adaptability and for which range, you can not design a solution (we are discussing a product/service still to be developed, not an off-the-shelve product to be sold to an inexpert user).
Alternatively, you may define ranges for various parameters in (the response to) the requirement specification, or specific areas for limited adaptations, and have the customer accept these (preferably by sign-off).
Techniques
There are various ways do describe functionality, for example (not excluding others):
- Formal input/output action/response
A very good method (in particular when there is experience with the required functionality), but may be cumbersome for some areas. Usefull as Acceptance Test.
- Use cases
Very illustrative and usefull to clarify desired functionality (in particular for a customer to describe what he expects), but usually provide insufficient coverage.
- Formal text
Can cover all, but less specific (clarity, risk of being verbose).
- Screen shots
In particular when there are a lot of man/machine-interactions, artificially created screens (e.g. forms) with sample input and output will be very helpful. Showing them to operators/users of existing systems will point out shortcomings and may reveal additional practical requirements to the system.
The best method depends on the specific product domain (and experience with the desired functionality), but preferred is a specification which can also be used as (acceptance) test specification.
Specifics for Architecture Design
Design Tactics
Start with the Context Diagram, an try to understand the intended position of the system in the world. Investigate the external interfaces regarding complexity and bandwidth.
Try to use a layered approach; this has proven to be the most flexible (but also not the easiest). Attempt to create a solution not for a specific problem but for a problem area, i.e. with all kinds of variants.
The top layer should reflect the Service Model, and the bottom layer the Platform Model. Of course there are potentially more layers in between, but you can start with these outer layers and when these layers become more discernable (after some iterations), they help to define the layers in between.
A good approach is to limit the top level to rather trivial controller-like assembly/combination of functionalities produced by the next level. If a slightly different service is required, it should be easy to addapt or extend the top layer as it is not a heavy and complex layer. The top layer should appear to be conceptualy simple; the next level will be rather generic and highly parameterised.
Consider shifts in requirements ('flexibility', some are forseeable) and how they can be solved with this architecture.
For the bottom layer it is important to find good abstractions (i.e. not model equipment directly but functional types) to assure technology indepence. When equipment changes –and it will– only drivers have to be adapted.
Consider technology trends in the Platform Model (processor improvement: 16%/year in speed, 40%/y in costs; RAM: 10%/y in speed, 60%/y in size, and 75%/y in price; disks: 105%/y in price; improvement speed seems to be getting gradually lower).
Analyse according to the layered approach:
- external service request (request for various services), sample scenarios ('use cases') of major service transactions;
- services provided visibly used by external request (service components: connects, mailboxes, file storage and/or data bases, calculation/transformation functions, report generator, etc.);
- internal transaction scenarios for all (major) services, with all required transaction (progress) management;
- resources needed identifies solution components (and their resource management); check for potential bottlenecks, in particular for scarce/expensive resources: processing, memory or disks space, access & communication bandwidth & latency, etc.);
- platform (e.g. equipment: servers, O.S., communication links, protocol stacks, etc.).
The first two points should provide the Service Model, 3 & 4 provide the solution architecture, and the last for the abstract Platform Model.
For each layers (virtual machine model):
- define the purpose of each layer;
- define what functionality each layer should provide (what abstraction); and
- define how each facility in that layer operates in terms of facilities (mechanisms, e.g. API's) provided by the (next) lower layer.
Document above. Check whether:
- each functional component has all data available to operate;
- the processing time for each event/transaction is sufficiently limited;
- function interaction has acceptable (limited) bandwidth;
- the amount of detail is reasonably balanced.
See above sections on Architecture (e.g. Composition-Criteria on interfaces).
Investigate in more detail:
- Start identifying the bottlenecks in the system, e.g. complex processing, high bandwidth interfaces, massive data look-up, etc.
- For each bottleneck try to do at least a feasibility study, modeling or a simulation, or ultimately a prototype.
- Look for (sub)function commonality. Hide data, use abstract operations.
The Architectural Design is finished when:
- the system is fully modeled;
- all major events/transactions/data structures/functions have been identified (not fully elaborated, but sufficiently to show feasibility);
- risky/critical/vague areas sufficiently investigated (no problems foreseen);
- documentation is finished.
Note: Be aware that automating/mechanising a system affects/changes the problem.
=O=