Architectural Significance Criteria and Some Core Decisions Required

What is important when analyzing and designing software architectures, and what is not? This post proposes 5+2 criteria for the architectural significance of requirements (and other artifacts) and applies them to several examples. It also collects a few architectural decisions to be made early due to their impact, cost and risk (an outcome of ECSA 2020).

Motivation

Let’s assume it is the first day for you on a project that has been running for a while (for instance, one that modernizes some legacy system); you might be an external consultant that has just been hired or have switched role and/or project internally. What to work on first?

As a software engineer with architecture responsibilities, I want to prioritize technical issues quickly so that architecturally significant issues are addressed at their most responsible moment and we do not have to revise decisions and designs unnecessarily later. When doing so, I want to be as objective as possible to avoid that loud voices receive more attention than they deserve so that the truly urgent and important things are tackled first.

The time budget per issue is one to two minutes at most, as it is not unusual to be confronted with 10 to 100 issues per day — thanks to email and auto-notifications in tools such as Kanban boards, issue trackers and source code management systems. Modern world! 😉

Seven Criteria for Architectural Significance

So much for motivation. Starting from insights from peers and then reflecting on my own heuristics, I ended up with seven criteria to assess whether issues qualify as Architecturally Significant Requirements (ASRs):

1. The requirement is directly associated with high business value (benefit vs. cost) or business risk.¹
2. The requirement is a concern of a particularly important stakeholder such as the project sponsor or an external compliance auditor.
3. The requirement includes runtime Quality-of-Service (QoS) characteristics (such as performance needs) that deviate from those already satisfied by the evolving architecture substantially.
4. The requirement causes new or deals with one or more existing external dependencies that might have unpredictable, unreliable and/or uncontrollable behavior.²
5. The requirement has a cross-cutting nature and therefore affects multiple parts of the system and their interactions; it may even have system-wide impact, short term and/or in the long run (examples: security, monitoring).³
6. The requirement has a First-of-a-Kind (FOAK) character: For instance, this team has never built a component or subsystem that satisfies this particular requirement before.
7. The requirement has been troublesome and caused critical situations, budget overruns or client dissatisfaction on a previous project in a similar context.

The list is ordered logically, from project goals and key players to analysis and design elements to organizational matters; this outside-in order does not imply that criterion 2 has less weight than criterion 1 or 3 (and so on). Criteria 1 to 5 are somewhat more objective and easier to agree upon and generalize than criteria 6 and 7, which are highly context-specific.

I have been using and teaching these criteria for several years now; I do not claim that they are complete, but picked the ones that I deem most useful.⁴

A Scoring/Assessment Template (ASR Test)

A table is an obvious format for the resulting seven-criteria ASR Test:

One option to fill the cells in such a table are basic values such as Y = yes, N = no, ? = unknown; if you prefer, you can also enter a bit more elaborate ones such as H, M, L and n/a (acronym for “not applicable”). If you cannot agree on the significance of a particular requirement immediately, do not hesitate to mark it as open and track it as a new backlog item (or maintain a separate parking lot of open issues to be reconsidered periodically).

Instances of one criterion may appear several times when assessing a single requirement. For example, criterion 4 (external dependencies) is met twice if the splitting of user story unveils that two additional public Web APIs must be consumed to realize the story.

I do not fill out the table explicitly every time I screen and filter technical issues, but run the test rather (sub-)consciously while reading any issue that grabs my attention (in a chat message, an email, a Trello board and so on). If you do want to create an explicit representation, a project wiki, team space or other collaboration tool may host it.

Note: The ASR Test is not a quantitative tool, a weight calculator; relevance assessments are always qualitative, subjective and context-depending; the test makes formerly tacit knowledge somewhat explicit (“worst first”). Do not spend more time on assessing relevance than on tackling and solving issues!

Examples and Light Tool Support: miro Board

The following table scores some requirements and constraints according to their architectural significance:

For your convenience, I created a board in the online whiteboard platform miro for this table format (first five ASR criteria only); feel free to use and/or extend it (under a Creative Commons license): https://miro.com/app/board/o9J_kl49CmA=/. Please contact me if you want to get access to the miro template at https://miro.com/app/board/o9J_kl44WKk=/ too.

Here is an example of a filled-out miro template (issues in German, sorry for that!):

Based on the test result, I’d probably look at the payment change request in the online shop first and the design of the security credential cache next. I’ll not get involved with the class renaming (unless I have to review the change w.r.t. project-wide coding guidelines).

ECSA Working Sessions 2020: ASRs and Core Decisions

The Industry Program of European Conference on Software Architecture (ECSA) 2020 that took place online triggered me to post my already existing ASR Test here and now: In one of the Working Sessions, my program co-chair Anton Jansen asked “Where and how to draw the line of architecting (and) decision making?”, and I replied “Can ASRs help? One of the participants commented that in his organization, ASRs are usually referred to as drivers, with business, functional, quality and constraints (see this presentation from Fraunhofer IESE for details).

Back to original question in the session, attendees nominated the following architectural decisions that have a rather early Most Responsible Moment (MRM) and qualify as examples of “core decisions” (a topic for a future post):

• “What minimal functionality is expected of a product? What regulations to adhere to?”
• “Which architectural style is used?”
• “What technology stacks are supported?”
• “How is integration done? What are the minimum integration options a product should have?”
• “What governance structure should be in place for a product? What is the process for a team to contribute their project to the main product?”
• “Compiler that teams must use, development environment, operating system version.”

Certainly not complete, but not a bad start towards a catalog of such core decisions either. Other topics we touched upon (2x30 mins run out so quickly!) were service granularity and ownership of edge hardware and software in cloud, edge and fog computing.⁵

Concluding Thoughts

Here are the take-away messages from this post and some additional advice/remarks:

• When deciding which technical issues to focus on, consider value, cost and risk implications. Follow an outside-in approach, starting from project goals and external interfaces that you will have to consume (and therefore rely on).
• The “five plus two” ASR criteria that work best for me are:

Business value and risk; key stakeholder concern; unusual quality-of-service requirement (at least one order of magnitude more advanced than previous ones); external dependencies that are uncontrollable, unpredictable or unreliable; cross-cutting, system wide impact

First-of-a-kind character (novelty for team); bad experience and trouble in the past

• I use the criteria not only when joining a project (as the motivating user story at the start of this post assumes), but also when returning to a project after an external event and checking my mailbox, when planning my week/day and so on.
• Balance tactic progress (for instance, indicated by iteration velocity and long-term thinking and development of the system; some non-functional properties only become relevant (and might start to hurt) in the long term.
• The ASR test presented in this post works for requirements (mostly non-functional but also functional), design elements (components and connectors, viewpoint depending) and architectural decisions.

ASR scoring is not an exact science, but qualifies as a “mighty” method element imho (Michael Keeling’s ECSA 2020 keynote has a definition and requirements/success criteria).

Do the above ASR criteria work for you? Is a criterion (test question) missing? Contact me!

– Olaf (a.k.a. ZIO)

PS: The ASR criteria and test are also available as a story on Medium.

Acknowledgements

I would like to thank Peter Eeles for his initial thoughts on qualifying architectural significance, Jasmin Jahic and Mirko Stocker for reviews of drafts of this post, Anton Jansen, Somayeh Malakuti and all other ECSA Working Session presenters and attendees for the lively discussions. I also thank Gerald Reif and HSR/OST Application Architecture students 2017 to 2020 for their input, feedback and lively discussions on the topic of architectural significance.

Notes