APIs should get to the POINT

This post proposes five principles for API design, summarized in the backronym POINT: purposeful, style-oriented, isolated, channel-neutral and T-shaped.

Context: Web API Design and Evolution (WADE)

Application Programming Interfaces (APIs) have been around for a long time, both local ones inside modular programs and remote APIs connecting operating system processes and distributed systems. Message-based technologies dominate the remote API realm; HTTP (more or less RESTful), gRPC and GraphQL seem to top the popularity list at present.

Any API, local or remote, should satisfy an actual client information or integration need. When delivering related functionality, three types of quality attributes are particularly relevant (I’d call these three categories “DOM triad” if the acronym wasn’t already taken 😉):

• Developmental qualities: APIs should be straightforward to learn and easily consumable for developers and their app(lication)s, collectively referred to as positive Developer Experience (DX).
• Operational qualities: APIs should meet the performance and reliability requirements that have been stated for them, including availability and API security.
• Managerial qualities: APIs should be maintainable and evolvable over time, preferably being both extensible and backward compatible, so that changes are possible but do not cause existing clients to break. Agility and stability have to be balanced.

While we can probably agree on these quality goals as an API incarnation of the general Functionality, Usability, Reliability, Performance, Supportability (FURPS) taxonomy for (non-)functional requirements rather easily, how do we get a new API there?

One way is to:

Establish API design principles as an intermediate step to drive responsible decisions later.

Getting local APIs right is one thing;¹ for remote APIs, the fallacies of distributed computing come into play; for instance, networks cannot be assumed to be reliable.²

Enter: Architectural Principles

Principles both qualify as requirements and design elements (and decisions about those). I go with Eoin Woods’ definition in his presentation “Using Software Architecture Principles in Practice”: “an architectural principle is a declarative statement made with the intention to guide architectural design decisions in order to achieve one or more qualities of a system”. I already listed some types of desired qualities in our API context; the API design decisions deal with number of endpoints, format and content of request and response messages, message exchange technologies, API versioning and so on.

Following Eoin’s definition, principles can guide quality goal-oriented decision making, constraining the solution space without imposing any particular design prematurely:

The loose coupling principle, for instance, applies to APIs and most other designs, as it makes the end-to-end system less brittle and its parts easier to evolve independently. Software engineering principles such as high cohesion and single responsibility per stakeholder can also guide the identification of API endpoints and the design of its operations. Such principles may be established on all responsibility levels, from entire organization (enterprise) to programs and portfolios to individual teams and products.

POINT: Five Principles for API Design

Let’s try to identify some additional, API-specific principles now. I propose a remote API should get (and stick) to five POINTs by default to get its job done, meeting or exceeding client expectations along the way:

1. P – an API must be purposeful and driven by the information needs of its clients, expressed as use cases, user stories or similar. It should be straightforward to answer questions like “why and when would an API client program call this operation?”. The number and flow of API calls required to satisfy the information need should support the client-side data usage paths (for instance, the flows in a single-page application running in a Web browser or in a rich desktop client program).³
2. O – an API should be oriented according to the form and concepts mandated by the chosen architectural style and integration technology (e.g., resources in RESTful HTTP or ports in WSDL/SOAP Web services); it may expose a view on domain objects and services via facades.⁴
3. I – each API call should be as isolated as possible, i.e., be free of unexpected, undesired side effects. It should not interfere with calls to other operations in the same or other APIs. Statelessness helps with that; establishing adequate pre- and postconditions does so too.⁵
4. N – an API should be channel-neutral, not be optimized solely for any particular stakeholder group and/or client technology such as Web applications (of various kinds) or rich clients (in mobile phones, on PCs). Typically APIs serve multiple clients, but this is not a must; several 1:1 customer-supplier relationships might be easier to manage than one open 1:n interface that pleases many, possibly diverging stakeholders. That said, some APIs specialize on a particular client technology; this might be the single actual purpose of an API.⁶
5. T – an API should be T-shaped in its scope and offer broad and deep functionality; for instance, it may be organized according to the overview-details metaphor and support both search-and-iterate-through-results as well as direct-lookup-via-keys in its retrieval operations. It is possible to provide multiple Ts per API, or move the vertical bar of the T upon consumer demand.⁷

Can we be confident that POINT helps us achieve the qualities identified above? The P corresponds to the functionality-F in the FURPS taxonomy (introduced above), and a T-shape brings usability. The O promotes supportability, a managerial quality. I improves the API client developer experience and simplifies testing (another developmental quality); it also has a positive impact on consistency, scalability and reliability (operational qualities ). N is good for supportability too. The five POINTs do not claim to be mutually exclusive or collectively exhaustive; feel free to add principles (I’ll give an outlook to two more further down).

Examples of POINT in Action

Let’s inspect the Lakeside Mutual sample application and some other APIs.

Cesare Pautasso and his team collected and visualized the URI-verb structure of 93 Web APIs in their e-book “Beautiful APIs”. You’ll find plenty of T-shaped APIs in it, for instance:

This resource path tree is part of the tvMaze user API, offering several Ts to inquire about followed people, shows and Web channels as well as voted shows and episodes, both generally (collection resources, plural resource names) and individually (_id resources).

How to become POINTed?

To get to the POINT during API design, consider to:

• Follow one or more recognized analysis and design methods to identify API call candidates and flesh out their invocation syntax. The Design Practice Repository (DPR) that I co-created collects elements from such methods.
• Apply a pattern-oriented software (and API) architecture approach. See below for further hints.
• When making design decisions, ask/evaluate whether the chosen option will promote or prevent P, O, I, N and/or T and include this discussion in your definition of done for architectural decisions. When not adhering to a principle, document why.
• Once you have decided for a style orientation, make this evident in the names and structures of the operation messages and API endpoints.
• Foresee one and only one way to execute a use case or user story implementation to make the API lean and easy to use, review and extend.⁸
• Plan the API testing early and adjust the test plans and cases as the design evolves. Add some informal POINT “compliance checks” to the API testing.
• Define an API lifecycle management policy that balances extensibility and backward compatibility; in a EuroPLoP 2019 pattern paper, we discuss conflicting requirements and related tradeoffs, and then report proven solutions resolving these design forces. You also might want to review the API regularly, for instance to analyze usage feedback.

Note: There might be good reasons to break a principle. For instance, empowering clients to come up with new Purposes for existing data implies that the eventual client needs can not be known upfront (see examples above). And a Backend-for-Frontend API deliberately will not be very Neutral.

So the journey from requirements to principles and patterns is (in an API design context):

The figure contains some categories of Microservice API Patterns (MAP). MAP relates to the five POINTs as this: Purpose comes from the application of responsibility patterns (and endpoint identification heuristics). As a pattern language, MAP is agnostic to style Orientation, but draws examples and known uses from REST. Several structural patterns such as Context Representation and some members of the quality patterns category address Isolation. Applying any patterns can help you be channel-Neutral. Both a single general-purpose API and multiple specialized ones require versioning and lifecycle management evolution patterns to become supportable. The right mix of role stereotypes and operation responsibilities yields T-shaped APIs.

What do other API designers recommend?

Blog Posts and Online Articles. Jordan Ambra presents “five golden rules” for Web API design: “Documentation, Stability and Consistency, Flexibility, Security, Ease of Adoption”.⁹ Mathieu Fenniak’s API checklist has 43 entries (some of which are specific to HTTP and REST). James Higginbotham blogs here and here; he suggests job stories to articulate API purpose. You will find very valid POINTs in all of these sources.

API Design Books and Patterns. Mike Amundsen features the P principle as the first section of API stories in “Design and Build Great Web APIs: Robust, Reliable, and Resilient”. API Handyman Arnaud Lauret devotes Chapter 2 of “The Design of Web APIs” to API goals; his API goal canvas establishes P(urpose). Arnaud also provides a handy API review checklist and many examples of good and bad (or more or less POINTed) API designs, touching on N and T.

Most pattern languages focus on design and architecture of API implementations and service infrastructure rather than the actual APIs and service message content; the “Service Design Patterns” book by Rob Daigneau is an exception; its patterns describe several styles that bring O(rientation). Erik Wilde suggests some patterns to achieve “Robust Extensibility”, including “Well-Defined Extension Points” and “Well-Defined Processing Model”. In “Cloud Strategy”, Gregor Hohpe positions principles to connect strategy and decisions and establishes two general ones “use before reuse” and “design from front-to-back” (P). Gregor then also establishes the FROSST principles: “Frugal, Relocatable, Observable, Seamlessly updatable, internally Secured, failure Tolerant” (I).¹⁰

Summary and Outlook

The take-away messages from this post are:

• Quality goals for APIs come in three forms: developmental, operational, and managerial. Note that such goals should be elicited (even in very agile settings); try the SMART Non-Functional Requirements activity in DPR to do so.
• General architectural principles and the API design-specific POINT criteria can guide the architectural decision making: purposeful, style-oriented, isolated from others, channel-neutral, and T-shaped.
• Applying the Microservice API Patterns and other practices increases the chances of meeting the principles. API testing helps to find out whether you succeeded.
• Violating one or more principle is perfectly fine in certain contexts as long as this is a conscious decision and decision rationale is captured. Why decouple a JavaScript Web client from the server-side API provider implementation if these two are developed by the same team at the same time and then packaged and shipped in the same deployment unit?
• Several books and numerous online articles on Web API design send similar and/or complementary messages. Most of these are rather technology-centric; the books I listed under “What Do Other API Designers Say?” also feature engineering approaches that work across technologies. Update: I am working on an ebook featuring many design practices, including stepwise service and API design. The current, 90% complete draft is available at Leanpub.

I do not suggest a closed set of strict rules; the five principles leave room for appropriate solution design and are extensible as well, for instance into POINTed. The e then stands for exemplified (by showing at least three sample request and response messages and providing executable test cases) and the d for documented — calling for explicit API Descriptions, possibly written in a technology-neutral format such as MDSL and published in API developer portals.

Do you agree with my POINTs? Contact me if you have feedback or input. Would you like to steer research on quality-driven API design and evolution? Please fill out this short questionnaire (15 minutes will do). Thanks!

– Olaf (a.k.a. ZIO, MAP author, socadk)

PS: A slightly shorter version of this post is available on Medium.

Acknowledgements

I would like to thank my Microservice API Patterns (MAP) co-authors Mirko Stocker, Daniel Lübke, Cesare Pautasso, and Uwe Zdun for their POINTed feedback and input on our MAPs since late 2016. Oliver Kopp commented on an intermediate draft of this post, and Tammo van Lessen contributed via insightful discussions.

Notes