1. Background
Current digital landscape and rapid transformation are challenging enterprise application teams and providers to find flexible solutions. Applications, systems and processes are changing and there is a need to adapt to changes in technology requirements and create loosely coupled services that provide agility and flexibility.
Currently applications and especially data (MDM, Data mesh) influences architecture and integrations. Previously the enterprise service bus was a widely used method to integrate applications and backends, but more lightweight solutions were needed. That is why API gateways exist. API gateway doesn’t have the overhead of adapters or the integration functionality like an ESB has. API gateway allows decoupling and encapsulation and offers methods and capabilities to control, secure, manage and measure API usage.
ESB integrations tend to create more complex solutions and organisations begin to create monoliths instead of agility. Central teams affect what good reusability and design of services would be, and the heavy cross-communication halted innovation and team agility.
2. How to transform
RESTful APIs are based on JSON. JSON is currently much more popular than SOAP or XML because its syntax is small and lightweight and current tools are supporting JSON. Another significant benefit is that JSON messages can be easily cached or stored into databases. API and integration platforms can store data that has been previously requested, then serve that data upon demand.
ESBs are still valid today but to speed up the development we need to to break up the ESB monolith. Instead of using the centralised dev model, the responsibility of building and exposing services will be given back to the underlying dev teams and domains.
With a decentralised model, domains can evolve at their own speed without holding up other domain development. In this new model, responsibility for business logic, executing process, persisting data, and validating context also goes back to the domains. Using the ESB or API gateway as a (centralised) database will be forbidden; aggregation and orchestration are allowed only within the domain.
API gateway and API management provides an abstraction layer that insulates the concerns of the app experience design away from the constraints and implementation details of the enterprise application functional logic. This abstraction allows the mesh applications to integrate capabilities and data from multiple enterprise applications and other systems and services, while supporting seamless user interactions across multiple channels that support a variety of user experiences. Mesh applications refers to the apps and app architecture that link into a broad mesh of different back-end services to create seamless user experience, and ensures users to have a consistent experience as they switch between devices.
3. How to solve the puzzle: mesh application service architecture
Gartner provides the idea of Mesh application service architecture (MASA). MASA incorporates best practices and patterns from API-centric, service-oriented architecture (SOA) and multichannel user experience (UX) design to build mesh-like applications from multiple apps, APIs and services.
Mesh application service architecture is a distributed architecture where the application comprises multiple fit-for-purpose app experiences provided by front-end UIs and multiple back-end services that provide core application functionality. It supports a heterogeneous and multigrained back-end ecosystem where functionality may come from custom systems, data virtualisation, large systems of record, microservices or enterprise applications.
Some benefits of MASA:
- Agility: A decoupled component architecture enables application teams to adapt to changing business priorities and replace aging components
- Scalability: Scalability lets you efficiently expand or collapse your solution’s capacity for individual features or capabilities in a fine-grained and dynamic manner. Independent services allow you to scale specific capabilities independently of others, whereas large back-end systems require you to launch entire instances of your application just to support increased demand in a narrow scope of functionality.
- Cohesive UX: MASA enables cohesive user experiences by providing touch points across multiple devices and channels, integrated with business functionality
- Extend legacy systems: MASA’s modular service architecture allows you to create new independent app experiences and custom integrations using legacy system capabilities, as well as add capabilities from other services to enhance legacy applications.
- Integration: Using a mediated consumer-centric integration approach provides the ability to connect front-end technologies with enterprise application services, functionality and data using APIs.
- Modernisation: MASA’s distributed nature of abstracted discrete components allows you to modernise individual parts of the application architecture at a pace your organisation can tolerate.
- Innovation: Individual fit-for-purpose apps allow you to select optimal technologies and enables application teams to optimise UI capabilities, support new user/customer channels, and create optimal experiences in the user’s preferred channels.
- Consistency: Using an API gateway to mediate APIs simplifies security and access policy enforcement, traffic management, monitoring and logging by providing a single place to configure policies and monitor the APIs.
- Faster delivery: Decoupling the apps and services from each other using APIs enables independent release cadences. You have the option to release new user-facing capabilities faster, get feedback, adapt and innovate for some apps, while using a slower cadence for apps that rarely change.
- Reduced technical debt: MASA supports the use of open standards technologies and the ability to minimise proprietary implementations that result in technical debt. Additionally, a modular architecture allows teams to iteratively reduce technical debt over time rather than a “big bang” approach, which, realistically, rarely happens.
4. The purpose of API mediation layer
Mediated API architecture allows to create a high degree of scalability, flexibility and agility in enterprise applications. It supports two sets of APIs: inner and outer APIs.
- Inner APIs create the functionality of back-end systems and services. For example some vendor specific application APIs or APIs provided by MDM, or data mesh applications.
- Outer APIs are exposed through your gateway and used to support the needs of the individual app experiences.
- Use a consumer-centric approach rather than the domain of the back-end implementation. This will minimise dependencies and make it easier to implement and maintain the apps and the quality.
- Outer APIs insulates the client apps from the complexities and limitations of the enterprise application APIs, as well as providing a central point for policy enforcement and API management.
Follow these best practices when implementing service architecture:
- Treat APIs like a product
- Apply API mediation and possible legacy support
- Implement effective access management
- Remember security expertise
- Avoid complex or orchestrations in mediation layer
- Avoid direct request to enterprise application
- Remember custom driven mindset
Mesh application service architecture is flexible when the integration is a common driver for organisations. A key strength of MASA is that it supports several different integration approaches, which reduce the barriers inherent to integrating enterprise applications with other systems.
MASA supports following methods:
- Integration middleware: Using ESBs, iPaaS, business process management (BPM) and other integration tools enable orchestration and other integrations between the enterprise application, other services and user experiences.
- Custom integration services: Creating custom services provide the enterprise application with custom integrations through orchestration, specialised processing and custom implementations.
- Directly coupled integrations: Using connectors or point-to-point (P2P) connections from within the enterprise application to integrate with other systems.
- Client app integration: Using custom UI code to integrate data and functionality from the enterprise application, as well as multiple other back-end sources.
5. Business perspective
Business priorities should affect the decisions when approaching the architectural design. So in that sense it is important to define and identify the business priorities. If the priority is to increase flexibility, customisability, dynamic integrations, agility and seamless user experiences the MASA approach will be correct architecture.
Sometimes the priorities conflict, for example lowering the complexity but there is a need to increase flexibility. Then you need to think about which priority is more critical for the business outcome? If flexibility, customisability and delivery of high-value, commercial, differentiating applications are higher priorities, choose MASA.
If those priorities come second to lowering operational complexity, stick with the capabilities of the enterprise application, and avoid significant customisation when possible. If the business is asking for a simple turnkey implementation, but the enterprise application won’t support the required functionality without significant customisation and integrations it is not viable to do a full scale MASA approach. If your skills and technologies are not allowing API mediation and MASA approach or these are not in place and you are not reasonably able to acquire them, you are limited to the capabilities of the traditional approach.
6. Conclusions
This blog refers to how Gartner approaches this integration dilemma (requires a client account). Some API tool vendors might have different approaches. What combines these is seek for flexibility and agility but also support the legacy. Modernisation should be done gradually. Architecture is always something that needs to live according to needs from business and IT. Interesting topic and maybe worth another blog later.