Integrations: From Simple Data Transfer To Modern Composable Architectures
Email Newsletter
Weekly tips on front-end & UX.
Trusted by 200,000+ folks.
This article has been kindly supported by our dear friends at Storyblok, a friendly headless CMS with a visual editor, nested components and customizable content blocks for websites and apps. Thank you!
When computers first started talking to each other, the methods were remarkably simple. In the early days of the Internet, systems exchanged files via FTP or communicated via raw TCP/IP sockets. This direct approach worked well for simple use cases but quickly showed its limitations as applications grew more complex.
# Basic socket server example
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 12345))
server_socket.listen(1)
while True:
connection, address = server_socket.accept()
data = connection.recv(1024)
# Process data
connection.send(response)
The real breakthrough in enabling complex communication between computers on a network came with the introduction of Remote Procedure Calls (RPC) in the 1980s. RPC allowed developers to call procedures on remote systems as if they were local functions, abstracting away the complexity of network communication. This pattern laid the foundation for many of the modern integration approaches we use today.
At its core, RPC implements a client-server model where the client prepares and serializes a procedure call with parameters, sends the message to a remote server, the server deserializes and executes the procedure, and then sends the response back to the client.
Here’s a simplified example using Python’s XML-RPC.
# Server
from xmlrpc.server import SimpleXMLRPCServer
def calculate_total(items):
return sum(items)
server = SimpleXMLRPCServer(("localhost", 8000))
server.register_function(calculate_total)
server.serve_forever()
# Client
import xmlrpc.client
proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")
try:
result = proxy.calculate_total([1, 2, 3, 4, 5])
except ConnectionError:
print("Network error occurred")
RPC can operate in both synchronous (blocking) and asynchronous modes.
Modern implementations such as gRPC support streaming and bi-directional communication. In the example below, we define a gRPC service called Calculator with two RPC methods, Calculate, which takes a Numbers message and returns a Result message, and CalculateStream, which sends a stream of Result messages in response.
// protobuf
service Calculator {
rpc Calculate(Numbers) returns (Result);
rpc CalculateStream(Numbers) returns (stream Result);
}
Modern Integrations: The Rise Of Web Services And SOA
The late 1990s and early 2000s saw the emergence of Web Services and Service-Oriented Architecture (SOA). SOAP (Simple Object Access Protocol) became the standard for enterprise integration, introducing a more structured approach to system communication.
<?xml version="1.0"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<soap:Header>
</soap:Header>
<soap:Body>
<m:GetStockPrice xmlns:m="http://www.example.org/stock">
<m:StockName>IBM</m:StockName>
</m:GetStockPrice>
</soap:Body>
</soap:Envelope>
While SOAP provided robust enterprise features, its complexity, and verbosity led to the development of simpler alternatives, especially the REST APIs that dominate Web services communication today.
But REST is not alone. Let’s have a look at some modern integration patterns.
RESTful APIs
REST (Representational State Transfer) has become the de facto standard for Web APIs, providing a simple, stateless approach to manipulating resources. Its simplicity and HTTP-based nature make it ideal for web applications.
First defined by Roy Fielding in 2000 as an architectural style on top of the Web’s standard protocols, its constraints align perfectly with the goals of the modern Web, such as performance, scalability, reliability, and visibility: client and server separated by an interface and loosely coupled, stateless communication, cacheable responses.
In modern applications, the most common implementations of the REST protocol are based on the JSON format, which is used to encode messages for requests and responses.
// Request
async function fetchUserData() {
const response = await fetch('https://api.example.com/users/123');
const userData = await response.json();
return userData;
}
// Response
{
"id": "123",
"name": "John Doe",
"_links": {
"self": { "href": "/users/123" },
"orders": { "href": "/users/123/orders" },
"preferences": { "href": "/users/123/preferences" }
}
}
GraphQL
GraphQL emerged from Facebook’s internal development needs in 2012 before being open-sourced in 2015. Born out of the challenges of building complex mobile applications, it addressed limitations in traditional REST APIs, particularly the issues of over-fetching and under-fetching data.
At its core, GraphQL is a query language and runtime that provides a type system and declarative data fetching, allowing the client to specify exactly what it wants to fetch from the server.
// graphql
type User {
id: ID!
name: String!
email: String!
posts: [Post!]!
}
type Post {
id: ID!
title: String!
content: String!
author: User!
publishDate: String!
}
query GetUserWithPosts {
user(id: "123") {
name
posts(last: 3) {
title
publishDate
}
}
}
Often used to build complex UIs with nested data structures, mobile applications, or microservices architectures, it has proven effective at handling complex data requirements at scale and offers a growing ecosystem of tools.
Webhooks
Modern applications often require real-time updates. For example, e-commerce apps need to update inventory levels when a purchase is made, or content management apps need to refresh cached content when a document is edited. Traditional request-response models can struggle to meet these demands because they rely on clients’ polling servers for updates, which is inefficient and resource-intensive.
Webhooks and event-driven architectures address these needs more effectively. Webhooks let servers send real-time notifications to clients or other systems when specific events happen. This reduces the need for continuous polling. Event-driven architectures go further by decoupling application components. Services can publish and subscribe to events asynchronously, and this makes the system more scalable, responsive, and simpler.
import fastify from 'fastify';
const server = fastify();
server.post('/webhook', async (request, reply) => {
const event = request.body;
if (event.type === 'content.published') {
await refreshCache();
}
return reply.code(200).send();
});
This is a simple Node.js function that uses Fastify to set up a web server. It responds to the endpoint /webhook, checks the type field of the JSON request, and refreshes a cache if the event is of type content.published.
With all this background information and technical knowledge, it’s easier to picture the current state of web application development, where a single, monolithic app is no longer the answer to business needs, but a new paradigm has emerged: Composable Architecture.
Composable Architecture And Headless CMSs
This evolution has led us to the concept of composable architecture, where applications are built by combining specialized services. This is where headless CMS solutions have a clear advantage, serving as the perfect example of how modern integration patterns come together.
Headless CMS platforms separate content management from content presentation, allowing you to build specialized frontends relying on a fully-featured content backend. This decoupling facilitates content reuse, independent scaling, and the flexibility to use a dedicated technology or service for each part of the system.
Take Storyblok as an example. Storyblok is a headless CMS designed to help developers build flexible, scalable, and composable applications. Content is exposed via API, REST, or GraphQL; it offers a long list of events that can trigger a webhook. Editors are happy with a great Visual Editor, where they can see changes in real time, and many integrations are available out-of-the-box via a marketplace.
Imagine this ContentDeliveryService in your app, where you can interact with Storyblok’s REST API using the open source JS Client:
import StoryblokClient from "storyblok-js-client";
class ContentDeliveryService {
constructor(private storyblok: StoryblokClient) {}
async getPageContent(slug: string) {
const { data } = await this.storyblok.get(`cdn/stories/${slug}`, {
version: 'published',
resolve_relations: 'featured-products.products'
});
return data.story;
}
async getRelatedContent(tags: string[]) {
const { data } = await this.storyblok.get('cdn/stories', {
version: 'published',
with_tag: tags.join(',')
});
return data.stories;
}
}
The last piece of the puzzle is a real example of integration.
Again, many are already available in the Storyblok marketplace, and you can easily control them from the dashboard. However, to fully leverage the Composable Architecture, we can use the most powerful tool in the developer’s hand: code.
Let’s imagine a modern e-commerce platform that uses Storyblok as its content hub, Shopify for inventory and orders, Algolia for product search, and Stripe for payments.
Once each account is set up and we have our access tokens, we could quickly build a front-end page for our store. This isn’t production-ready code, but just to get a quick idea, let’s use React to build the page for a single product that integrates our services.
First, we should initialize our clients:
import StoryblokClient from "storyblok-js-client";
import { algoliasearch } from "algoliasearch";
import Client from "shopify-buy";
const storyblok = new StoryblokClient({
accessToken: "your_storyblok_token",
});
const algoliaClient = algoliasearch(
"your_algolia_app_id",
"your_algolia_api_key",
);
const shopifyClient = Client.buildClient({
domain: "your-shopify-store.myshopify.com",
storefrontAccessToken: "your_storefront_access_token",
});
Given that we created a blok in Storyblok that holds product information such as the product_id, we could write a component that takes the productSlug, fetches the product content from Storyblok, the inventory data from Shopify, and some related products from the Algolia index:
async function fetchProduct() {
// get product from Storyblok
const { data } = await storyblok.get(`cdn/stories/${productSlug}`);
// fetch inventory from Shopify
const shopifyInventory = await shopifyClient.product.fetch(
data.story.content.product_id
);
// fetch related products using Algolia
const { hits } = await algoliaIndex.search("products", {
filters: `category:${data.story.content.category}`,
});
}
We could then set a simple component state:
const [productData, setProductData] = useState(null);
const [inventory, setInventory] = useState(null);
const [relatedProducts, setRelatedProducts] = useState([]);
useEffect(() =>
// ...
// combine fetchProduct() with setState to update the state
// ...
fetchProduct();
}, [productSlug]);
And return a template with all our data:
<h1>{productData.content.title}</h1>
<p>{productData.content.description}</p>
<h2>Price: ${inventory.variants[0].price}</h2>
<h3>Related Products</h3>
<ul>
{relatedProducts.map((product) => (
<li key={product.objectID}>{product.name}</li>
))}
</ul>
We could then use an event-driven approach and create a server that listens to our shop events and processes the checkout with Stripe (credits to Manuel Spigolon for this tutorial):
const stripe = require('stripe')
module.exports = async function plugin (app, opts) {
const stripeClient = stripe(app.config.STRIPE_PRIVATE_KEY)
server.post('/create-checkout-session', async (request, reply) => {
const session = await stripeClient.checkout.sessions.create({
line_items: [...], // from request.body
mode: 'payment',
success_url: "https://your-site.com/success",
cancel_url: "https://your-site.com/cancel",
})
return reply.redirect(303, session.url)
})
// ...
And with this approach, each service is independent of the others, which helps us achieve our business goals (performance, scalability, flexibility) with a good developer experience and a smaller and simpler application that’s easier to maintain.
Conclusion
The integration between headless CMSs and modern web services represents the current and future state of high-performance web applications. By using specialized, decoupled services, developers can focus on business logic and user experience. A composable ecosystem is not only modular but also resilient to the evolving needs of the modern enterprise.
These integrations highlight the importance of mastering API-driven architectures and understanding how different tools can harmoniously fit into a larger tech stack.
“
If you want to dive deeper into the integrations you can build with Storyblok and other services, check out Storyblok’s integrations page. You can also take your projects further by creating your own plugins with Storyblok’s plugin development resources.
(yk)