Middleware and Integration Layer Development

Signs your business needs an integration layer

Most companies do not set out to build middleware. They connect two systems with a script, then a third with a Zapier zap, then a fourth with a nightly CSV, and a few years later nobody can explain why an order did not reach the warehouse. Tepia hears the same symptoms again and again:

• Integration scripts run on a server or a laptop that one person understands.
• Every system stores credentials for several other systems.
• A failure is discovered by a customer, not by an alert.
• Adding a new 3PL, store or CRM takes months because every existing connection must change.
• The same customer is transformed in four different ways by four different scripts.

The arithmetic explains why. Eight systems connected point to point can need up to 28 separate links, each with its own mapping and failure behavior. The same eight systems connected through a hub need eight adapters. Tepia builds that hub so each new system is a contained project rather than a ripple through everything you already run.

A typical example is a manufacturer that sells through dealers and its own store. Dealer orders arrive by EDI, store orders through Shopify, and service requests through a technician app, yet all three need the same customer, the same item catalog and the same stock picture from the ERP. With a hub in the middle, Tepia adds a new dealer channel by writing one adapter and reusing every rule that already exists.

What goes inside custom middleware

Middleware is not one product. It is a set of components that Tepia sizes to your volume and risk:

• Adapters for each system, such as NetSuite, Salesforce, Shopify, a WMS or a legacy SQL Server database, which handle authentication, paging and rate limits.
• A canonical data model that defines one shape for customer, order, item and shipment, so adapters translate into and out of a common language.
• Transformation and mapping rules kept as versioned configuration, so a new field does not require rewriting code.
• Managed queues such as Amazon SQS, Azure Service Bus or Google Pub/Sub, which hold messages when a target system is slow or down.
• Idempotency keys so a message delivered twice produces one order, not two.
• Retries with backoff and a dead letter queue for messages that still fail, each visible in a console with a replay action.
• Secrets management in AWS Secrets Manager or Azure Key Vault, so credentials live in one controlled place.
• An API gateway that exposes clean endpoints to partners and to your own mobile apps.

Tepia keeps business rules out of individual adapters wherever possible, so the logic for, say, which warehouse ships an order lives in one place and is tested once.

Point to point, iPaaS or custom middleware

Tepia does not recommend custom middleware by default. The right choice depends on how many systems you connect, how complex the rules are and who needs to change them.

A hybrid is common: Tepia keeps an iPaaS for simple notifications and builds custom middleware for orders, inventory and finance flows. Our Zapier vs custom integration page covers the tipping points in more detail.

Event driven or scheduled: choosing how data moves

Each flow through the middleware needs a trigger, and Tepia chooses it based on how quickly the business needs the result.

Webhooks and events suit anything a customer or employee is waiting on, such as an order placed in a mobile app or a payment confirmed by Stripe. The middleware acknowledges the event immediately, queues it and processes it, which keeps the sending system happy even when the receiving one is slow. Our webhooks and event driven integration page goes deeper on signatures and replay.

Change data capture reads changes straight from a database log, useful for legacy systems that have no API or webhooks. Scheduled polling works for systems that only offer search endpoints, with a stored high water mark so nothing is missed. Batch files remain the right answer for some partners and for EDI. Tepia often mixes all four inside one integration layer, each documented in the flow catalog.

How Tepia approaches middleware development

Tepia uses its six phase process, with a US based project manager and engineering leadership accountable for the architecture.

1. Discovery: an inventory of every existing integration, script and scheduled job, interviews with the people who fix them, and a third party review of each system’s API and limits. Deliverables are an Investigation Summary, an Interview Summary and User Stories such as an operations lead replaying a failed shipment message.
2. Design: the canonical data model, a flow catalog with triggers and owners, a cloud architecture diagram, and wireframes for the monitoring console.
3. Development and Testing: adapters built and tested one at a time, an Alpha running alongside existing scripts in shadow mode, and a Beta that moves flows over one by one with load and failure testing.
4. Training: sessions for IT and operations on reading the console, replaying messages and adding mappings.
5. Launch: retirement of old scripts in a planned order, with rollback ready for each flow.
6. Support: monitoring, bug tracking and new adapters as systems change.

Typically Discovery takes 1 to 3 months, development and QA 2 to 6 months, and launch about a month. Read about our process or talk to Tepia about the systems you need connected.

Frequently asked questions