Engineering Leadership Software Development

Published Jul 25, 2026

The Hidden Cost of Synchronous APIs

Synchronous APIs look simple until your product's latency and availability become whoever answers the other end of the wire. Production Notes #08.

Quick Brown Fox8 min

Production Notes #08 · Part of Binary and Beyond. LinkedIn newsletter edition follows.

The simplest integration design is also the most expensive one teams underestimate.

Call the other system. Wait. Map the response. Return success to the user.

It reads cleanly on a whiteboard. One arrow. One round trip. No queue to explain to a product manager. No "eventually" in the acceptance criteria. The demo finishes in under a second when every dependency is healthy and nearby.

Then production happens, and the cost of that simplicity shows up as someone else's latency, someone else's outage, and a product that cannot tell the truth about time.

That is the hidden cost of synchronous APIs: they borrow reliability and speed from every system they touch, then pretend the debt does not exist.

The checkout that waited on a warehouse

Picture a mid-market retailer that wants "real-time stock confirmation" at checkout.

The commerce platform calls the warehouse availability API on submit.

If the warehouse says yes, the order is accepted.

If the warehouse says no, the customer sees an error.

If the warehouse is slow, the customer waits.

On a quiet Tuesday, the warehouse answers in 180ms. Everyone is happy. The ticket is closed as done.

On a sale afternoon, the warehouse p95 climbs to four seconds. Checkout threads pile up. The commerce app's connection pool thins out. Pages that never called the warehouse start timing out because workers are stuck waiting on stock checks.

Support does not get a warehouse outage ticket.

They get "site is slow" tickets.

Finance does not see a clean failure. They see abandoned carts and duplicate submits from customers who clicked twice while the spinner spun.

Nobody designed a distributed failure. They designed a synchronous call and inherited one.

Production Notes #02 argued that every integration is a distributed system. A synchronous API is how that fact enters the request path of your product: for the duration of the call, your availability is their availability, and your latency budget is their latency budget.

Sync is not wrong. Sync is a coupling decision.

Synchronous request/response is a legitimate tool.

You need it when the next user-visible step cannot proceed without an authoritative answer: payment authorisation, fraud decision, login, inventory reservation that must be exclusive before you promise a unit.

The mistake is treating sync as the default shape of every integration because it is easy to code and easy to demo.

Sync says three things at once:

I need the answer now to continue this user journey.
I will hold my resources (threads, connections, user attention) until you answer or I time out.
Your failure modes are my failure modes for that window.

Those are product commitments wearing an HTTP method.

When teams say "we will just call their API," they are often agreeing to all three without writing any of them down.

Where the cost actually lands

1. Latency becomes a shared tax

Every synchronous hop adds variance you do not control.

Your p95 is no longer your code's p95. It is the convolution of your code, the network, their auth layer, their datastore, their noisy neighbour, and their deploy schedule.

Product still sells "instant." Engineering still reports "API latency." Customers experience the sum.

Teams then optimise the wrong layer. They cache aggressively. They raise timeouts. They add retries (Production Notes #05). Each move can help a metric and hurt the business: stale answers, longer hangs, duplicate side effects.

2. Partial failure becomes a user-facing hang

Production Notes #04 covered brownouts: systems that fail in pieces while the rest stays green.

A synchronous call turns a brownout next door into a stall at home. The warehouse is "mostly up." Your checkout is "mostly waiting." From the customer's seat, those feel identical: the button did nothing useful.

Async designs make time visible. Sync designs hide time until it hurts.

3. Capacity collapses inward

Workers waiting on remote I/O are workers not serving local work.

I have watched healthy services tip over because a dependency slowed, not because the dependency fully died. The dependency returned 200 slowly. That is enough. Thread pools empty. Health checks fail. Autoscaling adds more waiters. The blast radius is your process model, not their status page.

4. Truth gets deferred until timeout theatre

When the call finally fails, the product still has to decide what is true: did the remote side commit? Did we? Who reconciles?

Sync does not remove that question. It concentrates it into a timeout handler that was never designed as a state machine. Production Notes #06 matters here because the user will retry the button while your worker is still waiting on the first attempt.

The meeting where sync wins for the wrong reason

In planning, sync usually wins on narrative:

"The user needs to know immediately."
"Async is more complex."
"We do not want eventual consistency."

Often the real preference is: we do not want to design a pending state, a notification path, or a reconciliation job.

So the complexity does not disappear. It moves into production incidents, support macros, and midnight fixes.

Async is not automatically better. A poorly designed queue with silent poison messages is worse than a boring synchronous read.

The honest comparison is:

Sync: complexity in the request path (latency, coupling, timeout semantics).
Async: complexity in the state model (pending, failed, reconciled, visible to the user).

Enterprise software always pays one of those bills. Sync just lets you postpone the invoice until a partner has a bad day.

When synchronous is the right cost

Prefer sync when:

The remote answer is authoritative for a commitment you are about to make to a human or another system.
The operation is safe to wait on (bounded latency, clear SLA, known failure contract).
You have an explicit degraded mode if the call cannot complete (refuse the action, not hang).
Retries are either forbidden or made safe with idempotency keys.

Prefer async (or sync with a local decision + async confirmation) when:

The work is a side effect (notify warehouse, enrich CRM, update search index).
Partner latency is unbounded or political (their "real-time" is your afternoon).
The user can accept accepted / processing / failed as first-class states.
Fan-out would multiply sync hops into a latency tax the product cannot afford.

A useful test: if product cannot describe what the screen shows while waiting, you are not ready for sync. You are ready for a spinner and a hope.

A design checklist before the next "just call their API"

I use these questions when a feature reaches for a synchronous dependency:

What user-visible step is blocked on this answer? If none, why is it on the request path?
What is our latency budget, and whose SLA consumes it? Write the numbers.
What do we do at 500ms, 2s, and 10s? Timeout is a product decision, not only a client setting.
If they are slow but not down, do we shed load, serve stale, or refuse? Brownout policy first.
If the response never returns, what is true in our system? Durable state, not a log line.
If the user double-submits while we wait, what happens twice? Idempotency before retry.
Could this be "local accept + async confirm" without lying? Many "real-time" needs are "fast enough acknowledgement" needs.

If the answers are vague, the API call is not simple. The diagram is.

AI workflows inherit the same trap

Tool-calling agents love synchronous tools: call CRM, wait, call policy service, wait, call ticket system, wait.

Each wait is a coupling. Each slow tool is a stalled conversation. Each timeout is a half-finished side effect with a confident tone.

Production AI delivery that chains synchronous tools without budgets, circuit breakers, and explicit degrade/refuse rules is just integration theatre with a chat UI. The hidden cost did not change. The blast radius moved into the answer the customer reads.

A different standard for "real-time"

Real-time is not a wire protocol.

It is a promise about how long a human may wait, and what is allowed to be undecided during that wait.

Synchronous APIs are one way to keep that promise. They are also a way to accidentally promise that every dependency you touch will stay fast forever.

The teams I trust do not ban sync. They price it.

They put partner latency on the same page as UX copy. They treat timeouts as states. They move side effects off the request path until someone can name why the user must wait.

Enterprise integrations do not get expensive because HTTP is hard.

They get expensive when "wait for the other system" becomes the architecture without anyone admitting that waiting is the product.