Mapping classic latency patterns onto the agent request path

A Generative Programmer post adapts 19 distributed-systems latency patterns to AI apps, focusing on everything outside the model call.

If your agent feels slow, the model may not be the whole story. A post on The Generative Programmer, shared by @bibryam on X on October 25, 2026, sets out 19 latency patterns in four categories. It adapts ideas from Pekka Enberg's book Latency and focuses on non-model latency: RAG retrieval, databases, model gateways, MCP or other tool servers, and orchestration loops.

The post's starting point is that "a fast database query or LLM call does not prove that the application is fast." It says to measure the full path as a distribution, then find what controls the percentile you care about. Each pattern comes with when to use it and its main trade-off. Examples include:

• Locality: colocate chatty dependencies, partition state by tenant or session, and cache repeated retrieval, schema discovery or policy lookups.
• Work reduction: use regular code or indexed search instead of a model call, reuse tool connections, and cap how much tool output enters the prompt. The post warns that trimming too much context can force an extra model round trip.
• Concurrent execution: run independent tool queries in parallel, and stream partial results when they're useful on their own.

To find the slow segment, you need traces. SentryML describes OpenTelemetry GenAI metrics for time to first chunk and overall operation duration. It also notes the conventions are still at Development stability. Lyzr's guide walks through an example agent trace where a slow finance API took up more than half the run. One Tempo walkthrough adds a warning: if you nest tool spans under the model call that requested them, tool latency looks like model latency.

The post is commentary rather than new data, and we found no independent reaction to it yet.

Sources

• Latency Patterns for Faster AI Applications · The Generative Programmer
• OpenTelemetry GenAI Semantic Conventions Explained (2026) · Lyzr AI
• OpenTelemetry GenAI Semantic Conventions: 2026 Status Guide · SentryML
• OpenTelemetry GenAI Semantic Conventions: Agent Spans in Tempo · Matthews Wong Portfolio