How software works · Guide for product managers

System design for product managers

Architecture discussions can feel like a different language — queues, caches, replicas, services. You do not need to design systems yourself, but you do need to understand what each building block buys, what it costs, and which product requirements drive those choices. That is what this guide covers.

Guide 5 of 10 in the technical PM path

12 min read · Updated 25 September 2026 · By the TechPMer team

Why a PM should care

Architecture is where product requirements turn into cost, speed and risk:

• Requirements drive architecture. "Real-time updates", "works offline", "99.99% uptime" and "data stays in the EU" each change the design — and the budget.
• Architecture drives delivery speed. A system that is easy to change lets you ship faster for years; one that is too complex slows every feature.
• Trade-offs need product input. Engineers can tell you the options; deciding whether speed, cost or consistency matters most for users is a product call.

Start from requirements, not technology

Good system design starts with questions a PM can answer:

Numbers matter. "Fast" and "scalable" mean nothing; "search results under 500 ms for 10,000 daily users" is something engineers can design for.

The building blocks

Clients, servers and the load balancer

Users' browsers and apps (clients) send requests to servers. When one server is not enough, a load balancer spreads requests across several identical servers. This is horizontal scaling — adding more machines — as opposed to vertical scaling, using a bigger machine.

Databases, replicas and sharding

The database is usually the hardest part to scale. Common steps, roughly in order: better indexes and queries; read replicas (copies that serve reads); caching; and eventually sharding (splitting data across databases, for example by customer). Each step adds complexity, so teams take them only when needed.

Caching

A cache stores the result of expensive work so it can be reused — in the browser, on a CDN, or in a fast in-memory store such as Redis. Caching makes things fast and cheap, and introduces the classic problem: stale data. "Why does the dashboard still show the old number?" is often a caching question. Product input: how stale is acceptable for each screen?

CDN

A content delivery network keeps copies of static files (images, scripts, sometimes whole pages) on servers around the world, close to users. It is one of the cheapest big performance wins for global products.

Queues and background jobs

Some work should not happen while the user waits: sending emails, generating reports, processing uploads, calling slow AI models. A queue holds these tasks and workers process them in the background. The product consequence: the result arrives later, so you need states like "processing…", notifications, and a plan for failures and retries.

Services and APIs

Parts of the system talk to each other through APIs. Third-party services — payments, email, search, AI — are also parts of your architecture, with their own limits, costs and outages. See APIs for product managers.

Monolith, modular monolith or microservices?

The common mistake is adopting microservices too early because large tech companies use them. For most products, a well-structured monolith is faster to build, cheaper to run and easier to change. Microservices mainly solve an organisational problem — many teams needing to ship independently.

Serverless

Serverless platforms run your code on demand without you managing servers, and charge per use. They are excellent for spiky or low traffic and small teams. Trade-offs include cold starts (a slow first request), time limits per request, and costs that can grow unexpectedly at high volume.

Reliability and failure

Everything fails eventually — servers, networks, third-party APIs. Reliable systems plan for it:

• Timeouts and retries so one slow dependency does not freeze everything.
• Graceful degradation: if recommendations are down, show the page without them.
• Redundancy: more than one server, backups, sometimes more than one region.
• Monitoring and alerts so the team knows before customers tell them.

Uptime targets have real cost. 99.9% allows about 8.8 hours of downtime a year; 99.99% allows about 53 minutes — and usually costs far more to achieve. Choose deliberately.

Consistency trade-offs

In distributed systems you often trade perfect consistency for speed or availability. "Eventually consistent" means different parts of the system may briefly disagree — a like count that updates a second later. That is fine for likes and not fine for bank balances. Deciding which data needs strict consistency is partly a product decision.

Technical debt

Technical debt is the future cost of shortcuts taken today. Some is deliberate and healthy — shipping an MVP fast. It becomes a problem when nobody tracks it and every new feature pays "interest" in slower delivery and more bugs. Useful PM habits: ask engineers to name the debt a shortcut creates, keep a visible list, and reserve regular capacity to pay it down.

What engineers may tell you

• "We need to cache this, but then it can be up to five minutes stale." A trade-off to decide with them.
• "That should be async." Move it to a background job; the user gets the result later.
• "It's a single point of failure." If that one component fails, everything fails.
• "This doesn't scale past X." Fine for now, but a redesign will be needed at a known threshold — ask when you might reach it.
• "Let's not split this into a service yet." Keeping it in the monolith is simpler until there is a clear reason.

Questions a good technical PM asks

1. Which of our requirements is driving this design choice?
2. What is the simplest design that meets the requirement for the next 12 months?
3. What happens when this dependency is slow or down?
4. How stale can this data be?
5. What will this cost to run at 10× today's usage?
6. What debt are we taking on, and when would we pay it back?

Red flags

• Choosing an architecture because it is fashionable rather than because a requirement demands it.
• No monitoring or alerts on critical flows.
• "It will never go down" or "it will scale infinitely".
• Every feature requires changing five services.
• Performance targets that exist only as "it should be fast".

System design with AI coding agents

AI agents will happily generate a complex architecture if you ask for "a scalable, production-grade system". For most projects, ask instead for the simplest architecture that meets explicit requirements, and make the agent explain its trade-offs:

Propose an architecture for this app. Requirements: ~1,000 users, pages under 1 s,
data may be up to 1 minute stale on the dashboard, emails can be sent async.
Prefer the simplest option (one app + managed database + background job).
List the trade-offs and what would need to change at 50× the users.

Try it yourself

Pick one feature you know well and sketch it as boxes and arrows: client, server, database, any third-party services, any background jobs. Then mark where it would break first if usage grew ten times. Bring the sketch to your next conversation with engineers — it is a much better starting point than "is it scalable?".

In the TechPMer course, week 4 (“Choosing the Stack — SPA, SSR, SEO and Frameworks”) applies these trade-offs to your own project before you build it.