Pinned Tweet
Most backend engineers learn in fragments
A tutorial here,
A Stack Overflow answer there.
I spent 10 weeks building the complete picture.
APIs → Caching → Queues → Load Balancing → Microservices → DB Scaling → API Gateway → Monitoring → Security → CI/CD
10 weeks, 10 articles, One foundation.
Full series + all Medium articles in first reply 👇
Ritesh Roushan retweeted
every AI giant has a different strategy,
who do you think will win the AI race: OpenAI, Anthropic, Google, Meta, xAI?
every AI giant has a different strategy,
who do you think will win the AI race: OpenAI, Anthropic, Google, Meta, xAI?
Ritesh Roushan retweeted
two devs, same deadline
A) spends 5 hours coding manually.
B) ships in 1 hour using AI and verifies everything.
who's the better engineer?
two devs, same deadline
A) spends 5 hours coding manually.
B) ships in 1 hour using AI and verifies everything.
who's the better engineer?
Ritesh Roushan retweeted
which developer would you hire?
A) uses AI to build faster
B) writes everything from scratch
speed or fundamentals?
which developer would you hire?
A) uses AI to build faster
B) writes everything from scratch
speed or fundamentals?
Ritesh Roushan retweeted
System Design: How would you build an AI agent orchestration platform?
Imagine asking 3 AI agents to complete one task:
→ Agent 1 writes the code.
→ Agent 2 reviews it.
→ Agent 3 runs tests before merging.
Sounds simple, right?
The real challenge is coordinating everything reliably.
1. Orchestration Layer
- A central orchestrator assigns tasks, tracks workflow state, and coordinates agents.
2. Workflow Engine
- Represent dependencies as a DAG.
- Code writing must finish before review starts, and tests run after approval.
- Independent tasks can execute in parallel.
Each task follows a state machine:
"PENDING → READY → RUNNING → COMPLETED"
On failure:
"RUNNING → RETRY_WAIT → READY"
If retries are exhausted, the task moves to "FAILED".
Persist state transitions so workflows can recover after crashes.
A task becomes "READY" only when its dependencies are satisfied.
3. Execution Queue
- Use Kafka or a durable job queue to distribute tasks to workers.
- Workers claim jobs, execute them, and report results.
- Use idempotency keys to prevent duplicate processing.
4. Failure Handling
Failures are inevitable. Design for them.
- Transient errors: Retry with exponential backoff and jitter.
- Repeated failures: Move tasks to a dead-letter queue for investigation.
- Worker crashes: Use leases and heartbeats to detect abandoned tasks and safely reassign them.
- Model/API timeouts: Apply timeouts, retry limits, and circuit breakers.
- Partial failures: Persist completed outputs so successful agents don't need to rerun unnecessarily.
- Invalid agent output: Validate results before allowing dependent tasks to proceed.
Never assume a retry means the previous execution didn't happen.
Use idempotency and durable checkpoints to avoid duplicate side effects.
5. Shared Context
- Store task instructions, agent outputs, code diffs, logs, and conversation history in shared storage.
- Pass each agent only the context it needs.
6. Conflict Prevention
- Give coding agents isolated Git branches or worktrees.
- Validate diffs, run tests, and merge only after conflict checks and required approvals.
7. Observability & Cost Control
- Track token usage, model costs, execution time, retries, and failure rates.
- Set budgets and stop runaway workflows.
8. Pause, Resume & Cancel
Persist workflow state in PostgreSQL.
- Pause: Stop scheduling new tasks and let active workers reach safe checkpoints.
- Resume: Requeue eligible tasks.
- Cancel: Stop pending tasks and signal active workers to terminate safely.
Keep "FAILED" and "CANCELLED" separate: one means execution failed; the other means execution was intentionally stopped.
Architecture at a glance:
Developer → API → Orchestrator → Workflow Engine → Durable Queue → Agent Workers
Supporting services:
- PostgreSQL - workflow state and checkpoints
- Redis - caching and short-lived coordination
- Object storage - logs, artifacts, and code outputs
- Retry/DLQ mechanism - failed task recovery
- Observability - logs, metrics, traces, and cost tracking
The key principle: assume every component can fail, and make recovery part of the architecture.
How would you prevent two agents from modifying the same files and accidentally overwriting each other's work?
System Design Problem: Design an AI Agent Orchestration Platform.
A developer asks multiple AI agents to work on a large task.
One agent writes code, another reviews it, and another runs tests before the changes are merged.
Requirements:
- Coordinate agents working on the same task
- Manage task dependencies and execution order
- Share context and results between agents
- Handle agent failures and retry tasks
- Prevent conflicting code changes
- Track token usage and execution costs
- Pause, resume, or cancel workflows
How would you design the architecture?
System Design: How would you build an AI agent orchestration platform?
Imagine asking 3 AI agents to complete one task:
→ Agent 1 writes the code.
→ Agent 2 reviews it.
→ Agent 3 runs tests before merging.
Sounds simple, right?
The real challenge is coordinating everything reliably.
1. Orchestration Layer
- A central orchestrator assigns tasks, tracks workflow state, and coordinates agents.
2. Workflow Engine
- Represent dependencies as a DAG.
- Code writing must finish before review starts, and tests run after approval.
- Independent tasks can execute in parallel.
Each task follows a state machine:
"PENDING → READY → RUNNING → COMPLETED"
On failure:
"RUNNING → RETRY_WAIT → READY"
If retries are exhausted, the task moves to "FAILED".
Persist state transitions so workflows can recover after crashes.
A task becomes "READY" only when its dependencies are satisfied.
3. Execution Queue
- Use Kafka or a durable job queue to distribute tasks to workers.
- Workers claim jobs, execute them, and report results.
- Use idempotency keys to prevent duplicate processing.
4. Failure Handling
Failures are inevitable. Design for them.
- Transient errors: Retry with exponential backoff and jitter.
- Repeated failures: Move tasks to a dead-letter queue for investigation.
- Worker crashes: Use leases and heartbeats to detect abandoned tasks and safely reassign them.
- Model/API timeouts: Apply timeouts, retry limits, and circuit breakers.
- Partial failures: Persist completed outputs so successful agents don't need to rerun unnecessarily.
- Invalid agent output: Validate results before allowing dependent tasks to proceed.
Never assume a retry means the previous execution didn't happen.
Use idempotency and durable checkpoints to avoid duplicate side effects.
5. Shared Context
- Store task instructions, agent outputs, code diffs, logs, and conversation history in shared storage.
- Pass each agent only the context it needs.
6. Conflict Prevention
- Give coding agents isolated Git branches or worktrees.
- Validate diffs, run tests, and merge only after conflict checks and required approvals.
7. Observability & Cost Control
- Track token usage, model costs, execution time, retries, and failure rates.
- Set budgets and stop runaway workflows.
8. Pause, Resume & Cancel
Persist workflow state in PostgreSQL.
- Pause: Stop scheduling new tasks and let active workers reach safe checkpoints.
- Resume: Requeue eligible tasks.
- Cancel: Stop pending tasks and signal active workers to terminate safely.
Keep "FAILED" and "CANCELLED" separate: one means execution failed; the other means execution was intentionally stopped.
Architecture at a glance:
Developer → API → Orchestrator → Workflow Engine → Durable Queue → Agent Workers
Supporting services:
- PostgreSQL - workflow state and checkpoints
- Redis - caching and short-lived coordination
- Object storage - logs, artifacts, and code outputs
- Retry/DLQ mechanism - failed task recovery
- Observability - logs, metrics, traces, and cost tracking
The key principle: assume every component can fail, and make recovery part of the architecture.
How would you prevent two agents from modifying the same files and accidentally overwriting each other's work?
System Design Problem: Design an AI Agent Orchestration Platform.
A developer asks multiple AI agents to work on a large task.
One agent writes code, another reviews it, and another runs tests before the changes are merged.
Requirements:
- Coordinate agents working on the same task
- Manage task dependencies and execution order
- Share context and results between agents
- Handle agent failures and retry tasks
- Prevent conflicting code changes
- Track token usage and execution costs
- Pause, resume, or cancel workflows
How would you design the architecture?
Ritesh Roushan retweeted
I was asked this in an Uber SDE-1 interview last year.
Problem:
Design the low-level system for matching a rider with the nearest available driver.
My answer:
I’ll store drivers in a hash map keyed by driver ID. When a rider requests a ride, I’ll scan all online drivers, calculate distance for each, and pick the closest one. Then mark that driver as busy.
But then they asked:
How does this work when you have 50k online drivers in a city, and 200 ride requests come in at the same second?
I knew the interview was cooked right there.
What’s your response to get selected?
I was asked this in an Uber SDE-1 interview last year.
Problem:
Design the low-level system for matching a rider with the nearest available driver.
My answer:
I’ll store drivers in a hash map keyed by driver ID. When a rider requests a ride, I’ll scan all online drivers, calculate distance for each, and pick the closest one. Then mark that driver as busy.
But then they asked:
How does this work when you have 50k online drivers in a city, and 200 ride requests come in at the same second?
I knew the interview was cooked right there.
What’s your response to get selected?
Ritesh Roushan retweeted
How Kafka works under the hood?
Kafka is not a simple message queue,
It is a distributed commit log.
Let's see what actually happens when a producer writes and a consumer reads:
1. Messages are written to a partition
• A topic is split into partitions
• Each partition is an ordered, append-only log
• Producers write to one partition (by key or round-robin)
• Order is guaranteed only inside a single partition
2. Data is stored on disk, not just in memory
• Messages are appended to log segments on disk
• Kafka uses sequential writes, which are fast
• Old segments are cleaned based on retention (time or size)
• This is why Kafka can hold data for days or weeks
3. Replication keeps copies
• Each partition has a leader and followers
• Producer writes only to the leader
• Followers pull data from the leader
• A message is considered committed only after enough replicas have it (ISR)
4. Consumers pull, they don’t get pushed
• Consumers track their own offset
• They ask the broker for the next messages from a partition
• This pull model lets slow consumers catch up without overwhelming the broker
• Offsets are stored (usually in an internal topic)
5. Consumer groups share the load
• One consumer group reads a topic together
• Each partition is assigned to only one consumer in the group
• Adding consumers can increase throughput, but only up to the number of partitions
• Rebalancing happens when consumers join or leave
6. Brokers are stateless for consumers
• Brokers don’t track what each consumer has read
• Consumers store their own progress
• This makes scaling consumers easy and keeps brokers simpler
7. High throughput comes from batching and sequential I/O
• Producers batch messages
• Compression reduces network and disk usage
• Sequential disk writes avoid random I/O
• This is why Kafka can handle millions of messages per second
Kafka’s power is not magic.
It is an append-only log, replication, and consumers that pull at their own speed.
Have you used Kafka for something other than simple event streaming?
How Kafka works under the hood?
Kafka is not a simple message queue,
It is a distributed commit log.
Let's see what actually happens when a producer writes and a consumer reads:
1. Messages are written to a partition
• A topic is split into partitions
• Each partition is an ordered, append-only log
• Producers write to one partition (by key or round-robin)
• Order is guaranteed only inside a single partition
2. Data is stored on disk, not just in memory
• Messages are appended to log segments on disk
• Kafka uses sequential writes, which are fast
• Old segments are cleaned based on retention (time or size)
• This is why Kafka can hold data for days or weeks
3. Replication keeps copies
• Each partition has a leader and followers
• Producer writes only to the leader
• Followers pull data from the leader
• A message is considered committed only after enough replicas have it (ISR)
4. Consumers pull, they don’t get pushed
• Consumers track their own offset
• They ask the broker for the next messages from a partition
• This pull model lets slow consumers catch up without overwhelming the broker
• Offsets are stored (usually in an internal topic)
5. Consumer groups share the load
• One consumer group reads a topic together
• Each partition is assigned to only one consumer in the group
• Adding consumers can increase throughput, but only up to the number of partitions
• Rebalancing happens when consumers join or leave
6. Brokers are stateless for consumers
• Brokers don’t track what each consumer has read
• Consumers store their own progress
• This makes scaling consumers easy and keeps brokers simpler
7. High throughput comes from batching and sequential I/O
• Producers batch messages
• Compression reduces network and disk usage
• Sequential disk writes avoid random I/O
• This is why Kafka can handle millions of messages per second
Kafka’s power is not magic.
It is an append-only log, replication, and consumers that pull at their own speed.
Have you used Kafka for something other than simple event streaming?
Ritesh Roushan retweeted
if you think you're good at backend development,
this bug shouldn't survive your code review.
what's the bug, and how would you fix it before it costs real money?
Ritesh Roushan retweeted
if you think you understand rate limiting,
explain why your Redis-based limiter might still fail when traffic hits 1M requests/sec.
which architecture would you trust in production?
if you think you understand rate limiting,
explain why your Redis-based limiter might still fail when traffic hits 1M requests/sec.
which architecture would you trust in production?
if you think you're good at backend development,
this bug shouldn't survive your code review.
what's the bug, and how would you fix it before it costs real money?
Ritesh Roushan retweeted
these days, a lot of candidates are getting rejected in interviews after riding the AI hype.
i've seen people share their interview experiences where they couldn't answer basic questions about the fundamentals behind the projects they built using AI.






