System Design Interview Rubric: Score the Reasoning

“Strong technical depth, good communicator” sounds like feedback until someone asks what the candidate actually did. I want a scorecard that can answer that question a week later, without depending on the interviewer's memory or confidence.

The framework below is how I would structure that evidence for a 45-minute design round. It is a rubric to calibrate for a role, not a universal hiring bar.

The five dimensions should separate observable engineering decisions that otherwise collapse into a general impression. Framing, trade-offs, depth, failure handling and adaptation each reveal something different. Record evidence for each, and mark a dimension unobserved when the interview did not give it a fair test.

I would use four anchors: absent, partial, solid and strong. “Strong” should mean more complete reasoning against a written expectation, not “surprised the interviewer.” Novelty is not a fair requirement.

Keep “not observed” separate from “absent.” If I use the whole round on storage, I have not earned the right to score the candidate's incident response as weak.

Good problem framing turns an open-ended prompt into a small set of requirements that determine the design. The evidence is not the number of questions or minutes spent. It is whether the answers change what the candidate proposes and what they leave out.

Use one example throughout the interview: a system that sends appointment reminders.

A useful opening establishes channels, volume, timing, consent, cancellation and the consequences of duplicate or late messages. The candidate does not have to cover every topic immediately. They should identify which unknowns matter most.

“Duplicate SMS messages are worse than a short delay” leads to a different retry discussion from “late reminders are useless.” “We send push only” removes an entire carrier boundary.

The system design books I recommend help with a repeatable opening sequence. The interview should then test whether the candidate can adapt that sequence rather than recite it.

Trade-off reasoning connects a decision to the current workload and explains when another choice would be better. A memorised comparison can be accurate without answering the question. Ask what would have to change for the candidate to reverse the decision.

For the reminder system, compare these constructed examples:

None of these sentences alone decides a hire. The follow-up matters: what retention, ordering and recovery behavior does the design actually need?

The same caution applies to reference architectures. Studying 55 architecture examples is useful for seeing options. It is not evidence that the same option belongs in this system.

Depth reveals whether the candidate can turn a box into an implementation with state, concurrency and failure behavior. Let them identify a consequential component, then follow its reasoning down. Reward a coherent design, not a preferred vocabulary or one canonical solution.

For the reminder scheduler, I would ask:

1. How are due reminders found?
2. What happens when many become due at once?
3. How do workers avoid claiming the same work incorrectly?
4. What happens if a worker sends a request and crashes before recording the response?

The last question matters because writing “sent” before sending is not a solution. That can lose work. Writing it after sending leaves an ambiguous retry window.

A stronger answer distinguishes intent, attempt and confirmed outcome. It uses a provider's idempotency contract where available, and explains the remaining uncertainty where it is not. Agent workflow orchestration has the same problem when an external tool succeeds before the run records success.

Failure handling should connect the event, durable state, user impact and operator response. Retrying is only one part. A useful answer explains when retries stop, what evidence remains and how someone distinguishes a delayed result from a permanently lost one.

For example: the SMS provider times out after accepting a request.

The candidate should ask whether the provider supports idempotency or a status lookup. Without either, a local ledger cannot magically make the external effect exactly-once. The design needs an explicit policy for the ambiguity.

A monitoring answer should also name a signal. Oldest pending reminder age is more useful than “queue metrics.” Delivery receipts can help where the provider supports them, but an acceptance response is not proof that the person read the message.

Diagram chaos exercises can help rehearse these questions. They model the dependencies drawn; production tests establish what the deployed system actually does.

Test adaptation by changing one relevant constraint after the original design is understood, then give the candidate time to reason. Explain that the change is deliberate. The goal is to observe which assumptions they revisit, not how quickly they recover from being surprised.

A candidate may reasonably defend the original design if it still meets the new constraint. That is not automatically rigidity. Ask for the evidence.

Likewise, hesitation is not proof of weak reasoning. Time pressure, communication style and accessibility needs affect interview performance. Give consistent prompts and allow appropriate accommodations rather than rewarding only the fastest speaker.

The evidence should be compared with expectations written for the actual role, then considered alongside the rest of the hiring process. A scorecard supports a decision; it should not hide uncertainty behind a total. Record what was observed, what was prompted and what remains unknown.

A compact debrief entry might read:

Trade-off reasoning: solid. Chose a queue for one-consumer dispatch, explained the replay limitation, and said a second independent replaying consumer would reopen the decision.

That is more useful than “good systems thinker.” Another interviewer can challenge or corroborate the evidence.

For a staff role, I would also look for how the candidate handles ambiguity, ownership and consequences beyond one component. For a hands-on delivery role, I would ensure the loop gives enough room to demonstrate implementation. Company size alone does not settle those weights.

Before using the rubric, run it against a few mock answers with other interviewers. If you cannot agree on the evidence, fix the anchors before evaluating a real candidate.

The point is not to make hiring look mathematical. It is to make the reasoning visible, including its limits.