Panel interviews: how to run one that adds signal instead of noise
A panel interview is supposed to average out individual bias. Run badly it does the opposite — it launders one dominant person’s opinion through three signatures and calls the result a consensus.
- Three panellists is usually the ceiling for signal; beyond that you add scheduling cost, not accuracy.
- Divide the competencies between panellists so each has a defined remit and probes deeply rather than everyone skimming everything.
- Every panellist scores independently and submits before any discussion. This is the single rule that makes panels work.
- The most senior person speaks last in discussion, or their view becomes everyone’s prior.
- Panels are hard on candidates. Brief them on who is in the room and why, and keep it to a defined length.
What a panel is actually for
Three reasons are usually given for putting several interviewers in a room, and only two of them are good.
- 01Aggregation. Several independent assessments of the same evidence average out individual idiosyncrasy. This is the real benefit, and it depends entirely on the assessments being independent.
- 02Coverage. Different panellists bring domain knowledge that lets each probe deeply in their own area. Also real, and it requires an explicit division of remit.
- 03Buy-in. Everyone who will work with the hire gets a say, so nobody objects later. Understandable politically, and it degrades the assessment — a panel assembled for consent rather than judgement will produce consent rather than judgement.
Composition and remit
| Panellist | Remit | Should not |
|---|---|---|
| Hiring manager | Role-specific capability, delivery track record | Chair the discussion and score first |
| Peer or senior practitioner | Depth in the technical or craft competencies | Test for identical background to their own |
| Cross-functional stakeholder | Collaboration, communication with adjacent teams | Assess technical depth outside their field |
| Recruiter or TA partner | Process integrity, consistency across candidates, motivation and logistics | Be silent — the consistency check is the panel’s safeguard |
Assign each panellist two or three competencies from the rubric, and tell them they are responsible for producing ratable evidence in those areas. A panellist with no defined remit asks whatever occurs to them, which is how the same three questions get asked three times and half the rubric goes unassessed.
On diversity of panel composition: a panel drawn entirely from one background will systematically find candidates from that background more familiar and more impressive, and no amount of good intent corrects it. Compose deliberately, and if you cannot, lean harder on anchored scoring and adverse-impact monitoring to catch what composition did not.
Independent scoring, then discussion
This is the rule that separates panels that work from panels that perform work.
- 01Every panellist completes their scorecard alone, with evidence, before leaving the room or opening a shared document.
- 02Scores are submitted and locked. Nobody sees another panellist’s score until all are in.
- 03Discussion begins from the divergences, not from an overall verdict — "we scored judgement three points apart, what did each of us hear".
- 04The most senior person in the room speaks last, every time.
- 05The final recommendation is recorded with the original independent scores intact, not overwritten by the consensus.
Preserving the original scores matters beyond the individual decision. Systematic divergence between two panellists across many candidates is the clearest available signal that one of them is applying the rubric differently, and it disappears entirely if you only keep the consensus.
The candidate side of a panel
Panels are stressful in a way that is largely unrelated to the job. A candidate facing four people, unsure who matters, being assessed on composure they would never need to display in the role, is being tested on the interview rather than the work.
- Send names, roles and a one-line reason for each panellist before the interview. Withholding it does not test anything useful.
- Say who will chair and how long each section runs.
- Have one person, not four, ask each question — cross-talk from a panel reads as interrogation.
- Leave real time for the candidate’s questions, with someone who can answer them honestly.
- Offer an adjustment path. For some candidates a four-person panel is a genuine accessibility barrier, and refusing to vary it is a Disability Discrimination Act problem, not a standards problem.
The AI panel: aggregation without the scheduling
The mechanism that makes a human panel valuable — several independent assessments of the same evidence — is also the hardest part to schedule. Four senior people in one room for an hour, per candidate, does not survive a pipeline of four hundred.
MERIT-8 applies the same principle differently. Eight agents each own one competency and assess the same interview transcript independently, each required to cite verbatim evidence or abstain. There is no dominance effect because the agents cannot see one another’s ratings, and no scheduling cost because the assessment happens after the interview rather than during it. The human panel then does what it is genuinely best at: meeting the small number of candidates who already have evidence behind them, and making the decision.