Methodology

FQPPI produces three kinds of output for each Question Period session: scores, enrichment, and flavour. This page documents exactly what each one measures, what the scales mean, and what the data can and cannot support.

Data source

The source of record for every session is the official Hansard published by the House of Commons and indexed by openparliament.ca. FQPPI fetches Hansard HTML, cleans the markup, and parses it into individual exchanges — one question-and-answer unit per exchange. The official Hansard includes Hansard editors' translations of remarks delivered in the other official language; FQPPI preserves both the original and the translation.

Where available, FQPPI also links to CPAC video archives so exchanges can be watched as delivered. The written Hansard and the spoken word sometimes differ; the Hansard is the source used for scoring.

What counts as an exchange

An exchange is one question-and-answer unit, corresponding to a single Hansard topic block. A session of 45 minutes typically contains 25–40 exchanges. Each exchange has one questioner and one or more ministers responding.

Government-backbench exchanges — a member of the governing party asking a minister — are included in the data but scored differently. They are not accountability moments by design: the question score reflects that these are typically softball setups, and the answer score reflects whether the minister used the opportunity to communicate real information.

Procedural exchanges, statements, and ministerial introductions are excluded.

Scores

Every exchange receives two scores, each from 0 to 10, assigned by Claude using the rubric below. Scores use decimal precision (e.g. 5.5, 8.2).

Question score

Measures the specificity, factual grounding, and accountability value of the question.

9–10	Highly specific — cites documents or data, asks a single clearly answerable question, or holds a minister to a prior public commitment
7–8	Reasonably specific — some factual grounding, identifiable ask, but could be sharper
5	Mixed — combines substantive and rhetorical elements, or the ask is implicit
2–3	Mostly rhetorical — partisan framing dominates, ask is vague or compound
0–1	Pure talking points — no real question, unanswerable by design, or a speech in question form

Answer score

Measures whether the minister substantively addressed what was asked.

9–10	Direct — addresses the specific question with facts, numbers, or a clear commitment
7–8	Partial — engages the topic but misses the specific ask, or provides relevant context without fully answering
5	Tangential — some connection to the topic but does not address the question
2–3	Deflection — pivots to the opposition's record, changes subject, or answers a different question
0–1	Non-answer — pure attack, boilerplate, or no connection to the question asked

Session-level scores shown in charts are the arithmetic mean of all exchange scores in that session. No weighting is applied — a terse one-round exchange counts the same as a multi-round policy debate.

The Index

The two scores answer different questions: how good was the question, and how good was the answer. The Federal Question Period Productivity Index combines them into a single number that describes whether the exchange did its constitutional job — opposition extracting accountable answers from the executive.

For a session:

FQPPI = √(avg_Q × avg_A)

For the site headline number, across a sample of sessions:

FQPPI_site = √(mean(avg_Q) × mean(avg_A))

Same 0–10 scale as the inputs.

Why the geometric mean

Question Period is only productive when both halves work. A sharp, specific question stonewalled with talking points produced no accountability. A direct, substantive answer to a vacuous partisan setup produced no accountability either. The arithmetic mean would launder these asymmetries — a 10/0 session and a 5/5 session would both score 5 — while the geometric mean penalizes imbalance: 10/0 collapses to 0, and 5/5 stays at 5. This matches the editorial claim the name makes.

Worked example

A session with avg_Q = 7.0 and avg_A = 4.0:

FQPPI = √(7.0 × 4.0) = √28 ≈ 5.3
Arithmetic mean would have been 5.5

The 0.2-point gap is the cost of asymmetry. As the gap between question and answer quality widens, the FQPPI drops faster than an average would.

Sample

The headline FQPPI on the homepage is computed across the third-Wednesday-of-each-month sample (2000–2025) — the same dataset the quality chart plots. Whether that sample is representative of Question Period as a whole is a separate methodological question we have not yet defended; the index reflects the sample, not the population.

Enrichment

Each exchange also receives a structured enrichment pass, which produces:

Direct answer — a boolean judgment: did the minister substantively address the question, or not? This is a binary version of the answer score, useful for computing the fraction of questions that received a direct response in a session.
Accountability score — an integer from 1 to 5. 1 is complete evasion or redirection with no engagement with the question. 3 is partial engagement. 5 is a direct, specific response with facts or commitments. This scale is finer-grained than the binary direct-answer field but coarser than the 0–10 answer score.
Topic tags — 2–4 short lowercase tags describing the policy area (e.g. housing, cmhc, affordability). These drive the topic stream chart on the homepage.
Key quotes — the sharpest sentence from the question and the most revealing sentence from the minister's response, as identified by Claude. These appear on individual exchange pages.

Flavour

The flavour pass analyses the rhetorical construction of each exchange — not whether the answer was good, but how both sides used language. Each exchange receives scores from 0.0 to 1.0 on 23 tags, applied independently to the question side and the answer side. Tags do not sum to 1.0; they are independent assessments.

Substance

facts_and_figures: Numerical data, statistics, or quantifiable claims
policy_detail: Specific mechanisms, legislation, or program details
specific_commitment: A clear promise or commitment to a future action

Epistemic

evidence_backed_claim: Claim supported by verifiable evidence or data
assertion_without_support: Claim presented without evidence or justification
ambiguous_or_nonfalsifiable: Vague or unverifiable statement that cannot be tested
uncertainty_acknowledgment: Explicit acknowledgment of not knowing or needing to follow up

Rhetorical

emotional_appeal: Language intended to evoke fear, pride, or sympathy
value_signalling: Moral stance expressed without substantive detail
contrast_framing: Framing the issue as we-versus-they
partisan_attack: Direct criticism of an opposing party or member
blame_shifting: Assigning responsibility to others
clap_line: Phrase crafted for rhetorical impact or applause
repetition_talking_point: Reuse of generic or scripted messaging

Anecdotal

illustrative_anecdote: A story that supports evidence or explanation
substitution_anecdote: A story used in place of evidence or a direct answer

Deflection

topic_pivot: Shifting to a different topic than the question
question_reframing: Altering the meaning or scope of the question
opponent_attack_deflection: Responding by attacking the questioner
statistic_substitution: Using statistics in place of addressing the question
talking_point_insertion: Inserting pre-prepared messaging regardless of relevance
explicit_non_answer: A response that does not address the question at all

Sampling and coverage

FQPPI does not score every Question Period session. Coverage is organised into data windows — defined sets of sessions targeted for processing. The current windows are:

Third Wednesday — the third Wednesday of each month from 2000 to present. This produces one session per month as a long-run sample, enabling trend charts going back 25 years. The third Wednesday has no special parliamentary significance; it was chosen to produce consistent spacing and avoid the first and last weeks of a sitting, which can be procedurally atypical.
Full recent weeks — all sessions in recent completed parliamentary weeks, producing complete short-run coverage for current analysis.
Friday sessions — Fridays are shorter and have different participation patterns. These are processed separately and labelled.

Sessions not in a defined window are detected and stored but not scored. Coverage is shown on each session page.

Accuracy and limitations

Scoring consistency. Claude applies the rubrics above consistently across sessions, but AI scoring is not perfectly reproducible — the same exchange scored twice may receive slightly different values. Scores should be interpreted as estimates with an implicit margin of a point or two, not precise measurements.

Cross-time comparisons. Changes to the scoring prompt produce scores that are not directly comparable to those produced by an earlier version. Each score is tagged with a prompt version in the underlying data. When the prompt changes materially, it will be noted here with a date.

Translation. Hansard translations are produced by parliamentary editors, not by FQPPI. Where a member spoke in French and the Hansard translation is in English, the score is applied to the translation. Nuance may be lost.

What the scores don't measure. The question score does not measure whether the question was important or politically significant — only how specific and accountable it was in form. The answer score does not measure whether the minister's answer was true — only whether it addressed what was asked. A minister who gives a direct, specific, factually wrong answer scores well. The Hansard is the source; fact-checking is not part of this pipeline.

Accountability

FQPPI is a project of Peanut Gallery. The scoring rubrics, flavour taxonomy, and pipeline are maintained by the project. Questions, corrections, and methodology feedback can be submitted via the feedback form.

The source data — Hansard and openparliament.ca — is published by the Parliament of Canada and the openparliament.ca project respectively. FQPPI is not affiliated with either.