Tabellab

Message Testing Scorecard: Clarity, Relevance and Demo Intent Rates, a Weighted Message Score and a Shortlist

Lesson 1 of 20Lab

All lessons
  1. Message Testing Scorecard: Clarity, Relevance and Demo Intent Rates, a Weighted Message Score and a ShortlistLabFree
  2. Message Fit by Segment: Appeal Rates with Mixed References, Market-Weighted Appeal and the Winning Message per SegmentLab Locked
  3. Messaging Tests on a Live Page: Conversion Rate per Headline, Lift against the Control, z-Score and a VerdictLab Locked
  4. Competitive Comparison Matrix: Importance-Weighted Vendor Scores with SUMPRODUCT, Lead or Lag and a Ranked ChartLab Locked
  5. Feature Parity and Gaps: Best Competitor per Capability, Lead/Parity/Gap Labels with IFS and the Importance in GapsLab Locked
  6. Win/Loss Analysis Basics: Closed Deals, Wins and Win Rates per Competitor with COUNTIFS, Won Revenue and Win Rate by ValueLab Locked
  7. Loss Reasons as a Pareto Table: Counts, Shares, Cumulative Share with an Expanding Range and Lost Revenue per ReasonLab Locked
  8. Message Recall Surveys: Unaided and Aided Recall, Attribution to the Brand and a Recall Index per AudienceLab Locked
  9. Recall Tracking over Time: Exposed vs Not Exposed, Lift in Points, Margin of Error, SLOPE and a Linear ForecastLab Locked
  10. Feature Adoption after an Announcement: Cumulative Adopters, Adoption Rate, Active Share and Weeks to TargetLab Locked
  11. Announcement Channels: Adoption Rate, Cost per Adopter, Share of Adopters, RANK.EQ and the Blended CostLab Locked
  12. Adopters vs Non-Adopters: COUNTIF and AVERAGEIFS by Group, Growth before and after, and the Growth GapLab Locked
  13. Announcement Effect with a Holdout Group: Weekly Usage Rates, the Gap in Points, Phase Averages, Naive Growth and a Line ChartLab Locked
  14. Sales Enablement Usage: Rep Adoption, Win Rate When Used, Lift in Points, an Action Label and Shares by Asset TypeLab Locked
  15. Sales Onboarding and Certification: Certified Flag, Days to First Deal with DAYS, Median Ramp, AVERAGEIFS and the Top PerformerLab Locked
  16. Sizing the Next Survey Wave: z Values with NORM.S.INV, a Sample-Size Grid with Mixed References, Budget Check and a Heat MapLab Locked
  17. Survey Data Quality: Speeders, Straight-Liners, Duplicate Panel IDs, a Keep/Remove Rule and the Duplicate HighlightLab Locked
  18. Perceptual Positioning: Distance to the Buyers' Ideal Point with SUMXMY2, Weighted Ratings, Shortfalls and the Largest GapLab Locked
  19. Battlecard Rollout: Quarterly Win Rate, a Pooled Moving Rate, Before and After, a 95 % Interval for the Change and Weak-Quarter HighlightLab Locked
  20. Final Project: Dispatch Launch Scorecard with Direction-Aware Indices, Improvement, Status, a Weighted Score and a ChartLab Locked

You are not signed in — the lab works, but your progress is not saved. Sign up

Lesson 1 of 20 is free. Unlock all 20 hands-on labs, the final exam and your certificate.

Buy course — $9.90
Loading the spreadsheet…

Why test messages before a launch?

A product launch fails more often because buyers do not get the message than because the product is poor. The sentence on the homepage, in the sales deck and in the announcement email decides whether a field service manager understands, cares and asks for a demo. Many teams pick that sentence in a meeting where the loudest opinion wins. A message test replaces the opinion with evidence: show each candidate to a panel of target buyers, count what they say, and compare the candidates on the same scale.

This article walks through the simplest useful version of such a test. You will see how to turn raw survey counts into rates, how to combine three rates into one message score, how a threshold turns the score into a shortlist, and how far you can trust the result. It is the first lab of the course Product Marketing: Positioning & Messaging Tests, and every step can be rebuilt in any spreadsheet.

The example company is Corvellis, a fictional software maker. Its product Dispatch plans the day of technicians who visit customers: heating engineers, electricians, pest controllers. Corvellis, the survey panel and every number in this article are invented for teaching, and the benchmarks mentioned are illustrative, not industry data.

Positioning and messaging are not the same thing

Positioning is a decision about the market: which buyers the product is for, which alternative it replaces and why it is the better choice for them. It is usually written as an internal statement and is rarely shown to customers word for word. Messaging is the translation of that decision into sentences, claims and proof points that a buyer can understand and repeat to a colleague.

The distinction matters for testing. You cannot survey a positioning statement; you can only survey the messages that express it. If every candidate message fails, the problem may sit one level up, in the positioning itself. If the messages differ widely, the test tells you which expression of the same position works best.

  • Positioning answers: for whom, instead of what, and why us. It changes rarely.
  • Messages are short claims such as “Plan every technician’s day in minutes”. A product has several, and they change with the audience.
  • Proof points are the facts behind a claim: a customer story, a number from a trial, a screenshot. A claim without proof is the first thing a sceptical buyer discards.
All eight messages in this article describe the product honestly. A message test chooses between truthful expressions; it must never be used to find the most persuasive exaggeration.

How a monadic message test works

In a monadic test every respondent sees exactly one message. That keeps the groups independent: nobody compares two sentences side by side, which is closer to how a buyer meets a message in real life. The price is that you need a separate panel for each candidate. Corvellis recruited about 200 target buyers per message, and every respondent answered the same three questions.

  1. Clarity: “Does this tell you what the product does?” The respondent either understood the message or did not.
  2. Relevance: “Is this relevant for how your company works?” A message can be perfectly clear and still beside the point.
  3. Intent: “Would you book a demo after reading this?” This is the closest answer to action that a survey can give.

From positioning to a shortlist: the message test1 Positioningwho, instead of what, why2 Candidate messages8 claims in plain words3 Survey panelone message per respondent4 Three ratesclarity, relevance, intent5 Weighted scoreweights add up to 16 Decisionshortlist or dropFictional workflow of Corvellis, maker of the Dispatch field service software
Figure 1. The message test in six steps. Everything before step 3 is a decision, everything after it is arithmetic you will build in the lab.

The respondents are selected by behaviour and business facts: they run or work in a field service company of the right size and trade. The panel is never selected by personal characteristics such as age or gender, and it takes part with consent and a clear explanation of how the answers are used.

From counts to rates: always divide by everyone who saw the message

The survey tool delivers counts: how many respondents saw a message and how many answered yes to each question. Counts cannot be compared directly, because panels differ slightly in size. The step is to convert them to rates: rate = respondents who said yes ÷ all respondents who saw the message.

The denominator is the point where beginners go wrong. It is tempting to calculate relevance among those who understood the message, and intent among those who found it relevant. That produces a chain of conditional rates that looks encouraging, because every step only keeps the people who already agreed. A buyer who did not understand the message is a real loss for the message, so the base for all three rates is the same: every respondent who saw it.

  • Clarity rate = understood ÷ respondents. For M1 that is 176 ÷ 210 = 83.8 %.
  • Relevance rate = found it relevant ÷ respondents. For M1: 139 ÷ 210 = 66.2 %.
  • Demo intent rate = would book a demo ÷ respondents. For M1: 48 ÷ 210 = 22.9 %.
Write the three rates in three separate columns. It costs nothing, makes the sheet readable for colleagues and lets you see at a glance which message is clear but irrelevant, or relevant but confusing.

One number per message: the weighted score

Three rates per message and eight messages make 24 numbers. To shortlist, the team needs a single figure per message, and that requires a decision about importance. Is a clear message worth as much as one that triggers demo requests? Most teams say no: the intent is the closest to a result, clarity is the precondition. Corvellis agreed on these fictional weights before looking at the data: clarity 0.25, relevance 0.35 and demo intent 0.40.

The weights add up to 1, so the message score stays on the same 0 to 1 scale as the rates: score = clarity × 0.25 + relevance × 0.35 + intent × 0.40. A simple average of the three rates would treat clarity like intent; the weights state the team’s view in the open, where it can be challenged and changed.

Agree the weights and the threshold before the results arrive. If you pick them afterwards, it is too easy to choose the numbers that crown the message you liked from the start.

Worked example: eight messages for Dispatch

Corvellis tested eight candidate messages for the launch of Dispatch. The table shows the survey counts, the three rates, the weighted score and the decision at a threshold of 0.53. This is exactly the data you will find in the lab below.

MsgCore claimResp.UnderstoodRelevantDemoClarityRelevanceIntentScoreDecision
M1Fewer missed appointments2101761394883.8 %66.2 %22.9 %0.5326Shortlist
M2Plan every technician's day in minutes2051811505788.3 %73.2 %27.8 %0.5880Shortlist
M3One screen for jobs, routes and parts1981411093171.2 %55.1 %15.7 %0.4333Drop
M4Cut drive time by a fifth2121871315388.2 %61.8 %25.0 %0.5368Shortlist
M5Built for field teams, not for IT2011491263874.1 %62.7 %18.9 %0.4803Drop
M6Invoices out the same day2071701122982.1 %54.1 %14.0 %0.4507Drop
M7AI-assisted scheduling you can overrule1961221043562.2 %53.1 %17.9 %0.4128Drop
M8Customers always know when you arrive2041841554990.2 %76.0 %24.0 %0.5875Shortlist

Message M2, “Plan every technician’s day in minutes”, has the highest score at 0.5880. Message M7, “AI-assisted scheduling you can overrule”, has the lowest at 0.4128: only 62.2 % of its panel understood it, which is the weakest clarity of all eight messages. Four messages reach the threshold and are shortlisted: M1, M2, M4 and M8.

Weighted message score per message (threshold 0.53)ShortlistDrop0.00.20.40.60.533M10.588M20.433M30.537M40.480M50.451M60.413M70.587M8threshold 0.53Message (fictional survey panel, about 200 respondents per message)
Figure 2. Weighted message score of the eight candidate messages. Four bars reach the dashed threshold of 0.53 and are shortlisted: M1, M2, M4 and M8. M1 clears the line by only 0.0026.

The formulas, step by step

The sheet keeps the weights and the threshold in their own cells (C14 to C17 in the lab). Formulas then refer to those cells with absolute references, so the reference does not move when the formula is copied down. For the first message, in row 4, the formulas are:

  • Clarity rate (G4): =D4/C4, copied down to row 11.
  • Relevance rate (H4): =E4/C4.
  • Demo intent rate (I4): =F4/C4.
  • Message score (J4): =G4*$C$14+H4*$C$15+I4*$C$16. The dollar signs lock the three weights.
  • Decision (K4): =IF(J4>=$C$17,"Shortlist","Drop").

Walk through message M3, “One screen for jobs, routes and parts”. Of 198 respondents, 141 understood it (71.2 %), 109 found it relevant (55.1 %) and 31 would book a demo (15.7 %). The score is 0.7121 × 0.25 + 0.5505 × 0.35 + 0.1566 × 0.40 = 0.4333. That is below 0.53, so the decision is Drop. The message is clear enough, but few respondents saw a reason to act.

Message M2: from respondents to demo intentSaw message M2205 (100.0 %)Understood it181 (88.3 %)Found it relevant150 (73.2 %)Would book a demo57 (27.8 %)Each rate divides by all 205 respondents who saw M2, never by the step before.
Figure 3. Respondents, clarity, relevance and demo intent for message M2. The three rates are 88.3 %, 73.2 % and 27.8 %.

In the funnel for M2 the largest drop comes between relevance and intent: 73.2 % found the message relevant, but only 27.8 % would book a demo. That gap is normal, because stating an intention is a bigger step than agreeing with a sentence. It is also the reason why intent carries the highest weight.

It is worth testing how much the ranking depends on the weights. With equal weights of one third each, M8 would score 0.6340 and M2 0.6309, so the top two would swap places. When a ranking flips after a modest change of the weights, the messages are close and the team should say so, instead of announcing a winner.

The threshold has a similar status. A value of 0.53 is a fictional team decision: it lets four of the eight candidates through to the next round. A stricter threshold of 0.55 would keep only M2 and M8, and a looser one of 0.45 would add M5 and M6 and keep six. Choose it so that the next round, usually a live test, stays affordable.

How far can you trust the numbers?

A survey panel is a sample, and samples are noisy. For a rate p measured on n respondents, the margin of error at 95 % confidence is roughly 1.96 × √(p × (1 − p) ÷ n). For the demo intent of M2 (p = 27.8 %, n = 205) that is about ±6.1 percentage points; for M4 (n = 212, p = 25.0 %) it is about ±5.8 points.

Both margins are about six points, which is larger than the difference between M2 (27.8 %) and M4 (25.0 %) on intent. The test cannot separate these two messages, and it should not try to. What it can do reliably is to separate the strong group from the weak group: M7 with 17.9 % intent and M6 with 14.0 % are far away from the top.

  • Use the survey to shortlist, not to crown a winner.
  • Confirm the shortlist with a live test on a page or in an email, where people act instead of answering.
  • Report the panel size next to every rate, so that readers can judge the precision themselves.

Common mistakes and how to spot them

  • Dividing by the previous step. Relevance among those who understood the message looks better than relevance among everybody. If a rate is suspiciously high for a confusing message, check the denominator.
  • Forgetting the dollar signs. Without them the weights move down with the formula and row 5 multiplies by an empty cell. The whole score column turns into small, wrong numbers.
  • Weighting counts instead of rates. Multiplying the head counts by the weights makes messages with larger panels look better. Weight the rates.
  • Averaging the three rates. A plain average hides the team’s priorities and treats clarity like intent.
  • Swapping the labels of the IF. A score at or above the threshold is the shortlist. Read the formula aloud once before copying it down.
  • Treating a close call as a result. Two scores a few hundredths apart on 200 respondents are a tie.

How to read the result and what to do next

The scorecard produces a decision the whole team can follow: four messages go to a live test, four are retired. The result also teaches something about the market. All four shortlisted messages name an outcome the buyer already measures: missed appointments, the time to plan a day, drive time or customers who know when the technician arrives. The two messages that describe how the product works, a single screen (M3) and AI-assisted scheduling (M7), were both dropped.

  • Take the shortlisted messages into the live test of the next lessons.
  • Read the drops for reasons: M7 failed on clarity, M3 and M6 on intent. Each pattern suggests a different fix.
  • Check the winners by segment. A message that works for plumbers may not work for pest controllers, which is the topic of lab 2.
  • Keep the sheet. When the next message round arrives, the same formulas and weights make the results comparable.

Real-world uses. The same structure works far beyond product launches. Teams use it to compare subject lines before a campaign, value propositions for a pricing page, taglines for a rebrand, or the wording of a sales pitch. Whenever several candidate wordings compete for the same audience and a small panel can rate them, counts become rates, rates become a score and a threshold becomes a decision.

Product marketers also use it as a conversation tool. Showing the sales team a scorecard where their favourite message scored below the threshold is easier than arguing about taste, as long as the weights were agreed beforehand and the data are shown openly.

Quick reference: functions and shortcuts in this lesson

  • =D4/C4: a rate, count ÷ base.
  • $C$14: an absolute reference; press F4 in most spreadsheet programs to add the dollar signs.
  • =G4*$C$14+H4*$C$15+I4*$C$16: a weighted sum with locked weights.
  • =IF(J4>=$C$17,"Shortlist","Drop"): a decision from a threshold.
  • Copy a formula down a column by dragging the fill handle or by pressing Ctrl+D (Cmd+D on a Mac).

What you will build in the rest of the course

This article covered the first of 20 labs. The remaining 19 follow the product marketer’s workflow from the first message test to the launch review. Each lab uses a new data set of the same fictional company and ends with checked formulas.

  1. Message Fit by Segment: appeal rates per buyer segment with mixed references, market-weighted appeal and the winning message per segment.
  2. Messaging Tests on a Live Page: conversion rate, lift against a control, z-scores and verdicts for eight headlines.
  3. Competitive Comparison Matrix: importance-weighted vendor scores with SUMPRODUCT, lead or lag and a ranked chart.
  4. Feature Parity and Gaps: best competitor per capability, Lead, Parity or Gap labels with IFS and the importance that sits in gaps.
  5. Win/Loss Analysis Basics: closed deals, win rates per competitor and won revenue with COUNTIFS and SUMIFS.
  6. Loss Reasons as a Pareto Table: counts, shares, cumulative share and lost revenue per reason, with a chart.
  7. Message Recall Surveys: unaided and aided recall, attribution to the brand and a recall index per audience.
  8. Recall Tracking over Time: exposed versus not exposed, margins of error, SLOPE and a linear forecast.
  9. Feature Adoption after an Announcement: cumulative adopters, adoption rate, active share and the week the target is reached.
  10. Announcement Channels: adoption rate, cost per adopter and rank for eight channels.
  11. Adopters vs Non-Adopters: COUNTIF and AVERAGEIFS by group and the growth gap.
  12. Announcement Effect with a Holdout Group: a before-and-after comparison that removes the background trend.
  13. Sales Enablement Usage: rep adoption, win rate when an asset is used and an action label.
  14. Sales Onboarding and Certification: days to first deal, median ramp and quota attainment by certification.
  15. Sizing the Next Survey Wave: a sample-size grid, a budget check and a heat map.
  16. Survey Data Quality: speeders, straight-liners and duplicate panel IDs.
  17. Perceptual Positioning: the distance of each option to the buyers’ ideal point.
  18. Battlecard Rollout: a win rate before and after, with a 95 % interval for the change.
  19. Final Project: a launch scorecard that combines message, recall, win rate, adoption, enablement and cost.

Summary

A message test turns survey counts into three rates with the same denominator, combines them with weights agreed in advance into one score, and uses a threshold to decide which messages go on. The arithmetic is simple; the care goes into the denominators, the absolute references and an honest reading of the sampling noise. Treat the shortlist as the start of the evidence, not the end.

Now scroll up to the lab and build the sheet yourself. It takes about fifteen minutes, and the first lesson is free.

Frequently asked questions

What is a monadic message test?

A monadic test shows each respondent exactly one message and asks the same questions about it. Because nobody compares messages side by side, the groups stay independent and the result is close to how a buyer meets a message in real life. The drawback is that every candidate needs its own panel.

Why divide every rate by all respondents who saw the message?

Using the same base for clarity, relevance and demo intent keeps the rates comparable and honest. If relevance were calculated only among people who understood the message, a confusing message would look better than it is, because the people it lost have already been removed from the base.

How do I choose the weights of the message score?

Agree them with the team before the results arrive, and write them in cells. A common choice is to give the answer closest to action, such as demo intent, the highest weight and clarity the lowest. Then test how much the shortlist changes if the weights move a little; if it flips, the messages are close.

How many respondents do I need per message?

About 200 per message gives a margin of error of roughly six points on a rate near 25 percent, which is enough to separate strong messages from weak ones but not to rank two similar ones. Lab 16 shows how to calculate the sample size for a margin of error you choose.

Can a survey tell me the winning message?

It can shortlist. Survey answers describe what people say, not what they do. Take the shortlisted messages into a live test, such as a headline test on a page, and judge the final choice by conversions.

Which spreadsheet skills does this lesson use?

Division for rates, absolute references such as $C$14 for the weights, a weighted sum, and an IF formula for the shortlist decision. The later labs add functions such as SUMPRODUCT, COUNTIFS, XLOOKUP, SLOPE and FORECAST.LINEAR, always with new data.

Keep going: Product Marketing: Positioning & Messaging Tests, Competitive Matrices, Win/Loss Analysis, Recall Surveys, Feature Adoption & Sales Enablement

This free lab is lesson 1 of 20. Unlock the other 19 hands-on labs, the final exam and your certificate for a one-time $9.90.

  1. Message Fit by Segment: Appeal Rates with Mixed References, Market-Weighted Appeal and the Winning Message per Segment
  2. Messaging Tests on a Live Page: Conversion Rate per Headline, Lift against the Control, z-Score and a Verdict
  3. Competitive Comparison Matrix: Importance-Weighted Vendor Scores with SUMPRODUCT, Lead or Lag and a Ranked Chart
  4. Feature Parity and Gaps: Best Competitor per Capability, Lead/Parity/Gap Labels with IFS and the Importance in Gaps
  5. Win/Loss Analysis Basics: Closed Deals, Wins and Win Rates per Competitor with COUNTIFS, Won Revenue and Win Rate by Value
  6. Loss Reasons as a Pareto Table: Counts, Shares, Cumulative Share with an Expanding Range and Lost Revenue per Reason
  7. Message Recall Surveys: Unaided and Aided Recall, Attribution to the Brand and a Recall Index per Audience
  8. Recall Tracking over Time: Exposed vs Not Exposed, Lift in Points, Margin of Error, SLOPE and a Linear Forecast
  9. Feature Adoption after an Announcement: Cumulative Adopters, Adoption Rate, Active Share and Weeks to Target
  10. Announcement Channels: Adoption Rate, Cost per Adopter, Share of Adopters, RANK.EQ and the Blended Cost
  11. Adopters vs Non-Adopters: COUNTIF and AVERAGEIFS by Group, Growth before and after, and the Growth Gap
  12. Announcement Effect with a Holdout Group: Weekly Usage Rates, the Gap in Points, Phase Averages, Naive Growth and a Line Chart
  13. Sales Enablement Usage: Rep Adoption, Win Rate When Used, Lift in Points, an Action Label and Shares by Asset Type
  14. Sales Onboarding and Certification: Certified Flag, Days to First Deal with DAYS, Median Ramp, AVERAGEIFS and the Top Performer
  15. Sizing the Next Survey Wave: z Values with NORM.S.INV, a Sample-Size Grid with Mixed References, Budget Check and a Heat Map
  16. Survey Data Quality: Speeders, Straight-Liners, Duplicate Panel IDs, a Keep/Remove Rule and the Duplicate Highlight
  17. Perceptual Positioning: Distance to the Buyers' Ideal Point with SUMXMY2, Weighted Ratings, Shortfalls and the Largest Gap
  18. Battlecard Rollout: Quarterly Win Rate, a Pooled Moving Rate, Before and After, a 95 % Interval for the Change and Weak-Quarter Highlight
  19. Final Project: Dispatch Launch Scorecard with Direction-Aware Indices, Improvement, Status, a Weighted Score and a Chart

Buy course — $9.90 See the full course