Preliminary · October 2026
Research
How we test the engine, in the open.
VibeScore reads eight emotions in a piece of text (joy, trust, fear, surprise, sadness, disgust, anger and anticipation) and returns a profile and a 0 to 1000 score. In October 2026 we tested that engine against labels written by people, and against itself. Every figure below names the section of the October 15, 2026 validation report it comes from, and links to where its method is described on this page.
What we found.
0.90
The models agree with each other
Average correlation between four AI models from four different companies on the 0 to 1000 score, across the 500 comments. On our own 500 phrases it is 0.96.
October report, section 7Method: The AI readings3.5×
Chance, on people’s own labels
The reading's strongest emotion is the person's label on 42% of comments, against 12% by chance. Its top two include the label 65% of the time.
October report, sections 1, 6 and 9Method: The comparison12–26
Points apart, out of 1000, on a second read
Read the same text twice and the score moves by about one to three percent. Measured on our own phrases.
October report, section 7Method: Our own phrases
Why we test.
Ad+Verb is additive advertising: advertising that adds something to a person's life. To know whether a message adds something, we need a reading of what it carries, and we need to know how far that reading can be trusted. So we test the instrument in the open and publish what we find.
Check it against people.
Compare the readings with emotions people labeled themselves, so we know where the engine agrees with people and where it does not.
Agreement with peopleKnow when a change is real.
Measure how far a model moves when it reads the same text twice, so a difference smaller than that reads as normal variation, not as a change.
Consistent with itselfPublish the method.
Show how every figure was made, beside the figure, with the full report one click away, so you can judge the instrument yourself.
The full report
Next
What we test next.
- Run the numbers again on the locked emotion-combination table, once it has been independently reviewed.
- Measure how well people agree with each other on the same comments, the ceiling to judge 0.46 against.
- Test eight anchored questions head to head against simply asking an AI for the emotional tone.
How we tested.
- 01
People label 500 comments.
- 500 Reddit comments, a balanced sample of about 58,000
- 102 carry two or more labels
- The comment below
I'm so uncomfortable.
People labeled it embarrassment, one of the 27 emotions they could choose.
- 02
Eight models read each one.
4,000 readings, 17 failed calls. The 4 production models (Claude Haiku 5.5, GPT-6 Luna, Grok 4.7, Gemini 3.5 Flash-Lite) are averaged into one consensus reading.
The consensus reading, shares of 100FEA 48DIS 23SAD 17 - 03
The reading meets the label.
The readingFEA 48DIS 23SAD 17Old table: one emotion per labelSAD 1000.30 matchBlend table: half fear, half disgustFEA 50DIS 500.89 matchHow you turn a person's label into emotions changes the number.
- 04
Then across all 500.
42%of comments: the reading's strongest emotion is the person's label.Overall agreement across all 500 comments and eight emotions: 0.462, where chance is 0.001. The engine gives a comparable reading of a text's emotional tone. It does not yet show that it reliably names the exact emotion a person felt.
500 Reddit comments from Google's GoEmotions dataset (Demszky et al., 2020, CC BY 4.0). Raters tagged each comment with one or more of 27 emotions, or neutral. The 500 are a balanced sample of about 58,000, and 102 of them carry two or more labels.
Eight models read each comment once and returned eight emotion strengths and a 0 to 1000 score: 4,000 readings, 17 failed calls. The four production models are averaged into one consensus reading.
500 phrases written by Claude Opus and 200 written independently by GPT-6, each read twice by the four production models. These are used for one thing: checking whether a model repeats itself.
The comparison.
The human side. Each comment's labels are turned into a blend of the eight emotions and averaged. A comment labeled love and gratitude becomes half joy, half trust.
The AI side. The eight emotion strengths are scaled so they add up to 100%.
Three comparisons. One overall agreement number across all 500 comments and eight emotions. How closely each comment's AI blend matches the human blend in shape, from 0 to 1. And whether the AI's top emotion is the person's.
Guards against fooling ourselves.
A chance level. The human labels are shuffled across comments 200 times. Real signal has to beat that.
An uncertainty range. The comments are re-sampled 1,000 times, so each change comes with a 95% range.
A reverse check. The AI readings are translated into the 27 GoEmotions labels and compared from that side.
Consistent with itself.
0.90
Agreement between models
Average correlation between models on the 0 to 1000 score, on the 500 comments. On our own 500 phrases it is 0.96.
October report, section 7Method: The AI readings0.76
The floor of agreement between models
Emotion by emotion, the models agree from 0.76 on trust to 0.90 on anger: every emotion is above 0.75.
October report, section 7Method: The AI readings12–26
Points apart on a second run
Out of 1000, when the same model reads the same text twice. Measured on our own phrases, not yet on the comments labeled by people.
October report, section 7Method: Our own phrases
- Claude Haiku 5.512.0
- GPT-6 Luna16.6
- Grok 4.718.3
- Gemini 3.5 Flash-Lite26.5
| Model, 500 of our own phrases, two runs | Run 1 against run 2 | What it means |
|---|---|---|
| Claude Haiku 5.5 | 12.0 | Points apart out of 1000, on average. |
| GPT-6 Luna | 16.6 | |
| Grok 4.7 | 18.3 | |
| Gemini 3.5 Flash-Lite | 26.5 | Repeats itself least, and sits furthest from the other three. |
October report, section 7. A difference between two readings that is smaller than these numbers is within normal variation. Method: Our own phrases
Agreement with people.
Well above chance. The reading's strongest emotion is the person's own label 3.5 times as often as chance would give, and the label is among its top two 65% of the time. Naming the exact emotion a person felt, every time, is what we are working on next.
- Consensus readingBy model, 38% to 45%42%
- Top two match65%
- Chance12%
- Chance level0.001
- One label per comment, as the benchmark originally scored it0.445
- Before the blend table0.461
- After the blend table0.462
Every agreement measure
- 129 Matched better
- 80 Matched worse
- 291 Unchanged
| Measure | Result | What it means | Method |
|---|---|---|---|
| Top emotion match | 42% | The consensus reading's strongest emotion is the person's label. Chance is 12%. By model it runs from 38% to 45%. A March run on the same comments gave 43%. | Method: The comparison |
| Top two match | 65% | The person's label is one of the reading's two strongest emotions. | Method: The comparison |
| Overall agreement | 0.461 → 0.462 | Correlation across all 500 comments and eight emotions, before and after the blend table. The change is +0.001, with a 95% range of -0.023 to +0.024. As the benchmark originally scored it, one label per comment, it was 0.445. Chance level is 0.001. | Method: Guards against fooling ourselves |
| Per-comment match | 0.560 → 0.590 | How closely each comment's AI blend matches the human blend. Up 0.030 with the blend table (95% range +0.012 to +0.047): 129 comments matched better, 80 worse, 291 unchanged. | Method: The comparison |
| Reverse check | 0.239 → 0.222 | AI readings translated into the 27 GoEmotions labels and compared from that side. Slightly worse with the blend table. | Method: Guards against fooling ourselves |
October report, sections 1, 6 and 9
By emotion: agreement with people's labels
How closely the consensus reading tracks people's labels for each emotion, where 1 is a perfect match. All eight emotions are shown.
- Fear0.651
- Joy0.604
- Sadness0.548
- Anger0.535
- Surprise0.391
- Trust0.379
- Disgust0.339
- Anticipation0.324
October report, section 6, consensus of the four production models with the blend table. Method: The comparison
One comment, scored two ways.
How you turn a person's label into emotions changes the number. This is a real comment from the test.
From the dataset
I'm so uncomfortable.
People labeled it embarrassment. The consensus reading was fear 48, disgust 23, sadness 17, as shares of 100.
| How the label is read | Match | Why |
|---|---|---|
| Old table: one emotion per label | 0.30 | Embarrassment was mapped to sadness alone, so most of the reading counted as a miss. |
| Blend table: half fear, half disgust | 0.89 | The same reading now lines up with the label. |
October report, worked example in section 5. The blend helped this comment. Across all 500 it did not move the overall figure. Method: The comparison
The full report.
Everything else from the October report: what didn't work, the limits, the engines still in research, and what we don't claim.
What failed.
An old headline.
An earlier figure on our old proof page covered four of the eight emotions, and a later audit found it was credited to the wrong model run. It was also set beside a result from a different, harder task. Both the figure and the comparison are withdrawn.
The blend theory.
We thought the low agreement was a mapping problem. Reading labels as blends left overall agreement unchanged. It helped anger, surprise and disgust, and hurt anticipation, sadness and trust.
Reply Fit.
A second number comparing a reply with the message it answers. It had to track what the customer wrote next better than the reply's own score. It did not: +0.053 against +0.086.
Limits.
The emotion-combination table is a draft. It has not yet been independently reviewed and locked, and the numbers will be run again on the locked version.
The 500 comments were picked to be balanced under the old one-emotion mapping, which may favor it.
How well people agree with each other on the same comments. Until that is known there is no ceiling to judge 0.46 against.
Whether eight anchored questions are steadier than simply asking an AI for the emotional tone. A head-to-head test is planned.
A positive against negative check exists, but its label grouping has not been checked against the official file, so we do not quote it.
Text alone. Images, web pages and video have no validation yet.
The figures describe the engine as tested in October with Claude Haiku 5.5, GPT-6 Luna, Gemini 3.5 Flash-Lite and Grok 4.7. A reading made with a different set of models is not covered by them.
The software passes 156 of 156 automated tests. Those check that it does what it was built to do, not that its readings agree with people.
Product, and research.
Ad+Vantage is experimental. It has not been validated, and we do not claim that its guided flow improves decisions.
| Engine | Status | The question it asks |
|---|---|---|
| Emotion (VibeScore) | In use | Which emotional associations does the message carry? Tested as described on this page. Ad+Vantage uses this engine alone today. |
| Perception and value | In testing | Which kinds of value does the message ask people to prioritize? Being built and tested. Not part of any product. |
| Context | Research | Under what circumstances does that value matter? |
| Interests and agency | Research | What does each party want, and what can they do? |
| Precedent | Research | How did comparable situations unfold? Precedent, never a forecast. |
What we don't claim.
No accuracy headline.
We don't lead with one flattering number. The measured figures are above, low ones included.
Words, not people.
A reading describes the content. It doesn't tell you how a specific person feels, or whether they'll act.
Reading, not forecast.
No forecast of audience reaction or sales. Agreement between models shows consistency, not truth.
Can I check these numbers?
The benchmark sample, the scoring scripts and the October report are kept with the VibeScore source. They are not public yet. Every figure on this page names the report section it comes from, so it can be checked the day they are.
Where to go next.
See what the engine reads, or the product built on it.
Request a free introduction
Twenty minutes to talk through what you're working on and whether Ad+Verb can help. If there's a fit, we'll agree on a written scope before work begins.