Skip to content

Prompt-engineering experiment using verbalized sampling to improve tone alignment, click-through performance, and audience targeting for creative UX content.

Notifications You must be signed in to change notification settings

ashleysally00/Verbalized-Sampling-Experiment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 

Repository files navigation

Verbalized Sampling for UX

A project that shows how self-evaluating LLM prompts can improve tone alignment, SEO, and audience engagement for creative content.

What is Verbalized Sampling?

Verbalized sampling means asking a language model not just to give one answer, but to generate several possible answers and explain how likely each one is to be the best.

It's a way of making the model think out loud about its own uncertainty.

Instead of giving a single, fixed response, the model produces a small set of alternatives (for example, five captions or ideas) and assigns each one a probability or confidence score.

This helps you see how it weighs different directions, exposes its internal "preferences," and often results in more creative or balanced output.

When to Use It

You'd use it when you want to explore diversity and reasoning, rather than get just one polished answer — for example, testing which phrasing might get more clicks or connect best with new readers through creative diversity.


Initial Prompt Using Verbalized Sampling

You are helping write Pinterest descriptions for a website called Penelope's Secret Pages, which features illustrated poems, doodles, and quotes written in a witty, confident, honest, and slightly rebellious voice.

About the Voice

The narrator's tone is candid, clever, and emotionally real — never sentimental or overly sweet. Avoid words such as gentle, heartfelt, or uplifting. The writing feels sincere but bold, introspective but self-aware.

About the Site

Penelope's Secret Pages is an online collection of illustrated poems and reflections about friendship, independence, humor, and self-expression. The site also includes links to books and cards that share this same voice and sensibility.

Audience

Women ages 18–35 who enjoy creative, relatable writing about life, emotions, and personal growth. Most are encountering this content for the first time and are unfamiliar with its origins.

Behavioral Challenge

On Pinterest, many users save or like the poem pins but rarely click through. They often think the poem image itself is the complete experience, not realizing that there's a full website with more writing and related books.

Goal

Write short Pinterest descriptions that make new readers curious enough to click the link to explore the site. The copy should hint that there's more to discover beyond the image while staying true to the same authentic, clever, emotionally aware tone.

Do not mention the narrator, the site name, or the character "Penelope" in the descriptions.

Task

Generate five short Pinterest descriptions (each under 20 words) that:

  • Capture the tone and personality of the writing.
  • Encourage users to click the link to explore more writing and books.
  • Avoid mentioning the narrator or site name.
  • Avoid clichés like uplifting, gentle, or heartfelt.

For each description, include:

  • The description text.
  • A short reasoning line explaining why it might drive clicks.
  • An estimated probability (0–100%) that it will have the highest click-through rate.

Model Output

Description Text (Under 20 Words) Reasoning for Clicks Estimated Highest CTR Probability
Real talk, illustrated. Hit the link for the next chapter of this chaos. Uses bold, conversational language ("Real talk," "chaos") and implies the user is missing a piece of the story ("next chapter"). 25%
This is only the appetizer. Find the full, honest poem collection here. Uses the metaphor "appetizer" to show that the pin is incomplete, creating curiosity and a clear call to action. 35%
If this got you, wait until you see the rest of the confessions. Click it. Appeals to shared emotion ("got you") and uses the personal word "confessions" to promise deeper content. 15%
More unfiltered writing & doodles for your beautifully messy life. Explore. Aligns with appreciation for honesty ("unfiltered," "messy life") and uses "Explore" as a clear action word. 10%
Your favorite honest quote? There are books full of them. Click the link. Frames content as something users already love ("favorite honest quote") and expands the offer ("books full of them"). 15%

Revised Prompt Using Verbalized Sampling (Tween-Appropriate Tone)

After reviewing the first results, it became clear that the responses sounded too mature for the intended audience. The model had defaulted to older phrasing because the prompt did not specify the age range. The revised version adds age and tone guidance to better match the target readers.

You are helping write Pinterest descriptions for a website called Penelope's Secret Pages, which features illustrated poems and doodles written in a friendly, funny, and relatable voice for girls and teens.

About the Voice

The writing sounds smart and real but age-appropriate — curious, playful, and honest. It can be thoughtful or a little sarcastic but never heavy, dark, or adult. Avoid slang that feels too old ("chaos," "confessions," "real talk," etc.) and avoid words such as gentle, heartfelt, or uplifting.

About the Site

Penelope's Secret Pages shares short poems and drawings about friendship, growing up, and finding confidence. The site also links to books and cards written in this same spirit.

Audience

Tween and teen girls (roughly 10–17) and their parents or teachers who enjoy encouraging, funny, or creative quotes. Many visitors will be discovering this content for the first time.

Behavioral Challenge

On Pinterest, viewers often like or save the poem pins but don't click through. They think the image is the full poem, not realizing there's a whole website with more poems and books in the same style.

Goal

Write short Pinterest descriptions that make new readers curious enough to click the link to explore the site. The copy should feel natural to the poems' tone and age group but must not mention Penelope or the site name.

Task

Generate five short Pinterest descriptions (each under 20 words) that:

  • Match a tween-friendly, creative, and sincere tone.
  • Encourage readers to click to read more poems and find the related books.
  • Avoid adult slang, brand names, or phrases like "real talk," "confession," or "next chapter."

For each description, include:

  • The description text.
  • A short reasoning line explaining why it might drive clicks.
  • An estimated probability (0–100%) that it will have the highest click-through rate.

Model Output

Description Text (Under 20 Words) Reasoning for Clicks Estimated Highest CTR Probability
Did you finish the poem? Click here for the rest of the honest thoughts! Asks a question that suggests the poem is incomplete, creating a clear reason to click. 35%
More poems about friends, feelings, and being yourself are right here. Uses age-appropriate keywords ("friends," "feelings," "being yourself") and promises "More" content. 25%
Love this doodle? There's a whole book full of them. Tap to explore! Elevates a single doodle to a full collection, motivating a click to see more. 15%
Cute poem! Click the link to see all the drawings and read more. Uses simple, positive language ("Cute poem!") and "all" to imply a big collection. 10%
If this made you giggle, wait for the others. Click for the secret collection! Focuses on a fun emotional response ("giggle") and uses the playful phrase "secret collection." 15%

Observation (So Far)

The initial version of the prompt produced language that sounded more mature than intended. This showed that when the age of the target audience is not defined, the model defaults to a generalized adult tone. Updating the prompt to specify a tween and teen audience immediately changed the phrasing and emotional range of the output. The revised descriptions used simpler vocabulary, positive emotions, and relatable topics such as friendship and self-expression. This suggests that adding age and tone context to a verbalized sampling prompt can meaningfully shape how a model interprets "voice" and "fit" for an audience.

Next Steps

These prompts have not yet been tested on live Pinterest pins. The next phase will involve posting a small set of caption variants and comparing real engagement metrics such as impressions, clicks, and saves. The goal is to see whether the model's predicted click-through probabilities align with actual audience behavior.

This experiment will later be included in the UX portfolio to show how prompt iteration and verbalized sampling can improve tone alignment and audience targeting in creative content.

About

Prompt-engineering experiment using verbalized sampling to improve tone alignment, click-through performance, and audience targeting for creative UX content.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published