
How to A/B Test Email With a Small List: 7 Simple Steps
To A/B test email with a small list, change one thing, as Vero advises, such as the subject line. Send both versions to random halves of your list at the same time and wait two days. Treat the result as a lean, not proof, because small lists can't reach statistical certainty.
If you own a local service business and email your customers a few times a month, a list of a couple hundred to a couple thousand names is enough to learn something. You're busy doing the work, so the method below fits in a lunch break.
Table of contents
- What is an A/B test on an email?
- Why does a small list change the rules?
- How do you run the test, step by step?
- What should you test first?
- Can you test the email that follows up on a quote request?
- How do you decide a winner when the list is small?
- What do small lists simply not prove?
- What are the common mistakes?
- Where do you run this without more tools?
- FAQ
- Your next step today
What is an A/B test on an email?
It's a fair fight between two versions of the same email. Klaviyo describes it this way: "Email A/B testing, also known as split testing, involves creating two or more variations of an email and sending each version to a different subset of your subscribers to observe which one wins based on a chosen metric."
So you write two subject lines, send one to half your list and the other to the other half, and see which one more people open. That's the whole idea. Where people go wrong is trusting a result that the list is too small to back up.
Why does a small list change the rules?
Because the big guides were written for big lists, and the numbers they ask for are out of reach for most local shops. HubSpot recommends sending an A/B tested email to at least 1000 contacts. Mailchimp says "To run an effective A/B test, we recommend having at least 5,000 subscribed contacts." Litmus asks for "a randomized sample size of at least 10,000 people."
If your list is 300 names, you're below every one of those lines. That doesn't mean you should skip testing. It means you should be honest about what the result tells you.
TechTarget's example shows the problem plainly: "if marketers with a contact list of 200 test a subject line on 20% of their audience, only 20 people would receive each version." Twenty people per version is a coin flip with a story attached. If 9 open one version and 7 open the other, you haven't learned much.
Think of it like taste testing two batches of chili on four neighbors. You'll get an opinion, and it's worth hearing, but you wouldn't print the recipe on the strength of it.
How do you run the test, step by step?
Follow these in order. Each takes a few minutes.
- Pick one email you were going to send anyway. A reminder, a seasonal offer, a maintenance note. Don't create extra work.
- Pick one thing to change. Vero puts it like this: "By running small, controlled experiments on a single variable at a time, you can compare and contrast results from different copy, timing, layouts, and more." For a small list, the subject line is the best place to start, because everyone sees it.
- Write your guess before you send. One sentence: "I think the subject line with the customer's first name gets more opens." This keeps you from explaining away the result later.
- Split the list at random. Half and half. Don't put your best customers in one group and new leads in the other, because then you're testing the people, not the email.
- Send both at the same time. Same day, same hour. A Tuesday morning send against a Friday night send tells you about Tuesday and Friday.
- Wait. Litmus says "an A/B test takes at least 48 hours to run its course, and even then, you may want to wait longer to achieve statistical significance." Two days is the minimum.
- Write down the result and what you'll do next. Keep a simple note: the date, both versions, the counts, the winner.
What should you test first?
Start with the thing the most people see and the least effort to change. This table gives an order that works for a small list.
| Test | What you change | What you count | Good for a small list? |
|---|---|---|---|
| Subject line | Wording, length, or a first name | Opens | Yes, start here |
| Follow up after a quote request | The first line or the subject | Replies and booked calls | Yes, and it's where leads are won or lost |
| Send day or time | Tuesday morning vs. Thursday evening | Opens and replies | Yes, once you have a few sends |
| Call to action | "Call us" vs. "Book online" | Calls and bookings | Maybe, results come in slowly |
| Email length | Short vs. long | Clicks and replies | Later, after the basics |
| Design and images | Layout, photo, buttons | Clicks | Later, the differences are small |
The closer a test is to a real booking, the fewer events you'll see on a small list. A subject line gives you opens within hours. A call to action might give you two phone calls in a week.
Heads up: opens are a rough signal. Some email apps load messages automatically, which can count an open nobody made. So when you can, count replies and calls next to opens, and trust those more.
Can you test the email that follows up on a quote request?
Yes, and for a local service owner it's the test that matters most. When someone asks for a quote and nobody answers, that lead goes to whoever replies first. That's the whole idea behind Speed to Lead: answer every inquiry fast, then make the answer worth reading.
So take the automatic reply that goes out after a quote request and test two openings. One can ask a plain question, like "What day works for a visit?" The other can state your next available slot. Count replies and booked visits, not opens. You'll get fewer data points than a newsletter, so run it for a month before you call it.
How do you decide a winner when the list is small?
Honestly, you decide a lean. The big guides ask for certainty. Litmus says, "Aim for 95% confidence (p < 0.05) before declaring a winner, and if you’re not able to do that, run the test again." Smaily sets the bar a little lower: "To determine whether your test is a winner or a loser, the test results must be at least 90% statistically significant."
On a list of a few hundred, you will rarely get there from one send. So use this rule instead. If one version is clearly ahead, say by a wide margin on both opens and replies, use it next time and test the idea again. If the two are close, call it a tie and move on. A tie is a real answer. It tells you that thing doesn't matter much to your customers, and you can stop worrying about it.
The cheap fix for a small list is repetition. Run the same kind of test across three or four sends. If the first name subject line wins three times out of four, you've learned something a single send could never show.
What do small lists simply not prove?
Three things, and you should say so out loud to yourself before you act on a result.
First, a small gap between versions. If 41 people opened one and 38 opened the other, that's noise. Second, anything about revenue. You won't send enough emails to link one subject line to dollars. Third, anything about a small slice of your list, like "customers over 60 prefer longer emails." You don't have enough people in the slice.
What you can learn is rougher and still useful. Which of two subject lines sounds more like you? Which day gets more replies? Do your customers answer a plain question better than a sales line? Those are small, practical leans, and they're worth having.
One more honest limit. If you see a big percentage in a blog post, check whether it came from a list of millions.
What are the common mistakes?
The most common one is changing two things at once. If you rewrite the subject line and swap the photo, and one version wins, you won't know why. Change one thing and keep everything else identical.
The second is calling the winner too early. You send at 9 a.m., check at 9:30, see a lead, and pick it. Opens keep coming in for a day or two, and the early leader often isn't the final one. Wait the 48 hours.
The third is testing something nobody cares about. A comma in the subject line won't move anyone. Test a real difference, like a plain question against a short offer.
The fourth is never writing anything down. Three months later you won't remember what won. A note with the date, both versions and the counts takes two minutes and makes the next test smarter.
The fifth is splitting the list by hand and favoring one side. Let your email tool split it at random.
Where do you run this without more tools?
If your email and your customer list already live in one place, the split and the counting are a few clicks. We built Launch CRM so you aren't paying for one tool to send email, another to text, and a third to follow up. Email marketing and workflows sit together, so your list, your sends and your follow up run on one login and one bill. Rocket, the built in AI, answers new leads in under 60 seconds while you're on the job, so the quote request you're testing a reply for doesn't sit waiting.
If you want to set up campaigns, our email marketing articles walk through it. The marketing automation and workflow articles cover sending the right message after someone books or asks for a quote. And if you're wondering what it looks like when one platform replaces your separate tools, read our overview of how a cloud based platform works. For the reply side of the story, see our lead response and Speed to Lead articles.
FAQ
What is the 60/40 rule in email?
We don't know of a standard 60/40 rule, and we won't make one up. The nearest thing is a 20/20/60 split: Litmus describes sending version A to 20%, version B to another 20%, then the winner to the remaining 60%.
What is A/B testing in digital marketing?
It means showing two versions of one thing, such as an email, ad or web page, to different groups of people. You change one element, count how each group responds, and keep the version that did better.
Should I trust open rates on a small list?
Only a little. Opens can be counted when nobody actually read the message, and a small list makes any gap noisy. Count replies, calls and bookings beside them, and repeat the test before you change how you write.
What should I test first in a follow up email?
The first line, or the subject. Try a plain question against a stated time slot, send both at once to random halves, and count replies and booked visits. Give it a month, because follow ups are sent only when someone asks.
Your next step today
Open the next email you plan to send and write two subject lines for it. Name your guess in one sentence, split your list in half at random, send both at the same time, and put a reminder on your calendar for two days from now to count the opens.
