
Snake River Farms Swaps Testimonials for an Email Gate — Is It Working?
Snake River Farms is running a same-page A/B test that replaces testimonials with an email-gated $25-off form and exit-intent popup. Here's the math on whether that trade is likely a net win.
Sam Lee
Data Analyst · · Updated
TL;DR
- Brand: Snake River Farms (snakeriverfarms.com), premium meat DTC ecommerce.
- Page: landing page at
/pages/ad-top-equal, detected via same-run variance across 7 visits. - Variant A (Safari, visit 1): full page with a testimonials section, no popup.
- Variant B (Firefox, Edge, Chrome x2, Firefox — visits 3-7): testimonials replaced with an email-gated "Sign up for access / $25 off sitewide" form, plus an exit-intent discount popup.
- Detection method: same_run_variance — the split held across three distinct browser engines (WebKit, Gecko, Chromium), ruling out a rendering bug.
- Analyst confidence: 80%, logged as a genuine live experiment, not a caching artifact.
What did the variance actually show?
Across 7 recorded visits, one browser engine saw testimonials and six saw an email gate plus exit popup instead — a pattern too consistent to be noise. The tell here is dispersion across engines, not just across visits: Safari rendered the "control" (testimonials), while Firefox, Edge, and Chrome — two separate visits each on Chrome and Firefox — all rendered the "treatment" (email gate + exit-intent popup). If this were a caching or rendering glitch, you'd expect it to cluster by engine or by session recency, not to split cleanly into two consistent buckets across six of seven visits. That's the signature our same_run_variance method is built to catch — genuine bucket assignment, not artifact.
The magnitude of the change also argues for "real test": this isn't a button color swap, it's a full section replacement (social proof → gated capture form) plus the addition of an entirely new interaction (exit-intent popup). Conversion Sciences calls this a "Big Swing" — the kind of test teams run when they suspect the page's core value prop or funnel logic is underperforming, not when they're doing incremental polish.
Why would Snake River Farms trade testimonials for a gate?
The likely hypothesis is that email capture plus a $25 discount lever generates more downstream value than on-page social proof alone. Testimonials do work — they're a trust signal for a $150+ box of wagyu you can't touch before buying — but they only convert visitors in that session. An email-gated $25-off offer converts visitors into a list asset that can be remarketed for months. On a landing page explicitly named ad-top-equal (almost certainly a paid-traffic landing page, likely for an "equal" or top-of-funnel ad set), the marginal value of a captured email address for retargeting could easily exceed the marginal trust lift from testimonials, especially if paid CAC is already doing the trust-building work upstream via ad creative.
That said, this is a layout+element test bundled together (gate section and exit popup), which technically muddies attribution. As TruConversion notes, testing more than one variable at once turns a clean A/B into something closer to a multivariate test — if the variant wins, Snake River Farms won't cleanly know whether the lift came from the gate, the popup, or the combination.
What would a real read on this test require?
You'd need enough sessions per arm to detect a plausible ecommerce lift with reasonable power, and nothing in the detection tells us they have it. For a landing page conversion rate baseline of, say, 3%, detecting a 15% relative lift (0.45pp absolute) at 80% power and 95% confidence needs roughly 14,000–16,000 sessions per arm — not unusual for a paid landing page, but worth stating plainly rather than assuming. Contentful's guidance on layout testing is right that layout changes "can greatly affect how visitors interact," but effect size claims without a sample size and interval are just anecdotes with better formatting.
There's also a real risk of a novelty effect running the wrong direction: exit-intent popups reliably lift email capture in the short run (that's well documented), but they can suppress the browse-to-purchase path for return visitors who get annoyed on repeat sessions. If Snake River Farms is only measuring immediate opt-in rate, they may be logging a "win" that's actually a shift from purchase-intent traffic to lower-intent list traffic. A rough sanity check: if the gate variant lifts email signups by 25% but purchase conversion on that same page drops even 2 points, on a landing page doing $2–5M in attributed annual revenue, that 2-point drop is a $40K–$100K annualized hit that a signups-only dashboard would never surface.
How should you read this if you run similar pages?
This table shows how the two variants differ and where the ambiguity sits:
| Element | Variant A (Safari) | Variant B (Firefox/Edge/Chrome) | Confound risk |
|---|---|---|---|
| Testimonials section | Present | Removed | High — removes trust signal entirely |
| Email capture form | Absent | "$25 off sitewide," gated | Primary test variable |
| Exit-intent popup | Absent | Present, discount-based | Bundled second variable |
| Browser engines seen | WebKit (1 visit) | Gecko + Chromium (6 visits) | Rules out rendering bug |
The practical fix for anyone replicating this test on their own roadmap: split the gate and the popup into separate cells (A: testimonials, B: gate only, C: testimonials + popup, D: gate + popup) so you can isolate which lever is doing the work, and pre-register your primary metric as revenue-per-visitor, not signup rate, before you launch. Per WOLF Financial's benchmark table, social proof positioning changes typically move conversion 8–20%, while form-field changes move it 20–45% — meaning the email-gate swap likely has more raw conversion leverage than the testimonial removal costs, but that's a population-level benchmark, not a guarantee for a niche premium-meat audience with a materially different trust threshold.
Common questions
Is this confirmed to be a real A/B test and not a bug?
Yes, with 80% analyst confidence. The same split (testimonials vs. gate+popup) appeared consistently across three different browser engines — WebKit, Gecko, and Chromium — across 6 of 7 visits, which is the pattern same_run_variance detection is designed to flag as genuine bucketing rather than a caching or rendering artifact.
Why would removing testimonials be worth the risk?
If the page's traffic source (paid ads, given the ad-top-equal URL slug) already carries brand trust, on-page testimonials may add less marginal conversion than a $25-off email capture that extends the relationship past one session. It's a bet that list-building beats immediate on-page social proof for this specific funnel stage.
What sample size would Snake River Farms need to trust a result here?
To detect a 15% relative lift on a 3% baseline conversion rate at 95% confidence and 80% power, you need roughly 14,000–16,000 sessions per variant. Without knowing this page's traffic volume, we can't confirm they've reached that threshold, so any early "winner" should be treated as directional only.
Could the exit-intent popup be hurting more than it helps?
Possibly. Exit popups reliably boost short-term email opt-ins but can suppress purchase conversion for repeat visitors who find them intrusive. If Snake River Farms is optimizing for signup rate alone rather than revenue-per-visitor, they risk shipping a variant that grows the list while quietly shrinking sales on the same page.
What should teams running a similar test do differently?
Split the gate and the popup into separate test cells instead of bundling them, so you can attribute lift to a single variable. Also pre-register revenue-per-visitor as the primary metric, not signup count, since email capture and purchase conversion can move in opposite directions on the same variant.
Takeaway for your next sprint
If you're planning a similar swap — trust content out, email gate in — don't ship it as one bundled variant. Run testimonials-vs-gate and popup-on-vs-off as separate cells, hold revenue-per-visitor as the primary metric rather than opt-in rate, and don't call a winner until you've cleared the sample size math for your actual baseline conversion rate. A 25% lift in signups on a few hundred sessions per arm isn't a result — it's a hypothesis waiting for power.
Read next
More from the ABWatcher blog
Nov 22, 2026
Snake River Farms Tests a $25-Off Popup on Its Ad Landing Page
We caught Snake River Farms split-testing an email-capture form and $25-off modal on its landing page. Here's the math on what it would take to call a winner.
Nov 21, 2026
Snake River Farms Tests Mid-Page Email Capture on Its Ad Landing Page
SRF is A/B testing a mid-page lead-capture block and a $25-off popup against a simpler hero-only promo banner on its paid landing page, detected 2024 week-over-week.
Nov 20, 2026
Attio Is Testing a New Tier, Higher Prices, and a New Pitch
Attio is running a live pricing page test with a new $1,000 "Ultra" tier, higher Plus/Pro prices, and a new headline — here's what we caught and what it means.
Nov 19, 2026
Found Quietly Tested a 25% Price Cut on Its Pricing Page
We caught Found running a live pricing A/B test with an annual billing toggle that drops Found Plus from $35 to $26.25/mo. Here's what changed and why it matters.
Nov 8, 2026
Astro Proves "Zero Lock-In" With a Live Code Demo, Not a Claim
Astro's homepage replaces a trust badge row with a working, clickable framework demo — here's the hierarchy and interaction design behind it, detected 60% rare on astro.build.
Nov 7, 2026
Basecamp's Big Numbers: Why Hyper-Precise Stats Beat Vague Social Proof
Basecamp's homepage swaps "trusted by thousands" for 1.7 billion to-dos and 5 petabytes of data. We break down why precision beats vagueness as social proof — and how to test it.
From the glossary
Vocabulary the article uses. See all terms →
- Shrinkage Estimator— A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
- Flicker Test— A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
- Pre-Experiment Bucketing Audit— A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
- Above-Fold Exclusion Rule— A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
- Flicker-Free Testing— A technical setup that prevents users from briefly seeing the original page before a test variant loads.
- Orthogonal Test Design— A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See more like this
ABWatcher catches A/B tests like this across 1,000+ brands.
Watch live experiments at 1,000+ high-converting brands, complete with hypothesis and takeaway. 7 days of Pro, free, no card, then Free forever for ten watched companies.