Mailmodo Is A/B Testing Its Own Homepage Length — On Itself

There's something delicious about catching an A/B testing company mid-A/B-test. Mailmodo's homepage at mailmodo.com/home-d-one/ is currently split into two very different experiences depending on which browser bucket you land in — and the gap isn't a button color or a headline tweak. It's the entire architecture of the page.

What We Actually Saw

Our vision-AI analyser flagged a same-run variance across five visits to the same URL. Visit 1, on Safari, rendered a short page: one hero section, a "Drive 3X engagement with actionable emails" module, and a lightweight footer. Done. Visits 2 through 5 — Chrome, Firefox, Edge — rendered a substantially longer page with the same hero up top, but then kept going: testimonials, a "Every email task, done in seconds with prompts" feature section, "Need a hand with setup?", "Email automations as easy as texting", an FAQ block, and more before the footer.

The cookie layer confirmed what the layout already suggested — Optimizely and EXP experiment keys were present only in the non-Safari visits, which is a clean signal of deliberate bucketing rather than a rendering bug. Safari visitors are seeing what looks like a control. Everyone else is getting the experiment variant. Confidence on this read sits at 92%.

Reading the Control: A Page Built for the First Scroll

The short variant is a bet on restraint. One hero, one supporting proof point, exit. There's no scroll depth to manage because there's barely any scroll. Everything a visitor needs to decide "is this relevant to me" lives above or just below the fold — no F-pattern decay to worry about because the page never asks the eye to travel that far.

This is the classic single-CTA landing page logic: fewer decision points, fewer places for attention to leak. If the hero copy and the "3X engagement" module are doing their job, the visitor either converts on the strength of that one clear promise or bounces. There's no room for a wandering FAQ section to introduce doubt that wasn't already there. It's a page designed to answer one question — what is this, and should I act — and then stop talking.

Reading the Variant: A Page Built for the Skeptic

The long version is a different theory of the visitor entirely. It assumes the person landing on this page isn't ready to act off a single hero claim — they need to be walked, in sequence, through proof, mechanism, and reassurance before the ask feels safe.

The ordering matters here. Testimonials come early, right after the hero, doing the social-proof job before the visitor has had time to generate objections. Then the page moves into mechanism — "Every email task, done in seconds with prompts" — which answers how, not just that. Then it addresses friction directly with "Need a hand with setup?", which is a smart placement: it's the exact moment a feature-curious visitor starts wondering about implementation cost, and the page meets that hesitation head-on rather than letting it fester into a bounce. "Email automations as easy as texting" reads like a closing reassurance beat — plain-language, low-intimidation — right before the FAQ mops up whatever's left.

Structurally, this is a page using progressive disclosure as a trust-building ramp. Each section is a rebuttal to the objection the previous section would have raised in a skeptical mind. That's a legitimate long-form playbook, but it only works if the visual hierarchy through each section is disciplined — if testimonials look like an afterthought, or the FAQ has no real contrast against a wall of body copy, the added length becomes added noise instead of added trust.

The Real Question: Fold Weight vs. Fold Count

What I find more interesting than "long vs short" is what each variant is asking of the fold. The short page puts all its fold weight on the hero — every pixel above that first scroll has to carry the entire value proposition, because there's nothing behind it to catch a visitor who almost clicks but hesitates. That's high pressure on a small amount of real estate, and it demands a hero with strong contrast, a legible hierarchy between headline and subhead, and a CTA that doesn't have to compete with anything else on screen.

The long page distributes that pressure across the scroll. The hero doesn't have to close the deal alone — it just has to earn the next scroll. That's a lower-stakes hero, which usually means the design team can afford to make it feel calmer, more editorial, less shouty. But it shifts the burden downstream: each section transition needs enough visual separation — spacing, background shifts, iconography changes — that the page doesn't blur into homogeneous scroll-fatigue by the time it reaches the FAQ.

Neither approach is inherently correct. It's genuinely a hypothesis worth testing, and it's exactly the kind of test most CRO teams under-run because it touches so many sections at once — it's not a one-variable test, it's a whole-page redesign wearing an A/B test's clothing. That's harder to ship, harder to instrument, and harder to read cleanly when it wins or loses, which is probably why it's rare to catch companies testing at this scope rather than button-color tweaks.

Why This Test Is Structurally Risky to Read

The tradeoff with a page-depth test this broad is attribution. If the long variant wins, is it the testimonials, the setup-friction section, the FAQ, or the cumulative effect of all of them together? Mailmodo's own A/B testing guide flags this exact trap — bundling too many changes into one variant makes the "why" unreadable even when the "what" (a lift or a drop) is statistically solid. They'll know which page performed better. They may not know which section did the work, unless there's a second layer of testing planned to pull the winning variant apart, section by section.

Takeaway for Your Sprint

Before you run a full page-depth test like this one, decide up front whether you're testing a hypothesis or testing a redesign. If it's a redesign, budget for a follow-up round of section-level tests on the winner — don't let "long page won" be the final answer when the real answer is buried three sections down. Test the whole page if you must, but plan to disassemble the winner immediately after.