
Attio A/B Tests Hero Demo and Section Order on Its Landing Page
Attio is running a two-variant test on attio.com swapping logo bar/pricing order and hero demo widget — here's why that's actually two experiments wearing one URL.
Sam Lee
Data Analyst · · Updated
TL;DR
- Attio (attio.com) is running a live landing page test detected 8 Feb 2025 at
/p/comp-0208-25. - Detection method: URL cluster variant analysis; analyst confidence: 75%.
- Two levers are bundled in one test: section order (logo bar vs. pricing table swap) and hero demo widget (search-style vs. AI workflow card).
- Variant A leads with pricing, then social proof; Variant B leads with logo bar, then pricing.
- Hero swaps a "Find experienced marketing agencies" search UI for an "Automatically qualify leads" AI workflow card.
- No lift numbers are public yet — this is a design-pattern catch, not a confirmed winner.
What exactly did Attio change on this page?
Attio's variant swaps two independent elements at once: the position of the logo bar relative to the pricing table, and the hero product screenshot. Our vision-AI analyser flagged the page at https://attio.com/p/comp-0208-25 via url_cluster_variant detection, clustering near-duplicate URLs that render structurally different DOMs. One funnel shows the logo bar of customer brands directly under the hero, with pricing pushed further down the page. The other funnel inverts that: pricing table surfaces almost immediately after the hero, with social proof relegated below it. Simultaneously, the hero itself changes — one version shows a search-style interface ("Find experienced marketing agencies"), the other an AI workflow card ("Automatically qualify leads").
That's a compound test, not a clean single-variable A/B split. Worth flagging up front because it directly affects how confidently Attio (or anyone reading this) can attribute a conversion delta to either lever.
Why does testing two elements at once matter here?
Bundling section order and hero imagery into one test means any observed lift can't be cleanly attributed to either change. As A/B Testing Fundamentals Every Marketer Should Know puts it plainly: if conversion goes up, "was it the headline, was it the different call to action layout... we simply don't know." Attio's setup has the same problem times two. If Variant B wins, is it because pricing showed up earlier, because the AI workflow demo resonated more than the search UI, or some interaction between the two? Without a full factorial design (four cells: 2 layouts × 2 heroes), you're stuck reporting a bundled effect and guessing at mechanism.
For a company at Attio's stage, that's a defensible tradeoff if traffic is the binding constraint — running four cells splits your sample four ways and blows up the time-to-significance. But it should be a deliberate tradeoff, not an accident of test tooling.
What's the actual hypothesis, and does the logic hold up?
Attio appears to be testing whether social proof before pricing outperforms pricing-first, and whether an AI-workflow demo communicates value better than a search-demo. That's a reasonable pair of hypotheses on their own — but conflated, they compete for explanatory credit. The pricing-vs-proof question is a classic sequencing test: CrazyEgg's guide to landing page testing notes that you can't test full redesigns with standard A/B tooling, but reordering existing sections is exactly the kind of low-risk, high-signal test that's tractable — "are you presenting information in the best possible order?" is literally the right question to ask before touching hero creative.
The hero-demo question is a different animal: search-UI vs. AI-workflow-card is a positioning test, not a sequencing test. It's testing which product story lands, not which page order lands. Running both simultaneously means Attio is really asking two separate product questions and hoping one URL cluster gives clean answers to both.
How should you size and read a test like this?
Here's where the math matters. Assume Attio's landing page runs at a modest B2B SaaS baseline conversion rate of ~3% to trial signup, and this URL sees, say, 20,000 monthly uniques split evenly across two variants (10,000 each). To detect a relative lift of 15% (3.0% → 3.45%) at 80% power and 95% confidence, you need roughly 9,000–10,500 visitors per arm for a two-proportion z-test — meaning Attio is sitting right at the edge of adequate power for even a single-variable test at that traffic level. Bundle in a second lever and you'd want a true factorial split, which needs roughly 4x the sample per cell to hold power constant. At 20,000 monthly uniques, that's not close to feasible in a single month.
Practical read: unless Attio has considerably more traffic on this specific URL than a typical landing page, this test is more likely running for directional signal over several weeks than for a same-month statistically clean call. That's fine — but any "the AI hero won" claim coming out of this should come with a confidence interval and a note on which lever it's actually isolating, per the SRM and early-engagement checks any mature CRO program runs before trusting a top-line number.
Table: what changed, cell by cell
| Element | Variant A (pricing-first) | Variant B (proof-first) |
|---|---|---|
| Section order | Hero → Pricing → Logo bar | Hero → Logo bar → Pricing |
| Hero demo | Search UI ("Find experienced marketing agencies") | AI workflow card ("Automatically qualify leads") |
| Likely hypothesis | Pricing anchors early, reduces friction to see cost | Social proof builds trust before ask |
| Risk | Novelty of AI card may inflate short-run engagement | Proof-first may lose bottom-funnel visitors who came for price |
Common questions
Is this a confirmed winning variant?
No. This is a live-test detection at 75% analyst confidence based on URL clustering, not a reported result. There's no public conversion data showing either variant outperforming the other, and Attio has not disclosed a winner as of this writing (detected 8 Feb 2025).
Why does testing two changes at once reduce the value of the result?
Because you can't attribute the lift to a single cause. If conversion moves, you won't know if it was the section reorder, the hero demo swap, or an interaction between them — you'd need a 4-cell factorial design to isolate each lever's independent effect.
What sample size would Attio need to trust a result here?
Roughly 9,000–10,500 visitors per arm to detect a 15% relative lift on a ~3% baseline conversion rate at 80% power, 95% confidence — and that's for one variable. Testing two levers simultaneously without a factorial split muddies attribution regardless of sample size.
Could the AI-workflow hero be winning just due to novelty?
Possibly. New UI patterns often show a short-term engagement bump — a "novelty effect" — that fades over 2-4 weeks as users acclimate. Any early lift on the AI card should be re-checked after the novelty window before being logged as a durable win.
What should teams take from this test structure?
Isolate one variable per test where traffic allows, and if you must bundle, use a full factorial design so you can decompose the effect. Section-order tests are cheap and low-risk; hero-narrative tests are higher-stakes and deserve their own dedicated cell.
The takeaway for your roadmap
If you're planning a similar test this sprint, split section-reorder and hero-creative into two sequential single-variable tests rather than one bundled variant — you'll get a clean read on each in roughly the same total traffic, just spread over two phases instead of guessing at a conflated result in one.
Read next
More from the ABWatcher blog
Nov 22, 2026
Snake River Farms Tests a $25-Off Popup on Its Ad Landing Page
We caught Snake River Farms split-testing an email-capture form and $25-off modal on its landing page. Here's the math on what it would take to call a winner.
Nov 21, 2026
Snake River Farms Tests Mid-Page Email Capture on Its Ad Landing Page
SRF is A/B testing a mid-page lead-capture block and a $25-off popup against a simpler hero-only promo banner on its paid landing page, detected 2024 week-over-week.
Nov 20, 2026
Attio Is Testing a New Tier, Higher Prices, and a New Pitch
Attio is running a live pricing page test with a new $1,000 "Ultra" tier, higher Plus/Pro prices, and a new headline — here's what we caught and what it means.
Nov 19, 2026
Found Quietly Tested a 25% Price Cut on Its Pricing Page
We caught Found running a live pricing A/B test with an annual billing toggle that drops Found Plus from $35 to $26.25/mo. Here's what changed and why it matters.
Nov 8, 2026
Astro Proves "Zero Lock-In" With a Live Code Demo, Not a Claim
Astro's homepage replaces a trust badge row with a working, clickable framework demo — here's the hierarchy and interaction design behind it, detected 60% rare on astro.build.
Nov 7, 2026
Basecamp's Big Numbers: Why Hyper-Precise Stats Beat Vague Social Proof
Basecamp's homepage swaps "trusted by thousands" for 1.7 billion to-dos and 5 petabytes of data. We break down why precision beats vagueness as social proof — and how to test it.
From the glossary
Vocabulary the article uses. See all terms →
- Shrinkage Estimator— A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
- Flicker Test— A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
- Pre-Experiment Bucketing Audit— A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
- Above-Fold Exclusion Rule— A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
- Flicker-Free Testing— A technical setup that prevents users from briefly seeing the original page before a test variant loads.
- Orthogonal Test Design— A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See more like this
ABWatcher catches A/B tests like this across 1,000+ brands.
Watch live experiments at 1,000+ high-converting brands, complete with hypothesis and takeaway. 7 days of Pro, free, no card, then Free forever for ten watched companies.