Meta’s Andromeda is breaking your A/B tests
Your traffic isn’t neutral anymore, and your tests feel it.
Hi,
If you’ve been running clean CRO tests on Meta traffic lately and felt a little frustrated by the results, you’re not alone.
Meta’s Andromeda update quietly changed something fundamental: ads now act as a filtering layer before users even reach your landing page.
That means the traffic hitting your tests is no longer neutral or random.
And that has major implications for CRO, it’s why so many tests now feel inconsistent or “off.”
I’ve written a longer article breaking down what Andromeda actually is and how it changes things, but here’s the short version for optimizers.
The key shift
Meta now decides which ads even enter the auction based on predicted value, not just targeting or bids.
Before a user reaches your site, the system has already made assumptions about:
Intent
Likelihood to convert
Economic value
Your landing page is no longer the first qualification step.
Abigail Laurel Morton, a Creative Manager at Aperture, explains it as follows:
“What looks like creative failure is often just Meta selecting the signals it understands best. Creative volume allows the algorithm to identify patterns and predict value.
Out of twelve UGCs, Meta might only scale three, but those three unlock multiple new avenues for iteration across creators, hooks, formats, and messaging.”
The ones that scale feel organic and stand out:
Why this breaks classic CRO assumptions
Under Andromeda:
Different creatives send very different quality users
Traffic composition changes as Meta learns
Variant performance can shift without any page changes
You might see the same number of users per variant, but with wildly different intent levels. The test isn’t necessarily wrong, the input has changed.
What to do differently as a CRO
A few practical changes that have helped in practice:
Validate demand through creatives before over-investing in landing pages
Match creative intent to the destination experience
Segment CRO analysis by creative type, not just by page variant
Treat creative and landing page as one optimization loop, not two separate systems
Instead of asking “Which page converts better?” start asking “Which creative to page combination creates the highest value?”
That mindset shift alone removes a lot of confusion.
For the full breakdown, including how Meta filters ads, what Advantage+ actually does, and how to analyze tests without gaslighting yourself, you’ll need to read the full article:
Recommended resource
Convert, an incredible testing platform for Shopify, sponsors this week’s newsletter. If you’ve got the traffic to run A/B tests, they’re a great platform to use.
If you found this article interesting, be sure to check out my previous piece on how to use ads to run A/B tests as well.
If you’ve seen tests behave strangely on Meta traffic lately, you’re not imagining it.
As always, I’m curious how this shows up for you in practice, so let me know if you’re experiencing the impact of the Andromeda Update.
Till next time,
Daphne
Spot-on breakdown, Meta's Andromeda update creates non-neutral traffic that breaks traditional A/B testing assumptions, with creative pre-filtering introducing intent variance before users even reach landing pages.
Critical compounding issue: Invalid traffic exacerbates this chaos, as bots infiltrate Andromeda-qualified cohorts (30-50% of early impressions) mimicking high-value behaviors, initial conversion spikes collapse when synthetic patterns drop off, destroying statistical significance beyond just creative differences.
Solution: Implement specialized pre-bid behavioral validation platforms before testing; these leverage device fingerprinting and anomaly detection to eliminate fake signals while preserving genuine filtered traffic, delivering clean variant splits for reliable CRO outcomes. Happy to share benchmarks on IVT distortion in post-Andromeda campaigns if helpful!
Thanks Daphne, wouldn't randomization take care of any differences introduced by Andromeda?
Assuming both control and variant are getting traffic from a bunch of differnt creatives, which maybe impacted by Andromeda.