A page needs 10 or more clicks in the 28 days before a change for Penlid to put a size on that change. Whether a test finds the change depends on the lift's size, the number of untouched comparison pages and how far those pages drift apart, so where pages drift, more traffic helps only up to a point.
- Penlid puts a size on a change only when the page had 10 or more clicks in the 28 days before it, and gives a direction and no size for 1 to 9 clicks (Penlid, SEO test calculator, read 30 October 2026).
- With six comparison pages and no drift, Penlid's simulations found a real +80% lift on a 4-clicks-a-day page 92.01% of the time and a real +50% lift 61.23% of the time (Penlid, methodology, read 30 October 2026).
- With 15% drift between pages, a real +80% lift on six comparison pages was found 79.55% of the time at 40 clicks a day and 82.20% at 400 clicks a day (Penlid, methodology, read 30 October 2026).
- Google says the time a reliable user-facing A/B test needs varies with factors such as how much traffic a site gets, but its page gives no instructions for designing tests (Google, Search Central documentation, read 30 October 2026).
Why does the number of clicks matter at all?
Because a count of n clicks wobbles by about the square root of n even when nothing changes, a small page's before-and-after percentage can be mostly noise. In a Poisson count, as Wikipedia puts it, the parameter λ is equal to the expected value of X and also to its variance
, and our calculator draws the clicks in each 28-day window from that model.
The relative noise of one count is 1 ÷ √n (our arithmetic): 32% at 10 clicks, 9.4% at 112 and 0.94% at 11,200, so cutting it tenfold takes 100 times the clicks. With 10 clicks in each window and no real change, a normal approximation puts the 95% range of the after-to-before ratio at about −58.4% to +140.3% (formula in the example below), though it is rough at counts that small.
Google makes the same general point for user-facing A/B tests, where the time a reliable test needs will vary depending on factors like your conversion rates, and how much traffic your website gets
. Its page gives no instructions on designing tests, so we read it as no traffic rule for SEO changes.
What answer does a page get at each level of starting traffic?
A page with no clicks in the 28 days before the change gets no measurement, one with 1 to 9 clicks gets a direction and no size, and one with 10 or more clicks gets a size with its 95% interval and p-value, as our methodology page sets out.
| Clicks before the change | What the answer contains |
|---|---|
| None | "New page": no baseline, so no lift to measure |
| 1 to 9 | A direction (improved, declined or too close to call) and no size |
| 10 or more | A size, its 95% interval and p-value, or "we cannot tell" plus the smallest lift it could have detected |
The size is withheld under 10 clicks because Any percentage from a starting point that small is mostly the luck of its first few clicks.
We tested a lower line of 5 clicks and kept 10: the intervals for 5 to 9 clicks covered the true value 97.78% of the time on average but only 89.65% in the worst scenario, short of the 95% that a 95% interval is meant to cover (our reading). The 95% interval held the true lift at least 94.70% of the time for pages with 1.4 clicks a day or more, about 39 clicks in 28 days (1.4 × 28 = 39.2); the methodology page gives no such figure for 10 to 39 clicks, so a size there deserves extra doubt (our reading).
How often does a test find a real change at each level of traffic?
It depends on the size of the lift as much as on traffic: with six comparison pages and no drift between pages, our estimator found a real +80% lift on a 4-clicks-a-day page 92.01% of the time but a +50% lift only 61.23% of the time. In statistics this is power, the probability of detecting an effect
that actually exists. All rates here come from simulated sites, and 4, 40 and 400 clicks a day are 112, 1,120 and 11,200 clicks in 28 days (daily clicks × 28).
| Clicks a day | Real lift | Drift | Pages | Found |
|---|---|---|---|---|
| 4 | +50% | none | 6 | 61.23% |
| 4 | +50% | none | 8 | 70.02% |
| 4 | +80% | none | 6 | 92.01% |
| 40 | +30% | none | 6 | 99.50% |
| 400 | +10% | none | 6 | 99.95% |
| 40 | +80% | 15% | 6 | 78.90% |
Eight comparison pages instead of six raised the +50% case from 61.23% to 70.02%, a gain of 8.79 points (our subtraction). The 92.01% only just meets the 92% target we set (184,022 of 200,000 simulated experiments against the 184,000 needed), and an exact textbook t test on the same data found 90.79%, so the estimator is not losing power that a standard test would keep.
The first five rows have no drift between pages, so they are best cases, and none covers a +10% lift on a small page. The calculator computes one for your numbers, and the guide to the minimum detectable effect shows how to read the smallest lift a test can see.
Why does ten times the traffic not give ten times the power?
Because traffic shrinks counting noise but not drift, the amount that untouched pages move apart from each other on their own. Our calculator calls it tau, describes it as the noise a real change has to stand out from, and says it does not shrink as a page gets bigger
. The bar for a result is higher on a site whose pages wander, because we assume the changed page can drift as far as the untouched ones.
| Clicks a day | Clicks in 28 days | Found | Gain over the row above |
|---|---|---|---|
| 1.4 | 39 | 36.95% | first row |
| 4 | 112 | 59.65% | +22.70 points |
| 40 | 1,120 | 79.55% | +19.90 points |
| 400 | 11,200 | 82.20% | +2.65 points |
Ten times the traffic, from 40 to 400 clicks a day, added only 2.65 points (82.20 − 79.55, our subtraction). With 35% drift the same four scenarios gave between 17.35% and 26.25%, so none found the lift more than 26.25% of the time. The 78.90% in the power table above is the same 40-clicks-a-day case run again on another seed, so any one figure can be a point or so off.
The calculator offers three drift settings, very alike (0.05), typical (0.15) and volatile (0.35), and suggests typical if you are not sure. Reading tau too low makes every chance look better than it is.
How many comparison pages does a test need, and how many clicks must they have?
A test needs at least three untouched comparison pages and uses at most eight, and each must clear a small-count floor that differs between our tools. With fewer than three usable pages there is no verdict, however much traffic the changed page has.
| Rule | Applies to |
|---|---|
| 10 or more clicks before the change | The changed page: it earns a size, while 1 to 9 earns a direction only |
| 5 or more clicks in days −56 to −29 | A comparison page in the paid product and the test calculator's simulations; for a sized answer it must also be within ten times the changed page's size |
| 10 or more clicks across both windows together | A comparison page typed into the free SEO significance calculator |
| 3,000 impressions in the trailing 90 days | The decay check only, not the effect estimator |
Choose the pages and write them down before you read their results; the guide to testing without page groups explains why.
How do you check a page before you change it?
Read the page's clicks for its last 28 days in Search Console, then enter them in the SEO test calculator with your comparison pages and a drift setting before you make the change.
- Read the clicks. In the Performance report (Search results), set a custom 28-day range that has fully ended, filter to the page and read the Clicks total; a click is
The number of times a user clicked your site from Google Search results.
Take every figure the same way for every page. - List three to eight pages you will leave alone, with their clicks over the same 28 days. The calculator keeps the 30 closest in size to your page and uses at most 8.
- Pick a drift setting. Typical (0.15) is the suggested start; or estimate tau from a page-by-day file of at least 84 days, which our calculator page says Search Console's own Export button does not produce.
- Read the chances. The calculator runs our own estimator on simulated pages shaped like yours and prints the chance of a clear verdict for lifts from +10% to +150% and for drops, the change that gets an 80% chance (a common target for a test's power) and when the answer arrives: 38 days after the change goes live (28 + 7 + 3 = 38, the 7 being our own allowance for the newest days to settle). If the chance for the lift you can realistically expect is low, expect "we cannot tell".
Example (illustrative numbers)
These counts are invented to show the arithmetic; they are not data from any site, and the ranges are a best-case calculation of ours, not Penlid's interval. Three pages on a site with typical drift (tau 0.15) earn 9, 52 and 150 clicks a day, called Pages X, Y and Z. Page X is worked in full; Pages Y and Z follow the same steps.
- Clicks in 28 days = clicks a day × 28, so 9 × 28 = 252, 52 × 28 = 1,456 and 150 × 28 = 4,200.
- With n clicks in each window and no real change, the spread (standard error) of ln(after ÷ before) from counting alone is √(1/n + 1/n) = √(2 ÷ n). For Page X, √(2 ÷ 252) = √0.007937 = 0.08909.
- A 95% range is 1.96 spreads either side: 1.96 × 0.08909 = 0.1746, and exp(−0.1746) = 0.8398 and exp(0.1746) = 1.1908, so after ÷ before sits between −16.0% and +19.1%.
- Drift adds variance: tau 0.15 gives 0.15 × 0.15 = 0.0225, and, as Penlid does, we assume the changed page can drift as far as untouched pages. For Page X, √(0.007937 + 0.0225) = √0.030437 = 0.17446, 1.96 × 0.17446 = 0.34194, exp(−0.34194) = 0.7104 and exp(0.34194) = 1.4077, so −29.0% to +40.8%.
| Page | Clicks in 28 days | Counting noise alone | Counting noise plus drift of 0.15 |
|---|---|---|---|
| Page X (9 a day) | 252 | −16.0% to +19.1% | −29.0% to +40.8% |
| Page Y (52 a day) | 1,456 | −7.0% to +7.5% | −26.1% to +35.4% |
| Page Z (150 a day) | 4,200 | −4.2% to +4.4% | −25.7% to +34.6% |
Page Y has about 5.8 times Page X's clicks (1,456 ÷ 252 = 5.78) and a counting-only range less than half as wide, yet with drift the top of its range is +35.4% against Page X's +40.8%. A real +15% change is hidden on Page X by counting noise alone, clears that range on Pages Y and Z, and still sits inside the drift range on all three. In this best case a change must pass about +35% to stand out on Page Y, and the real test needs more because it must also estimate the drift from only three to eight pages.
What this cannot tell you
- These rates come from simulated sites. We measured them by running the estimator on simulated sites where the true answer was known, with 2,000 experiments per scenario unless stated, so any one figure can be a point or so off. Counts are modeled as Poisson, and real daily counts are somewhat more variable.
- They hold only inside the grid we tested. That grid went to a drift of 0.35, 11,200 clicks in 28 days (400 a day) and 8 comparison pages. The published rates are stated in clicks and we do not extend them to impressions, and the calculator simulates neither seasonality nor regression to the mean; pages picked because they were declining come back falsely "improved" more often, as the guide to false wins in SEO testing explains.
- Your count stands in for a steady rate. A page that is growing or decaying will not have the count you typed in its real before window, and the answer changes with it.
- The thresholds are judgment calls. Our methods list says so about numbers such as 10 clicks, and 80% power is a convention: Wikipedia notes there are
no formal standards for power
. - A clear verdict is timing evidence, not proof. Enough traffic can show that a page moved more than the untouched pages predicted, not why, and it cannot say a change will improve a ranking.
Questions people also ask
How do you measure SEO success?
Compare the pages you changed with pages you left alone over identical dates, and read the size of the gap with its interval instead of a raw rise in traffic. The guide to confidence intervals in SEO reporting shows how to read an interval. The changed page needs enough starting clicks for any of this to work, which for Penlid means 10 or more in its previous 28 days, and the free SEO significance calculator does the arithmetic from clicks you type in.
How do you check how much traffic a page gets?
Open Search Console's Performance report for Search results, filter to the page, choose a date range and read the Clicks total, which counts clicks from Google Search results. Search Console can disagree with other tools, and Google's help page on data discrepancies explains why. The table shows at most 1,000 rows, so filter to the page if it is missing.
Which traffic metric should you use to measure an SEO change?
Use clicks where you can. Google defines a click as a user clicking through from Search results, and the rules and rates in section 4 of Penlid's methodology are stated in clicks. The calculator can also run on impressions, but this guide does not extend those rates to them. Click-through rate is a ratio of two counts, so it carries the noise of both.
How do spikes in traffic affect an SEO measurement?
A spike that lifts every page, such as a season, a holiday or a Google update, moves the changed page and the untouched pages on the same days, so a comparison over identical calendar days can remove it. A spike that hits only the changed page, such as a new link or a news mention, cannot be removed. On a site whose pages drift widely, a real change also needs to be larger to show.
Sources
- SEO test calculator, Penlid, read 30 October 2026.
It is the noise a real change has to stand out from, and it does not shrink as a page gets bigger.
- Methodology: how we measure, and what we can't, Penlid, read 30 October 2026.
Any percentage from a starting point that small is mostly the luck of its first few clicks.
- SEO significance calculator, Penlid, read 30 October 2026.
- Performance report (Search results): Overview and basic setup, Google, Search Console Help, read 30 October 2026.
The number of times a user clicked your site from Google Search results.
- Performance report (Search results): Troubleshooting data discrepancies, Google, Search Console Help, read 30 October 2026.
- Minimize A/B testing impact in Google Search, Google, Search Central, read 30 October 2026.
The amount of time required for a reliable test will vary depending on factors like your conversion rates, and how much traffic your website gets
- Power (statistics), Wikipedia, read 30 October 2026.
In frequentist statistics, power is the probability of detecting an effect (i.e. rejecting the null hypothesis) given that some prespecified effect actually exists using a given test in a given context.
Indeed, although there are no formal standards for power, many researchers and funding bodies assess power using 0.80 (or 80%) as a standard for adequacy.
- Poisson distribution, Wikipedia, read 30 October 2026.
The positive real number λ is equal to the expected value of X and also to its variance.