An inconclusive SEO test means the evidence cannot show whether your change did anything, which is different from showing that it did nothing. Read the smallest lift the test could have detected, check whether the page was big enough to show the change you hoped for, and plan a better-sized test rather than re-slicing this one.
- A page with 10 or more clicks in the 28 days before a change gets a size, a 95% interval and a p-value, one with 1 to 9 gets a direction only, and one with none gets no measurement (Penlid, methodology, read 30 October 2026).
- In Penlid's simulations with no drift between pages, a real +50% lift on a page earning 4 clicks a day was found 61.23% of the time with 6 untouched pages and 70.02% with 8 (Penlid, methodology, read 30 October 2026).
- Penlid reads a change 38 days after it goes live (28 + 7 + 3, a window, an allowance and reporting lag), so a new test means another 38-day wait (Penlid, SEO test calculator, read 30 October 2026).
- Google says no change is guaranteed to have a noticeable effect in Search results (Google, Search Central, read 30 October 2026).
What does an inconclusive SEO test result mean?
An inconclusive result means the measurement could not separate your change from chance, so it supports neither "it worked" nor "it did nothing". Wikipedia's articles on statistical power and evidence of absence give the reason: an underpowered study is likely to be inconclusive, and whether a null result counts as evidence of absence turns on whether the experiment would have detected the effect if it were there.
For a page that had any clicks in the 28 days before the change, Penlid's tools return an inconclusive answer in three forms, set by how many clicks it had and by how many untouched pages could be used. The Penlid methodology adds that on a sized result that cannot be told from chance, the result also says the smallest lift the measurement could have detected
.
| Clicks before the change | The answer | What it contains |
|---|---|---|
| 10 or more | "We cannot tell" (the SEO significance calculator prints "Too close to call") | Lift, 95% interval, p-value and the smallest lift it could have detected |
| 1 to 9 | Too close to call | No size, and a gap within what chance produces |
| 1 or more, but fewer than 3 usable untouched pages | Inconclusive or not measurable | The reason, and no number |
The 95% interval and the p-value come from one distribution, so an interval that includes zero and a p-value of 0.05 or more are the same statement. Neither means your change probably did nothing: Wikipedia notes that a p-value is not the probability that the null hypothesis is true.
Why do SEO tests come back inconclusive?
SEO tests come back inconclusive when the page has little traffic, the change is small, the untouched pages drift apart, or too few of them qualify; sometimes the change really did little, and one result cannot say which. The table is from the Penlid methodology, which measures the first four on simulated sites where the true answer was known.
| Clicks a day | Real lift | Drift between pages | Untouched pages | Found |
|---|---|---|---|---|
| 4 | +50% | none | 6 | 61.23% |
| 4 | +50% | none | 8 | 70.02% |
| 4 | +80% | none | 6 | 92.01% |
| 40 | +30% | none | 6 | 99.50% |
| 1.4 | +80% | 15% | 6 | 36.95% |
| 4 | +80% | 15% | 6 | 59.65% |
| 400 | +80% | 15% | 6 | 82.20% |
Our arithmetic on the table: more traffic reveals smaller changes (+30% at 40 clicks a day is found 99.50% of the time, +50% at 4 a day 61.23%), and two more untouched pages add 70.02 − 61.23 = 8.79 points. Drift costs more: at 4 clicks a day a +80% lift falls from 92.01% with no drift to 59.65% with 15% drift, a loss of 92.01 − 59.65 = 32.36 points. Traffic cannot fully repair that, because, as the SEO test calculator page notes, drift between untouched pages does not shrink as a page gets bigger: at 400 clicks a day the figure only reaches 82.20%.
The rest comes from the rules. A real change smaller than the lifts in that table will often come back as a cannot-tell answer. With three to eight usable pages the amount of drift between pages is itself uncertain, and, in the methodology's words, the interval says so by being wider
. With fewer than three, Penlid gives no number: We do not fall back to a bare before-and-after number and let it imply a cause
.
What should you do next, in order?
Take five steps in this order: read the result closely, leave the setup alone, size the test, plan one that can answer, and decide on other grounds if the page cannot be measured. The order is our own reasoning from Penlid's numbers, not a tested procedure.
- Read the result closely. Find which answer in the first table you got and read the 95% interval as the range of lifts the data fit. Treat the smallest detectable lift as the line a lift must cross to read "improved", not as the lift the test reliably sees: the SEO test calculator page says a real lift of that size is caught only about half the time when there are enough untouched pages.
- Leave the setup alone. Do not swap the untouched pages, move the windows or switch metrics until something clears p of 0.05. The SEO significance calculator refuses more than 8 comparison pages because
choosing the best 8 after seeing the result would choose the answer
, and each extra re-cut is another chance for a false win. - Size the test with the calculator. Give the SEO test calculator the page's clicks over its last 28 days, the number and size of your untouched pages, and how differently your pages move (very alike 0.05, typical 0.15, volatile 0.35). It prints the chance of a clear verdict for lifts from +10% to +150% and for drops, and the change that gets an 80% chance. If that change is larger than any lift you could expect, the page cannot answer the question at its size.
- Plan a test that can answer. The levers in Penlid's numbers are more traffic on the changed page, a bigger change, and a full set of 8 untouched pages that drift little from one another. Untouched pages must be left alone in both windows and earn clicks from the same kind of query; "not measurable" means fewer than 3 were usable. Choose the pages to change without looking at their trends (choosing pages to change without bias) and write the untouched pages down before reading their numbers (control pages). Penlid refuses to measure a page changed twice within 56 days, so a second change to the same page has to wait.
- If the page cannot be measured, decide on other grounds. A page with fewer than 10 clicks before the change gets a direction at best, and the methodology explains why:
Any percentage from a starting point that small is mostly the luck of its first few clicks
. Keep or revert the change for editorial reasons, and record the outcome as "no detectable effect at this sensitivity", never as "no effect".
Example (illustrative numbers)
These counts are invented to show the arithmetic; they are not data from any site. A page that earned 115 clicks in the 28 days before a rewrite, about 4 a day, earned 142 in the 28 days after. Six untouched pages were counted over the same dates.
| Page | Before | After | Change |
|---|---|---|---|
| Changed page | 115 | 142 | +23.5% |
| Untouched page A | 96 | 104 | +8.3% |
| Untouched page B | 121 | 118 | −2.5% |
| Untouched page C | 91 | 97 | +6.6% |
| Untouched page D | 138 | 145 | +5.1% |
| Untouched page E | 104 | 98 | −5.8% |
| Untouched page F | 127 | 135 | +6.3% |
| Six untouched pages together | 677 | 697 | +3.0% |
A plain before-and-after reads 142 ÷ 115 = 1.235, so +23.5%. The six untouched pages together went from 677 to 697 clicks, and 697 ÷ 677 = 1.030, so they predict +3.0%. Against that prediction the page did 1.235 ÷ 1.030 = 1.199 times as well, which is +19.9% versus what the untouched pages predict.
Typed into the SEO significance calculator, the same counts give +19.9% with a 95% interval of −8.5% to +57.0% and p = 0.19. The interval includes zero, so the answer is too close to call. The smallest lift it could have detected is about +31.0%, above the +19.9% seen, so our reading is that a rewrite really worth about 20% would usually have come back like this one.
Now make every count ten times larger: 1,150 and 1,420 clicks for the changed page, and ten times each untouched page's counts. The page-by-page percentages are identical and the hand figure is still +19.9%, but the calculator, which works from each page's log ratio, gives +20.0% with a 95% interval of +2.9% to +39.9% and p = 0.03: improved. The next step for the 115-click page is a larger page or a full set of 8 untouched pages, not a re-cut of these six.
What this cannot tell you
- It cannot turn "we cannot tell" into an answer. Re-cutting the same data adds no information; this guide is about the next test.
- The rates come from simulated sites, not yours. Penlid measured them where the true answer was known, its drift presets are judgment calls, and a site that drifts more than the preset you chose will detect less than shown.
- The calculator leaves out seasonality and regression to the mean. A page picked because it was declining was called "improved" 5.50% to 9.20% of the time in Penlid's simulations when nothing was done to it; see regression to the mean.
- A longer wait is untested. Penlid's windows are fixed at 28 days and nothing in the methodology measures a longer one, so we do not claim that waiting narrows the interval; see how long to wait after a content update.
Questions people also ask
What is SEO testing?
SEO testing means changing a page, or a set of pages, and checking whether clicks or impressions then moved more than they would have anyway. Penlid's calculator page notes that generic A/B calculators assume visitors are split at random between two versions, which a page edit is not, so an SEO test usually sets the changed page against pages you left alone over the same dates. The answer can be a gain, a loss or too little evidence to call.
How do you know if SEO is working?
SEO is working when the pages you changed gain more than comparable pages you left alone, over the same dates, by more than chance explains. When a test cannot reach that bar, the honest status is unknown rather than failing: Google's help page says it can be hard to tell whether a page change caused a gain, because other events also move performance. The guide to telling whether a content update worked covers the comparison.
How do you monitor the effectiveness of SEO?
Write down each change with its page and date before you make it, keep a set of pages you will not touch, and read the result only after its windows have fully ended. Keep the weak results in the log too, with the smallest lift each test could have detected, so that a later reader can tell a change that failed from a test that was too small to see anything.
What mistakes should you avoid when you test an SEO change?
Mistakes that leave a result unreadable or misleading include testing a page with very few clicks, changing a page twice within 56 days, choosing the untouched pages after seeing their numbers, picking pages because they were declining, and treating a gap that chance could explain as a win or a loss. The guide to choosing pages to change without bias covers the picking mistakes.
Sources
- Google Search's core updates and your website, Google, Search Central, read 30 October 2026.
- Performance report (Search results): Common tasks and use cases, Google, Search Console Help, read 30 October 2026.
- Power (statistics), Wikipedia, read 30 October 2026.
- Evidence of absence, Wikipedia, read 30 October 2026.
- p-value, Wikipedia, read 30 October 2026.
- Methodology: how we measure, and what we can't, Penlid, read 30 October 2026.
the result also says the smallest lift the measurement could have detected
We do not fall back to a bare before-and-after number and let it imply a cause
the interval says so by being wider
Any percentage from a starting point that small is mostly the luck of its first few clicks
- SEO test calculator, Penlid, read 30 October 2026.
- SEO significance calculator, Penlid, read 30 October 2026.
choosing the best 8 after seeing the result would choose the answer