Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal. 

First, we asked our Twitter followers to vote:

This time the majority guessed it right. The result was negative.

Read the full case study to find out why.

The Case Study

Greetings, SEO enthusiasts! 

Have you ever had a conflict over whose recommendation was the best when it came to an SEO question? We recently worked with a client who thought perhaps their page titles were a little too long, and they wanted to reduce the overall length by replacing the existing page title with just the primary product name H1. The hypothesis? We could easily test if the longer H1 was not earning the clicks a shorter H1 would.

Who do you think was right? Let’s test it, shall we?

Hypothesis

The working hypothesis was to validate, or not, the argument that reducing the overall length of the page titles of our client’s pharmaceutical description pages to just the primary H1 of the page would increase clicks to the tested pages.

The Test

We set up our test in SplitSignal. 

1. Split Pages – We selected a total of approximately 170 pages for this test. Fifty percent (50%) of the pharmaceutical pages were selected as the test variant, while the remaining pages would serve as the control group.

SplitSignal captured the primary H1 from each page and set them as a variable we could use to replace the existing H1, which was over 70 characters on average, to a much shorter version with just the **** name followed by a dash and the brand name, e.g.:

The Results

After running the test for 21 days, we were able to determine that there was a decrease in clicks by -29.4% or 14,789 clicks in the test group. You can observe the decline in clicks (orange line) in the following screenshot from the SplitSignal dashboard:

So, what happened?

As we said in the beginning, the purpose of this test was to conclusively end the argument that reducing the overall length of the page title would have a beneficial effect on overall clicks. While we doubted the above to be true, we’re not the sole possessors of all SEO knowledge, so off to test we went.

The overall reduction of the page title length, while it did include the major on-page keywords found in the H1, lost a lot of context and additional relevant phraseology and topical keywords. It’s interesting to note that there was no significant change in the average ranking between the test and control group, meaning users found the new title less compelling than the original titles, which, in our opinion, caused a drop in click-thru rate. (Please note: this test was run prior to Google’s August release of the controversial page title rewrite.)

This is a good example of how a negative test result can actually provide financial benefit to both agency SEOs and in-house SEOs. Rather than spend a large amount of time and budget to programmatically (or manually) rewrite page titles, we were able to show this would have ultimately wasted working hours and resulted in a loss of income as well.

What do you think? Have you ever implemented something your team felt was the right move, only for it to ultimately be a failure once implemented? Let’s talk about it on Twitter!

Have your next SEO split-test analyzed by LOCOMOTIVE Agency.