SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
•
13 likes•6,504 views
This document discusses the importance of split testing in SEO. It provides examples of split tests conducted by various large websites like Pinterest and Etsy. These examples show that traditional SEO best practices sometimes do not work, while other untested changes can significantly improve organic traffic. The key lessons are that search engine algorithms and ranking signals have become more complex, user-centric and interconnected, making testing critical for effective SEO. Hypothesis-based testing ensures changes are targeted and helps avoid wasting time on ineffective optimizations.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
Similar to SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us (20)
More from Tom Anthony
More from Tom Anthony (15)
Recently uploaded
Recently uploaded (20)
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
- 1. SEO TESTS ON BIG SITES & SMALL WHAT ETSY, PINTEREST AND OTHERS CAN TEACH US Tom Anthony
- 2. WOW OPENINGNBED 2 NBED SOME WOWNESS A COUPLE OF IDEAS BELOW FIRST DEMONSTRATES IT CAN BE DANGEROUS TO TRY SOMETHING WITHOUT KNOWING SECOND IS LIKE PEOPLE WHO BLOG STUFF AND CONVINCE OTHERS - BUT ARE WILDLY WRONG
- 3. 3
- 4. @TomAnthonySEO #SMX THIS IS WHY MEDICINE HAS CONTROLLED STUDIES
- 5. @TomAnthonySEO #SMX INTRO ODN WE RUN TESTS NBED
- 6. @TomAnthonySEO #SMX USERS ARE SPLIT INTO TWO GROUPS ‘TRADITIONAL’ CRO A/B TESTING Visitors A & B: control version of a page Visitors C & D: test version of the same page
- 7. @TomAnthonySEO #SMX SPLIT A GROUP OF SIMILAR PAGES INTO TWO GROUPS SEO SPLIT-TESTING Pages 1, 2, 3: Control Layout Pages 4, 5, 6: Test Layout All users see the same as each other
- 8. @TomAnthonySEO #SMX IMPORTANT - GOOGLEBOT IS TREATED NO DIFFERENTLY SEO SPLIT-TESTING Pages 1, 2, 3: Control Layout Pages 4, 5, 6: Test Layout Googlebot sees the same as users
- 9. @TomAnthonySEO #SMX LASTLY - MEASURE WHICH SET OF PAGES GOOGLE PREFERS SEO SPLIT-TESTING
- 10. @TomAnthonySEO #SMX A 12% UPLIFT FOR THIS UK RETAILER ADDING SCHEMA.ORG PRODUCT REVIEWS
- 11. @TomAnthonySEO #SMX Early on we found many SEO ‘best practice’ changes don’t help.
- 12. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. Dark Ages
- 13. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. Dark Ages Middle Ages
- 14. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. Dark Ages Middle Ages Renaissance
- 15. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. Dark Ages Middle Ages Renaissance Ranking signals become external. Knowing about your site is not enough.
- 16. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 WMT/GSC. Link indexes. Rank tracking. Dark Ages Middle Ages Renaissance Industrial Revolution
- 17. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Dark Ages 2 Middle Ages Pre-Google. Keyword stuffing reigns. 3 Renaissance Google arrives. PageRank. Links. 4 Industrial Revolution Pre-search engine, big list of sites. WMT/GSC. Link indexes. Rank tracking. The rise of data… Cannot be effective without these external data sources.
- 18. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. The Google algo becomes more complex: • Machine Learning • Explosion of Signals • User Signals
- 19. @TomAnthonySEO #SMX SHEER SCALE AND THE INTRODUCTION OF MACHINE LEARNING HAVE CHANGED IT… GOOGLE’S ALGO IS NO LONGER PREDICTABLE GOOGLE ALGO USED TO BE LOTS OF INDEPENDENT LEVERS BUT NOW IT IS A COMPLEX SYSTEM. IMPOSSIBLE TO PREDICT. Keyword in H1 # of Links Domain Age + + + Relevancy Score Site Speed Social Signals Domain Age Domain Trust User Signals Text above fold Structured Data # of Links Ad Density Keyword in Title
- 20. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. So, we have data, but it is hard to understand how to turn that data into decisions.
- 21. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking.
- 22. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Since 2014, we’ve been headed into this stage.
- 23. @TomAnthonySEO #SMX LOTS OF DATA - BUT HARD TO KNOW THE BEST DECISIONS TO MAKE WITH IT So many companies have started split-testing…
- 24. @TomAnthonySEO #SMX A GUIDE TO SPLIT-TESTING THE NEXT ERA OF SEO
- 26. USER SIGNALS ARE RANKING SIGNALS1 2 RANKING SIGNALS ARE NO LONGER LINEAR 3 DATA TO HYPOTHESIS TO ACTION
- 27. USER SIGNALS ARE RANKING SIGNALS1
- 29. @TomAnthonySEO #SMX OBSERVATION Search Console detected many duplicate title tags on board pages: e.g. “{board_name} on Pinterest” Test #1
- 30. @TomAnthonySEO #SMX ION tected many n board pages: on Pinterest” TEST IDEA Reduce duplicate title tags by including Pin counts in the title tag. e.g. “{board_name} on Pinterest | ({number} Pins) Test #1
- 31. @TomAnthonySEO #SMX RESULT No statistically significant improvement in traffic. EA tags by including e title tag. est | ({number} Pins) Test #1
- 32. @TomAnthonySEO #SMX OBSERVATION “Pinterest is filled with tens of billions of Pins, but the text descriptions can be lacking” Test #2
- 33. @TomAnthonySEO #SMX ION tens of billions descriptions king” TEST IDEA Add a better description, written by users of other Pins of the same image. Test #2
- 34. @TomAnthonySEO #SMX RESULT A 30% increase in organic traffic. EA tion, written by the same image. Test #2
- 35. @TomAnthonySEO #SMX RESEARCH PAPER FROM GOOGLE IN 2010 GOOGLE IS USING USER SIGNALS http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36363.pdf
- 36. @TomAnthonySEO #SMX GOOGLE IS USING USER SIGNALS “Another direction for future work is to incorporate active learning in order to gather a more representative sample of user preferences.” RESEARCH PAPER FROM GOOGLE IN 2010
- 37. @TomAnthonySEO #SMX BLOG POST FROM DAN PETROVIC — 3YEARS AGO GOOGLE IS USING USER SIGNALS https://moz.com/blog/user-behaviour-data-as-a-ranking-signal
- 38. @TomAnthonySEO #SMX EXPERIMENT BY RAND FISHKIN SHORT TERM CTR AFFECTS RANKINGS https://twitter.com/randfish/status/612730826615230464
- 39. Reduce duplicate title tags by including Pin counts in the title tag. Test #1 No statistically significant improvement in traffic. Why? Add a better description, written by users of other Pins of the same image. Test #2 A 30% increase in organic traffic. Why?
- 40. Reduce duplicate title tags by including Pin counts in the title tag. Test #1 No statistically significant improvement in traffic. Titles matter as they help users by making it clear what a page is about. The change didn’t help users. Add a better description, written by users of other Pins of the same image. Test #2 A 30% increase in organic traffic. Better descriptions help users and bots better understand contents of a pin. The change helped users.
- 41. Understand why you are seeing warnings. Try to address the underlying issue. TAKEAWAY Doing this is more likely to have a real impact.
- 42. Users are unpredictable, and so you must test whenever they are involved. TAKEAWAY Google is using user signals as a ranking factor.
- 43. 2 RANKING SIGNALS ARE NO LONGER LINEAR
- 44. “By running a large number of experiments, we found some well-known strategies for SEO didn’t work for us, while certain tactics we weren’t confident about worked like a charm.”
- 45. Best Practices weren’t having an impact. But they were wasting time and resource trying them. TAKEAWAY
- 46. @TomAnthonySEO #SMX TEST RESULT: ALT ATTRIBUTES HAD NO IMPACT WE HAVE FOUND THE SAME MANY TIMES OVERALT ATTRIBUTE TEST FROM LASTYEAR
- 49. Shorter title tags, consisting only of the target search keyword, appear more relevant/enticing to search users. “ ” Shorter title tags can improve CTRs. TAKEAWAY
- 50. H1 tags are still important and still effect SEO performance, but… TAKEAWAY We found that an H1 change can have a statistically significant impact on organic search traffic. “ ”
- 51. @TomAnthonySEO #SMX Title uplift: 5% H1 test uplift: 5% Uplift from both: 7% TESTS SHOW THAT UPLIFTS ARE NOT INDEPENDENTEXAMPLE NUMBERS TO ILLUSTRATE
- 52. All SEO changes affect one another. TAKEAWAY Don’t think one change won’t impact others.
- 53. 3 DATA TO HYPOTHESIS TO ACTION
- 54. Some of our earliest DistilledODN tests…
- 57. @TomAnthonySEO #SMX BUT BUT BUT… BEST PRACTICE SAID USP IS A GOOD IDEA?! NOTHING HAPPENED
- 59. Before After query search volume which tv should i buy 250 query search volume which tv to buy 1,300
- 60. @TomAnthonySEO #SMX MAYBE EVEN A BIT WORSE NO REAL IMPACT
- 62. <img src="product1.jpg" alt="Home 6 Piece Towel Bale - Geo Grey / Zest">
- 63. @TomAnthonySEO #SMX WE’VE NEVER HAD AN IMPACT FROM ALT ATTRIBUTE NADA… NOTHING… NO IMPACT…
- 65. “Stand back guys, I got this.”
- 66. Before After
- 67. @TomAnthonySEO #SMX WORST TEST RESULT WE EVER HAD OH GOD…
- 68. COULD BE OUR RESULTS AREN’T ACCURATE ?
- 69. @TomAnthonySEO #SMX THE RESULTS ARE CONSISTENT THE RESULT WAS ACCURATE — WE SUCK
- 70. @TomAnthonySEO #SMX Most SEO changes have a no or a negative impact.
- 71. @TomAnthonySEO #SMX IT LEADS TO MORE EFFECTIVE TESTING CRO EXPERTS KNOW - YOU NEED A HYPOTHESIS dis.tl/cro-hypothesis
- 72. @TomAnthonySEO #SMX A HYPOTHESISES SHOULD BE SPECIFIC Changing <abc> will have an impact on <xyz> & thus improve organic search performance. Based on <evidence>, we hypothesise…
- 73. @TomAnthonySEO #SMX HYPOTHESIS EXAMPLE Changing <title formats, by…> will improve <CTR> & thus improve organic search performance. Based on <survey data>, we hypothesise…
- 74. @TomAnthonySEO #SMX CRO HYPOTHESISES FRAMEWORK https://conversion.com/blog/introducing-hypothesis-framework/
- 76. @TomAnthonySEO #SMX EXAMPLE OF HYPOTHESIS APPROACH
- 77. CATEGORY PAGE HTML ISVERBOSE, AND CONTAINS A LOT OF JUNK. ADDING A LIST OF PRODUCTS AS STRUCTURED MARKUP WILL MAKE THE INTENT CLEARER. HYPOTHESIS
- 78. @TomAnthonySEO #SMX ADD STRUCTURED MARKUP
- 79. @TomAnthonySEO #SMX AN 11% UPLIFT IN ORGANIC TRAFFICOUR HYPOTHESIS WAS ACCURATE
- 80. @TomAnthonySEO #SMX REPEATED IN ANOTHER REGION - 11% AGAINWE SEE THIS OFTEN IN DIFFERENT REGIONS
- 86. THE HYPOTHESIS DID NOT FIT THESE OTHER SITES. !
- 87. Hypothseses ensure you understand why you are making a change. A hypothesis approach leads to more effective changes, and helps avoid wasting time on ineffective changes. DATAPOINT TAKEAWAY
- 88. @TomAnthonySEO #SMX HAVE A HYPOTHESIS - YOU WILL MAKE BETTER RECOMMENDATIONS
- 89. SUMMARY
- 90. @TomAnthonySEO #SMX TO BE COMPETITIVE TESTING IS BECOMING A NECESSITY WE ARE IN A NEW ERA OF SEO 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Just as tactics from here… …wouldn’t work well here…
- 91. @TomAnthonySEO #SMX TO BE COMPETITIVE TESTING IS BECOMING A NECESSITY WE ARE IN A NEW ERA OF SEO 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Just as tactics from here… …wouldn’t work well here… … being competitive here means testing!
- 92. SEO must be user-centric, which makes testing necessary. TAKEAWAY Google includes user signals. Users are unpredictable.
- 93. TAKEAWAY Realise you are not at optimised as you think you are. Many SEO tests get uplifts from updating titles, meta descriptions & H1s etc.
- 94. Ranking signals are no longer ‘summed’ together.They effect one another. TAKEAWAY This also means that what worked for someone else, may not work for you.
- 95. TAKEAWAY Have a hypothesis for what you expect from any SEO change. You will make more effective recommendations.
- 96. Testing allows you to stop chasing ranking factors. TAKEAWAY
- 97. @TomAnthonySEO #SMX FREE TOOL TO DO AN ANALYSIS OF TESTS BLOG POST - HOW YOU CAN TEST https://www.distilled.net/resources/do-it-yourself-seo-split-testing-tool-with-causal-impact/
- 98. @TomAnthonySEO #SMX SMX MUNICH 2017 DECK MORE TEST EXAMPLES https://www.slideshare.net/TomAnthony/next-era-of-seo-a-guide-to-seo-splittesting A GUIDE TO SPLIT-TESTING THE NEXT ERA OF SEO