More and more, we’re seeing articles written about companies making use of various specialists to conduct SEO tests and experiments. Spread across written articles, YouTube videos, and conferences, we’ve reviewed over a hundred of these tests. It has become clear to us that often, SEO tests can bring new information to light, inspiring you to check certain factors that you didn’t think to check before. 

However, the main issue with these tests is that often, specialists can draw come to different conclusions even though they’re testing the same thing, as they’re conducting the tests in a different way – and by “”often”, upraszczajmy”often,” let’s simplify.a different way”, we mean the adopted methodology of a specific study.

What are the preliminary assumptions of our SEO tests?

The way we see it, if you’re interested in carrying out a test that will be valuable and worthwhile, you need to do it in the cleanest environment possible – if you can, anyway. This may not always be possible when you’re working with organic search results in Google. Regardless, here’s how we do it:

We only conduct SEO tests on domains that belong to us. We verify whether they have the potential to rank and do not have the “handbrake” pulled. This can block position increases even with a correctly performed test. We have access to server logs and Google Search Console connected, so we can track e.g. crawling, re-indexing, indexing, and similar elements. It is crucial for us to refer directly to the main database, not to a local data center. The domain has to have been online for at least 3 months and its visibility and parameters must be stable (crawl budget). It’s best if the domain does not have external links, as these can distort the test values.

Each test must be repeated at least 6 times (each on a different domain) giving the same result each time so that we can be 95% sure that a given element works exactly as we assumed. In the case of a single error, we ensure that the test result can be duplicated in 7 out of 8 cases, or 10 out of 12. 

In the case of three different results of the same test, we can’t be certain that the given factor works in the way we assumed. At this stage, we usually look for errors in the test, such as a problem with the methodology or finding that the given element has no influence and therefore the final value is random.

The time of each test lasts a maximum of 2-3 weeks. We believe that longer tests can distort the effect due to changes in the algorithm. As for tests on dates on which algorithm updates are due, in most cases we simply discard and start from scratch.

And most importantly, we always test only one specific factor / one SEO element. We do not combine tests with each other, as one element can affect the value of other tests.

What were the technical tests in our SEO methodology?

In our technical tests, we examine the behavior of Googlebot, crawling, processing URLs on the Google side, indexing, or rendering. The most important thing is to accurately verify the indexing status in Google Search Console or crawl in server logs.

We often confirm certain actions of factors based on new 404/410 error addresses. This allows us to make sure that there are specific systems in Google that are responsible, for example, for adding addresses to the crawling queue, creating the queue in a specific way, or for the amount of time and crawling of URLs during one Googlebot crawling session.

We examine some factors by analyzing the URL crawling counter, or by comparing the crawling statistics of the individual URLs being considered. For example, we examined whether Googlebot crawls pages that have more internal links more often, and how often Googlebot crawls an external canonical compared to visiting the source page.

In tests on WordPress, plugins such as Redirection are useful as they allow us to quickly and efficiently manage 404 logs or logs from created redirects. Another useful plugin is the Simple Visit Googlebot plugin, which shows the last entries of Googlebot on individual subpages in the panel itself, without downloading logs from the server.

What was our link-building methodology?

In the case of external link tests, our assumptions are as follows:

• We select one specific phrase
• This phrase must be targeted at the appropriate URL (no cannibalization and ranking problems)
• For a given phrase, more than half of the ranking addresses cannot have an external link (phrases cannot be too difficult, because the target URL must be in the index)
• We first obtain the primary link with an exact match anchor
• The initial position of the phrase must be between the 30th and 100th position
• We follow the position in Incognito mode, with the location set to a specific country
• We measure the value of the link only when the URL is in the index
• We avoid any changes on the source and target domain during the test period
• The source and target address do not have external links that can distort the test value

In the case that problems arise, we analyze the entire source and target domain in Google Search Console to ensure the test was not distorted by factors of which we weren’t aware.

We start by testing the impact of a specific domain in the following order:

• We acquire a link
• We check the effect of the acquired link
• We remove the link
• We check the drop after the link removal

The impact after adding and the drop after removal must be in the vicinity of the same position.

In the case of differentiating tests (e.g., comparing different rel attributes), we first obtain the first EMA link, and only then do we introduce changes, reindex the URL with the given link, and monitor the impact of the change. Similarly, we do this in the case of canonical vs 301, EMA vs PMA, and URL vs brand.

How did we document the results of SEO tests?

We record all the documentation in Word, and we track changes in positions using Excel sheets. On these spreadsheets, we make a record of the: 

• Date
• Position
• Target URL (we exclude ranking problems)
• Date of inserting the link
• Date of indexing the URL from the link
• Date of the link’s impact
  

Date	Position	Target URL	Link Insertion	Indexing	Impact
27.06.2023	80	url1	YES	NO	–
28.06.2023	79	url1	–	YES	–
29.06.2023	30	url1	–	–	YES
30.06.2023	30	url1	–	–	–

Which SEO tests did we perform?

Here is a list of all the interesting SEO tests and experiments we conducted over the past few months:

Technical SEO Tests

1. How to verify the rel=”canonical” tag outside Google Search Console
2. Crawling internal links in accepted canonicals
3. Crawling internal links in non-accepted canonicals
4. How Google queues URLs from the sitemap
5. The use of the last mod in sitemap.xml by Googlebot
6. The use of priority in sitemap.xml by Googlebot
7. The use of frequency in sitemap.xml by Googlebot
8. How often Googlebot processes the sitemap.xml
9. How Googlebot sees the sitemap in the text version
10. Differences in operation between sitemap.xml vs sitemap.txt
11. How 301, 302, 303, 307, and 308 redirects work
12. How Googlebot verifies redirect loops
13. How Googlebot verifies the redirect chain
14. Difference between 301 and canonical
15. How window.location.href really works
16. How the http-equiv=”Refresh” redirect really works
17. When a 302 redirect becomes a 301 redirect (temporary -> permanent)
18. Crawling of infinity scroll
19. Crawling of products visible on infinity scroll
20. Crawling of products visible only on pagination
21. Crawling of 404 pages
22. Crawling of 410 pages
23. Crawling of 404 vs 410 pages
24. How long Google process / handles URLs
25. Impact of server redirects due to location
26. How pages are visited by Googlebot
27. How often Googlebot crawls robots.txt
28. Why Googlebot doesn’t always replace the robots.txt file
29. How to create your own indexer
30. How Googlebot processes URLs in the HEAD header
31. How to force Featured Snippet information in search results
32. The difference between crawl and index time
33. The difference between index and cache time
34. Why the search results don’t show the same information as in Google Search Console
35. How Indexing API works
36. Why indexing API doesn’t always index URLs
37. How 500 errors affect crawl budget
38. Do 4** errors affect the crawl budget?
39. Do 3** redirects affect the crawl budget?
40. How many languages does Googlebot have?
41. How many links in internal linking are taken into account by Googlebot?
42. Does Googlebot always respect the robots.txt file?

Content SEO Tests

1. How many characters from Title does Google index?
2. Does h1 affect ranking?
3. How Googlebot analyzes the alt attribute in images
4. Can you use cloaking in the service?
5. Does cookie information affect SEO?
6. Is page speed a ranking factor?
7. Is mobile-friendly a ranking factor?
8. Can changing the Title ruin your SEO?
9. Do comments affect SEO?
10. How to influence the SEO of a product/article/service/category with comments?
11. Does Google index hidden content?
12. How do you index hidden content?
13. Does the number of paragraphs affect SEO?
14. Does Google crawl URLs in the content of the page?
15. Is content duplication a ranking factor?
16. When is content duplication not bad in SEO?
17. Does schema.org affect SEO?
18. Does the number of repetitions in the text affect SEO?
19. Does the phenomenon of Keyword Stuffing exist?
20. Do synonyms affect the SEO of the main word?

Link Building Tests

1. How much reputation does canonical convey?
2. How much reputation does a 301 redirect convey?
3. Canonical vs 301 – what gives more power?
4. Does having more internal links increase the reputation of a given URL?
5. Does the number of paragraphs affect the value of the link?
6. Links from pages with negative reviews
7. Links from pages with negative sentiment
8. Does linking to old posts work?
9. How do links from unindexed pages work?
10. Does an internal link from the homepage affect the value of the URL?
11. How to verify the operation of an external canonical?
12. 1 link vs 2 links from the same subpage
13. Do links to Wikipedia affect SEO?
14. Do links to social media take away reputation from the page?
15. Is a sitewide link stronger than a link from a sponsored article?
16. The effect of a link to a page with an HTTP 4** code
17. The effect of a link to a page with an HTTP 5** code
18. How quickly can you get value from a link?
19. When and how is reputation from a link transferred?
20. Does the external linking of links cause a decrease in the portal’s reputation?
21. Does the visibility of the domain affect links?
22. Does the number of incoming domains vs the number of linked domains affect the links?
23. How Googlebot reads the “alt” attribute in external links
24. Does the length of the text affect the value of an external link?
25. How do non-existent rel attributes work? dofollow, follow, user-generated content, pr10
26. How much reputation does the default link convey?
27. How much reputation does a link with the rel=”nofollow” attribute convey?
28. How much reputation does a link with the rel=”sponsored” attribute convey?
29. How much reputation does a link with the rel=”ugc” attribute convey?
30. Comparison of follow links to nofollow links
31. How does Google evaluate REL attributes?
32. Does target=”_blank” affect SEO?
33. Does link echo exist?
34. How much reputation does a URL in content (ghostlink) convey?
35. How Googlebot crawls 301 redirects between domains
36. Does the second link from the same domain to the same URL affect SEO?
37. Does the second link from the same domain to a different URL in the same domain affect SEO?
38. Can linking from the outside ruin the value of a link?
39. The effect of EMA / PMA / URL / Brand links
40. The difference in link reputation EMA vs PMA
41. The difference in link reputation EMA vs URL
42. The difference in link reputation EMA vs Brand

In summary, our SEO methodology emphasizes conducting experiments in a controlled environment to ensure reliable and valuable results. We only test on our own domains with specific criteria, such as stability and absence of external links. 

Each test is repeated many times to achieve a 95% certainty of the results. We focus on one SEO element at a time and avoid combining tests to avoid data distortion. Our technical tests cover various aspects, including crawling, indexing, redirects, and sitemap behavior. 

Additionally, we delve into content and link-building tests, exploring factors like URL indexing, link reputation, and the impact of different link attributes. And in front of us is a list with over 500 different factors to test

Let us know what you think of our methodology – especially if you think we’ve forgotten something!