Getting indexed by Google is an essential step in the process of allowing people to find your website via search, but there can be some issues with getting your pages indexed. Indexing is the process where Google attempts to understand the content of your page. Titles, headings and text are the main ways that Google makes sense of your page content.
Websites often find that some of their pages never get indexed or are subject to long delays.
Unfortunately, not being indexed means your pages will not show up in search results and ultimately this will reduce the traffic to your site.
What factors affect indexing by Google?
Indexing is generally affected by similar factors to ranking such as content quality and including links.
Other factors are more technical and tend to affect those sites that utilise more web technologies.
Consistency with technical signals is important but the quality of your content is often the more critical issue. It is estimated that about 15% of pages across the web are not successfully indexed.
The following are the most common factors in indexing:
Crawled – currently not indexed
This message denotes that Google has found a page and visited but has not indexed it. More than likely this means that there is a problem with the quality of the content.
Google is becoming ever more discerning when it looks for quality content, so it is important to remember to use unique titles along with unique descriptions and copy.
How to fix this issue
Don’t copy product descriptions from other sources as this will cause the content to not be unique. Canonical tags can avoid duplication of content within your own site. You can also prevent Google from crawling the lower-quality parts of your site by using robots.txt file or the noindex tag to block Google.
Discovered – currently not indexed
This status can be caused by a variety of issues including:
Crawl budget issue – where too many URLs are in the queue and left to be crawled at a later date.
Quality issue – where Google decides that the pages are too low quality to crawl and index.
How to fix these issues
Look for patterns that are causing pages to have these issues. It may be a particular category of products with the problem that are not linked internally, or your pages may be all waiting in a queue.
You can optimize your crawl budget by identifying the low-quality pages that Google spends its time on. Things like internal search pages can take up Google’s time and prevent it from indexing the quality pages that will bring visitors to your site.
Duplicate content
There are lots of reasons why your site may contain duplicate content. For example, if you have produced separate content in English for the UK, the US and Australia the pages may end up unindexed due to duplication.
E-commerce sites often fall into the trap of duplication as they need to use product descriptions for items that are also listed by competitors.
How to Fix this Issue
Consider using creative ways to avoid duplication by creating your own style for descriptions that is unique. One idea is to use frequently asked questions to enhance descriptive content. Not only is it useful for customers who have questions, but it produces detailed, informative information from experts that is unique. Google will like this.
Understanding the Index Coverage Report
You can use the Google console to discover whether your website pages have been indexed.
Once you have found this report in Google Search Console you need to use some simple analysis to make sense of the information and what it means for your site.
Focus on the number of excluded pages that have not been indexed and look for a pattern that links them together.
Large commercial websites can expect to find many product pages listed here since there will be duplications and out of stock products that lack the quality for Google to index. What you want to concentrate on is finding out if any more important pages are listed.
Helping Google to Index your Pages
Whilst every website will have different causes for Google not indexing their pages the most common things that can be fixed are:
- Soft 404 signals
Soft 404 signals include things like “Not found”, “Not available” in the content of “404” in the URL. These will cause a false “404” status and will lead to Google not indexing the page.
- Internal linking
Make sure that all your site pages are included in your sitemaps to ensure that Google gets the signal that a page is important. These internal links are vital for essential pages to be visited and indexed by Google.
- Crawling strategy
Optimize your crawling strategy by ensuring that Google focuses on the important parts of your website rather than wasting your crawling budget on low-quality pages. You can use the server log analysis to identify any problems with low-quality pages taking up Google’s valuable time.
- Avoid low-quality or duplicate content
You will inevitably have some pages that you don’t want to be indexed, so make sure you use the noindex tag or robots.txt file to tell Google not to index them. You don’t want Google to assume that your lower quality pages reflect the quality of your whole site.
- Consistent SEO signals.
Inconsistency between using HTML and JavaScript can cause technical issues for Google that may make your pages undiscoverable.
- Accept that not all your pages will be indexed
Google itself has stated that with the growth of the internet it is impossible to index or even visit every page on the web. This means you cannot expect Google to index every page on your site, but it is important to make sure that Google visits and indexes the most important pages for your business. Focus on those essentials and you will ensure that your site appears in search results and gets the traffic it needs to drive business and sales.
Visit our blog to find more helpful information.
