Getting your web pages indexed by Google is essential if you want your content to appear in Google search results.
Google continually crawls the web to look for new and updated pages. It then analyses and stores them in its vast database.
To rank in SERPs, you need Google to choose your page from its index in response to relevant queries. Google algorithms determine ranking, but you can affect indexing by making your content more discoverable.
Is my site indexed by Google?
It is easy to check if Google has indexed your site. There are two main methods:
1. Google Search Bar Method.
- Type in “site:” followed by your domain name
- g. site:myblog.com
- Look at the results.
- All the pages shown have been indexed.
- No results mean your website has not been indexed.
It’s an easy and fast method, but it is difficult to spot if any pages on your website are missing from the list of indexed pages.
2. Google Search Console Method
It is slightly more complicated to check if your site is indexed on Google Search Console(GSC), but you can collect more information to help resolve issues.
You will first need to create an account and verify the ownership of your website. Next, you must create a sitemap which lists all the URLs you want to be indexed.
If you aren’t sure whether you have a sitemap, you can type the following into the search bar to check:
https://yourdomain.com/sitemap.xml
If no sitemap is found, you will need to follow the instructions set out here by Semrush to create one.
Once you have a sitemap, you can enter the URL in the sitemaps board on Google Search Console. If the sitemap works, you will see a green success status.
You can now also check the status of individual pages using the google search console URL inspection tool. You will get one of the following responses:
- URL is on Google – the page has been crawled and indexed. You can see information such as the last crawl date.
- URL is not on Google – the page is not indexed and will not appear in search results.
What can I do if a page has not been indexed?
If you discover that a page on your website has not been indexed, you will need to take action, or it will not appear in search results.
There are a number of reasons that may have prevented a page from indexing. The reason may show in Google Search Console, or you may need to conduct an SEO audit on the page that has not been indexed.
Here are some common reasons for a page not being indexed and how to solve them:
· Your page is newly published, or your website is new.
It can take a few weeks for crawling to take place, but you can speed up the process by requesting indexing in Google Search Console.
· You have used a robots.txt file to block crawling
A robots.txt file is used to tell Google not to crawl parts of your website, such as private pages, duplicate pages or PDFs.
Sometimes your robot.txt file will prevent Google from crawling anywhere on your site, which means it will not be indexed at all.
Check your Robot.txt file for errors to remove the block to crawling.
· You have noindex tags
A page with a noindex meta tag will not be indexed. You can check for noindex tags in Google Search Console.
Go to indexing, pages, and report. Choose the “excluded by noindex tag” section. You will see a list of URLs that have been excluded by a noindex tag.
You will have to remove the tag from the source code of any pages you wish to be indexed.
· You have canonical tags
Canonical tags advise crawlers which version of a page they should crawl to prevent duplication.
Use Google Search Console to check for a list of pages that have canonical tags. If any on the list should be indexed, you will need to remove the canonical tag from the page’s source code.
· Broken or insufficient internal links
Internal links are helpful to Google to navigate your pages.
No more than three clicks must be required to reach the relevant page, or it will not be crawled. Orphan pages, ones with no internal links, will also sometimes be ignored by Google crawlers.
Fixing or improving internal links can help Google to index your pages.
· General Website Issues
Indexing issues can also occur if your site is not mobile-friendly, is slow to load or suffers from redirects.
When deciding to index your pages, Google also considers the quality of your content, including your authority, expertise and backlinks. Publishing quality content will mean that Google is more likely to index it.
If you want some help with indexing and improving the ranking of your website pages, get in touch with us at SEO and Web to sort out any technical issues and improve your SEO.
