Google has shared more details on the indexing issues it experienced over the past few months, which could explain why some websites witnessed less traffic on specific days. 

In a blog post, Vincent Courson from Google Search Outreach explained the search engine giant’s recent problems, how it managed to fix them, and lessons learnt moving forward. 

Google’s various indexing issues in 2019

On 4th April, it was revealed that Google dropped pages inadvertently from its search index. Three days later, Google said the issue was fixed, only a day later to reveal it hadn’t been fully resolved. Finally, the indexing issue was resolved on 11th April. 

Then in May, Google’s indexing bug returned. If you tried to find new content from large news publishers, nothing was returned if the results were filtered to within the past hour. Even though it was fixed relatively quickly, the same issue occurred again in August.

Why is Google’s index so important?

Google’s index is where all the results you see in a search results page are stored. Therefore, if there’s an issue with the index, it will affect what Google can deliver to users. Google stores URLs in data centres all over the world and pushes updates to these data centres over time. 

In April, Google actually lost part of its index because of a deployment issue. As Courson explains:

“So, as we pushed some planned changes to the Search index, on April 5th parts of the deployment system broke, on a Friday no-less! More specifically: as we were updating the index over some of our data centres, a small number of documents ended up being dropped from the index accidentally. Hence: we lost part of the index.”

How Google intends to avoid these issues in the future

Google says that it has a strong “post-mortem” culture whenever anything breaks or goes wrong. It creates a document to debrief on the breakage and tries to avoid it happening next time. 

In the wake of these indexing issues, Google’s key decisions moving forward are to:

  1. Explore ways to more quickly share information within Search Console itself about widespread bugs, and have that information serve as the main point of reference for webmasters to check, in case they are suspecting outages.
  2. More promptly post to the Search Console data anomalies page, when relevant (if the disturbance is going to be seen over the long term in Search Console data).
  3. Continue tweeting as quickly as possible about such issues to quickly reassure webmasters that Google is aware the issue is on its end.

“We hope that this post will bring more clarity to how our systems are complex and can sometimes break, and will also help you understand how we communicate about these matters,” added Coulson. 

“It’s important to keep in mind that most website indexing issues are caused by an individual website’s configuration, which can create difficulties for Google Search to index that website properly.”