How to Fix Common Indexing Errors to Improve Website Traffic
April 23, 2021 | By:

How to Fix Common Indexing Errors to Improve Website Traffic

If you’ve noticed that your SEO rankings have dropped or organic traffic is decreasing, one of the most common reasons are indexing errors.

For your content to show up on Search Engine Results Pages (SERPs), search engines like Google need to be able to index and crawl—or in simpler terms, find and save—your content in the first place.

The more pages of your website are indexed, the better are your chances of showing up in SERPs, which subsequently drives organic traffic and boosts rankings. However, there can be instances where search engine bots are prevented from indexing your content.

How to Check for Indexing Errors

The easiest way to check for errors is to use Google Search Console’s Index Coverage Report:

  • Log in to Google Search Console and select your primary domain (website)
  • Under Index, select Coverage → Index Coverage Report

The report covers 4 categories:

  • Valid are those that have already been indexed
  • Valid with warnings are those that have been indexed but have issues you need to fix
  • Excluded are pages that weren’t indexed because search engine bots received signals that they shouldn’t be indexed (e.g., tagged as “noindex,” alternative page with proper canonical tag, etc.)
  • Error are pages that could not be indexed. We will focus on these indexing errors in this article.

5 Common Indexing Errors and How to Fix Them

Here are some common indexing errors you should be aware of and how you can fix them:

1. Redirect error

This means that indexing isn’t possible because the page redirects to another or a non-existing page. Google may have also encountered redirect loops and redirect chains that are too long.

Since there can be more than one reason, the best way to fix this is to use the URL Inspection Tool:

  • Input your webpage’s URL
  • Get more details about the error
  • Select Test Live URL
  • After fixing the error, select Request Indexing, then Validate Fix

It may also be helpful to use other tools to check for redirect loops, redirect chains, and HTTPS redirects, such as:

  • Lighthouse
  • Redirect Checker Tool
  • Redirect Path Chrome Extension
  • SSL Server Test
2. Submitted URL seems to be a Soft 404

In simpler terms, this may mean that your submitted page for indexing has little to no (empty) content. Technically, it means you submitted the URLs through an XML sitemap, but the URLs may be returning a HTTP status code 200 but is actually a 404 page.

As with the previous tip, it’s best to use the URL Inspection Tool to investigate further. In addition, you can do either of the following Soft 404 fixes:

  • If the page is no longer available, create a custom 404 (not found) page or resolve the 410 (gone) error by flushing your DNS.
  • If the page in question has been moved, create a 301 (permanent redirect) to the new page URL.
3. Submitted URL returns unauthorized request (401)

This means that the page is included in your sitemap, but Google isn’t authorized to index the page (401 HTTP response), usually because the page is password protected (not publicly available). To fix this, you need to:

  • Remove the password-protected page from the sitemap
  • Add a “noindex” directive in the page’s header
  • Block the protected areas in your robots.txt file
4. Server error (5xx)

This means that Google couldn’t access your URL, your website was busy, or the request timed out. To confirm this, check the host availability in the Crawl Stats Report from the URL Inspection Tool. Server errors can be fixed by:

  • Contacting your website’s hosting server to check whether the server is down, misconfigured, or overloaded.
  • Reducing excessive page loading for dynamic page requests.
  • Check for DNS configuration issues
5. Submitted URL blocked due to other 4xx issue

This means that Googlebots couldn’t access your submitted URLs because they received an unspecified 4xx response code—the error is other than 401, 403, and 404 that were previously mentioned and can range from 400 to 451.

Use the URL Inspection Tool to find out the specific problem. There are several ways to fix 4xx errors, and the most common ones are:

  • Clearing your browser’s cookies and cache
  • Checking for spelling errors in the URL
  • Simply refreshing the page
  • Not submitting/excluding the page for indexing

Improving your website traffic with a solid SEO strategy

Driving organic traffic is just one of the many facets involved in search engine optimization (SEO). It takes a comprehensive approach and a solid SEO strategy to ensure that all your digital marketing efforts are reaping the results you want. Ilfusion has the experience, the right tools, and the best SEO professionals to help you out and do just that.

Give us a call at 888-420-5115, or send us an email at [email protected] to get started!

Tags: , , ,

Categorized in:

Featured Articles

How To Start an SEO Campaign in 6 Steps
February 1, 2018 | By: Ilfusion Team

How To Start an SEO Campaign in 6 Steps

Starting a Search Engine Optimization (SEO) campaign can be tricky for any digital marketer. How do you begin? What is the most important element to implement? How do you know if it’s working? Don’t be overwhelmed, though. Starting an SEO... view more

5 signs you need to hire a marketing agency
September 26, 2017 | By: Tinny

5 Signs You Need to Hire a Marketing Agency

Marketing is a crucial part of your business strategy. It’s what keeps the gears turning and the driving force of your company’s sales. For small businesses, marketing is important in getting the word out there that your company exists and... view more

Key Advantages of Video Marketing for Your Growing Business
September 21, 2017 | By: Tinny

Key Advantages of Video Marketing for Your Growing Business

There has been a rise in the use of mobile devices to access the internet, and there are no signs of it slowing down anytime soon. In this era where most people thrive on living on the go, video marketing... view more

5 things you need to know image optimization for seo
September 13, 2017 | By: Tinny

5 Things You Need to Know: Image Optimization for SEO

If you’re a company whose content strategy relies on images, or if your business blog is image-heavy, it’s high time for you to think about optimizing your images for SEO — if you haven’t done so already. While there isn’t... view more

Tips on How to Choose the Best Keywords for Your Website
August 2, 2017 | By: Tinny

Tips on How to Choose the Best Keywords for Your Website

Not sure on what keywords to use? Read more on how to search for the best keywords for your webpage. When you have an SEO or PPC campaign, it all starts with one basic element: keyword research. Keywords are the... view more

Top 3 Search Ranking Factors You Should Know About
July 26, 2017 | By: Tinny

Top 3 Search Ranking Factors You Should Know About

Read more on what it takes to improve your search engine ranking. Landing on the first page of Google’s search engine results is one of the ultimate marketing goals of many website owners. While Google is keeping mum on what... view more

The Importance of Social Media in SEO
June 28, 2017 | By: Tinny

The Importance of Social Media in SEO

Here’s why you need social media for your SEO. Nowadays, social media has inarguably become an indispensable tool in any marketing strategy. SEO, of course, is also one of those tools that have evolved with the many trends and changes... view more

5 ways to dominate google search results
April 27, 2017 | By: Tinny

5 Ways to Dominate Google’s Search Results in 2017

Here’s how to place your website at the top of search engine results. Creating a website that doesn’t rank well in Google’s search engine results page (SERP) doesn’t translate well to your overall business effort. In recent years, Google has... view more

above the fold
July 2, 2015 | By: The Conductor

Let’s talk about “The Fold”

We hear it a lot; client feedback that is focused on what content is and is not “above the fold”. To be perfectly honest, this old newspaper term can be a cause of frustration when it comes to good user... view more

June 3, 2015 | By: The Doctor

The Turbit

This crazy ol’ world of ours is spinning faster and faster everyday. Not literally, as we would all be slung to our spacey-doom. I mean figuratively we are all moving more quickly and filling every moment with some bit of... view more

welcome to the future
May 6, 2015 | By: Ilfusion Team

How Your Website Can Survive Mobilegeddon

Think your website is mobile friendly? Think again. Google’s new functionality changes have created a frantic need for websites to be optimized for mobile. Mobilegeddon is upon us, and your website could be at risk. See infographic below.

want a ballpark
April 23, 2015 | By: El Jefe

Website Cost: A Day at the Ballpark

A question that I get on a regular basis (read: every day) is, “How much will my new website cost?”. To which I typically say something like, “Well, that varies based on the amount of work that’s required to complete... view more

April 7, 2015 | By: The Inventor

Are you a Webspammer?

SEO practices are very different today than how they were even just a few years ago. The number one key to ranking well in search engines today is to create unique quality content. First post it on your website or... view more

social media happenings
March 25, 2015 | By: Ilfusion Team

Social Media Happenings

Ilfusion loves to bring you news on the newest social media functions and trends. Recently, there have been three major happenings in the social media world. Meerkat, a new app that has risen to fame due to the 2015 music... view more

big picture
March 12, 2015 | By: The Conductor

The Big Picture

Whether you’re laser-focused on a specific project, or taking a comprehensive approach to your marketing efforts, one aspect doesn’t change: always look at the big picture. Companies often shop at least a few agencies when they’re considering engagement with an... view more