Friday, July 10, 2020

Fix "Indexed, though blocked by robots.txt"

What is "robots.txt" ?

Robots.txt is a security protocol file. This file tells search engine like Google which files to be crawled and which not. This files helps your site or blog from traffic overloading.

If you are a new blogger and Just started blogging. You will definitely get this message below from Google Services.


 "Indexed, though blocked by robots.txt"

But, there's nothing to be worried of, this is just a message saying that your URL has been indexed, but it is not allowing Googlebots (Google Robots) to crawl on it.

How to Fix "Indexed, though blocked by robots.txt"

This problem can be solved by updating your robots.txt file of your website. Kindly, follow the steps below to get rid of this problem.

1. Generate Your Robots.txt File :

If you don't have any idea how to generate robots.txt file

Visit the link below and generate it : 

Generate Robots.txt (XML sitemap) for your blog.

2. Open your Web-hosting site ( blogger, Wordpress, Hostinger or any other)

3. Go to settings, open "Search Preferences".

4. Click on robots.txt an Enable it.

5. Paste the generated Robots.txt (XML Sitemap) on the space given.

6. Add the below two lines into it after Disallow /Search

                                                Disallow: /category/
                     Disallow: /tag/ 

7. Click on save

9. Click on "Custom robots header tags" and Enable it.

10. Tick the option which i had done below.


Custom robots header tags

11. Go back to your Google Email or Message click "See Coverage"

12. Click "Start Validation"

This is the Way you can solve or Fix "Indexed, though blocked by robots.txt" Problem.

Also Check Out - How to make a professional blog or site and earn through it.

Also Read - Best ways to earn money online easily.

Share if you like my work, and feel free to comment if any error occurs or you find any infringes or copyright claims and want it to be removed. Feel free to comment.

No comments:

Post a Comment

Translate