vBulletin Mods

The Official vBulletin Modifications Site
https://www.vbulletin.org/forum/showthread.php?t=320307

About Robotx txt
by tanzeelniazi
21 Sep 2015 01:34

Hello all
I want to know why robots.txt file is necessary ?
Is this for SEO ? or use in security ?
When i open my robots.txt file i see this

Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

i dont know what is this and dont know what before add :(
When i search vbulletin robots.txt i see this


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

i want to add good things in my robots.txt file but i dont know what is the uses
Please tell me in simple words or make a robots file for me
thanks

--------------- Added 21 Sep 2015 at 01:39 ---------------

Before i have good guests and bots like alexa bot google bot many other and before i have good alexa rank and traffic now bad
please some one make a good robots file for me pplease

Max Taxable 21 Sep 2015 02:30

It's not "necessary" and also does nothing at all for security. It's just a signpost that says "please bots don't index any thing on this list" and few search engines pay any mind to it.

It's not a blocker, it's not a security measure, it's just a .txt file that "noble" search engine spiders do check and try to follow the guide of it.

RichieBoy67 21 Sep 2015 02:35

It is always good to use it though many seedy search engines ignore it. The main searches though like Google and Bing honor it. Those are the ones you care about really. Others you can use htaccess to block if you are concerned with them.

You can use it to block pages you do not want indexed in the Google, etc. For example if you have a really spammy advertising section or something. You also add your sitemap there to keep google crawling it though you should have your sitemap submitted using webmaster tools anyways.

On my sites I use it to keep the focus on my topic content. SEO is about quality, not quantity so I do not index my off topic forums.

Also, as done in yours you can use robots.txt to put a delay on search bots if your resources are limited.

As for Attractaca.. I would not waste time using that.

tanzeelniazi 21 Sep 2015 11:20

which sitemap is good for me ?? how to find more sitemap in my site ?
if i upload another sitemap any loss in my site ? or decreasing rank ?

Seven Skins 21 Sep 2015 11:30

Sitemaps don't usually have any affect on site ranking ... If you want to add sitemap to your site with vbulletin then add something like this to robots.txt.


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.



All times are GMT. The time now is 03:28.

Powered by vBulletin® Version 3.8.14
Copyright © 2020, MH Sub I, LLC dba vBulletin. All Rights Reserved. vBulletin® is a registered trademark of MH Sub I, LLC
Copyright ©2001 - , vbulletin.org. All rights reserved.