Register Members List Search Today's Posts Mark Forums Read

Reply
 
Mod Options
Have Googlebot Crawl EVERYTHING (theoretically) Details »
Have Googlebot Crawl EVERYTHING (theoretically)
Mod Version: 1.00, by filburt1 (Member) filburt1 is offline
Developer Last Online: May 2007 I like it Show Printable Version Email this Page

This modification is in the archives.
vB Version: 2.3.x Rating: (1 vote - 5.00 average) Installs: 107
Released: 25 Mar 2003 Last Update: Never Downloads: 78
Not Supported Is in Beta Stage  

Googlebot supposedly won't crawl links that have sessionhashes (32 character hex strings) in the URLs. So this hack detects when Googlebot is browsing your forum, and if it is, deletes the sessionhash. So, theoretically, Googlebot will be able to crawl every single guest-accessible link on your board. All of them. This means every thread, every forum, every post, even useless stuff like the memberlist and the FAQ.

Simple to install (1 step). This won't get instant gratification because Googlebot will only reindex your site on its next crawl.

Download Now

Only licensed members can download files, Click Here for more information.

Show Your Support

  • To receive notifications regarding updates -> Click to Mark as Installed.
  • This modification may not be copied, reproduced or published elsewhere without author's permission.
Comments
  #2  
Old 25 Mar 2003, 19:00
Intex Intex is offline
 
Join Date: Apr 2002
Another great hack filburt1. Not sure I need it crawling my internal site, but nevertheless very cool.
Reply With Quote
  #3  
Old 25 Mar 2003, 19:22
colicab-d's Avatar
colicab-d colicab-d is offline
 
Join Date: Dec 2002
Real name: Dave
lol so thats why u wanted the sessions code
Reply With Quote
  #4  
Old 25 Mar 2003, 19:38
filburt1 filburt1 is offline
 
Join Date: Feb 2002
No it's not...
Reply With Quote
  #5  
Old 25 Mar 2003, 19:54
VampireMan VampireMan is offline
 
Join Date: Jan 2003
Real name: chris
simple but very effective hack , thanks :-)
Reply With Quote
  #6  
Old 25 Mar 2003, 20:02
Xanthine Xanthine is offline
 
Join Date: Mar 2002
Sounds good thanks Filburt
Reply With Quote
  #7  
Old 25 Mar 2003, 20:35
Overgrow's Avatar
Overgrow Overgrow is offline
 
Join Date: Nov 2001
Excellent


One note: This may not allow Google to see all of your posts (theoretically). If you have old ones that would not naturally show up on forumdisplay, Google will not use the select box to see all of the old posts.

Elegantly simple though and makes perfect sense.


/me clicks install
__________________
Marijuana Growing Magazine - since vB 1.1
Reply With Quote
  #8  
Old 25 Mar 2003, 20:42
Davey Davey is offline
 
Join Date: Nov 2002
Real name: David Gane
Filburt how did you find that special part of code through all that 'mess'?
.
Thanks.
*Installs*.

Dave.
__________________
dgane.com - about me & my life

Last edited by Davey; 25 Mar 2003 at 20:45.
Reply With Quote
  #9  
Old 25 Mar 2003, 22:31
Webdork Webdork is offline
 
Join Date: Dec 2001
Real name: Angus
On my site I require registration for users to view threads. Is there a way to allow Google to become a "registered" user so it can crawl all threads still?
__________________
Australian eCommerce Community
Reply With Quote
  #10  
Old 25 Mar 2003, 22:32
filburt1 filburt1 is offline
 
Join Date: Feb 2002
Yes but smart users can get around it...

edit: actually I thought of a way to fix that, too. Expect that hack eventually.
note to self: check first few IP bits and HTTP_USER_AGENT, update $bbuserinfo.
Reply With Quote
  #11  
Old 25 Mar 2003, 22:35
corsacrazy's Avatar
corsacrazy corsacrazy is offline
 
Join Date: May 2002
Real name: wayne
installed nice one

off topic

how r u lot doing those funky sigs ? i beleive u started the trend filb !
Reply With Quote
  #12  
Old 25 Mar 2003, 22:37
Webdork Webdork is offline
 
Join Date: Dec 2001
Real name: Angus
I dont really mind if a smart user can get in without registering - my forums arent private I just like to know who is using them. But the true value is of course in having as many threads as possible in Google to attract new users.
__________________
Australian eCommerce Community
Reply With Quote
  #13  
Old 26 Mar 2003, 02:21
Velocd's Avatar
Velocd Velocd is offline
 
Join Date: Mar 2002
Real name: Mike
Nice hack, and I'll be awaiting that suggestion by Webdork in post #9 as well

/me clicks install
Reply With Quote
  #14  
Old 26 Mar 2003, 03:42
filburt1 filburt1 is offline
 
Join Date: Feb 2002
Forgot it, I can't stand looking at sessions.php after trying for an hour.
Reply With Quote
  #15  
Old 26 Mar 2003, 19:31
Slynderdale Slynderdale is offline
 
Join Date: Mar 2002
Real name: John
Hmm, theres a similar hack for this in Tech's vB Archive hack that also includes several searchbots, not just google.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Mod Options

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 14:41.

Layout Options | Width: Wide Color: